2024-09-12T12:44:36.915 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-12T12:44:36.931 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-12T12:44:37.023 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-09-11_21:48:03-krbd-squid-testing-default-smithi/7901765 branch: squid description: krbd/mirror/{bluestore-bitmap clusters/2-node conf install/ceph ms_mode$/{legacy-rxbounce} tasks/compare-mirror-images} email: ceph-qa@ceph.com first_in_suite: false job_id: '7901765' kernel: flavor: default kdb: 1 sha1: 2ca44a2f35ca8f36983aaaa46d49312239ef519a last_in_suite: false machine_type: smithi meta: - desc: 2 ceph clusters with 1 mon, 1 mgr and 3 osd each name: teuthology-2024-09-11_21:48:03-krbd-squid-testing-default-smithi no_nested_subset: false os_type: centos os_version: 9.stream overrides: admin_socket: branch: squid ceph: conf: client: rbd default map options: ms_mode=legacy,rxbounce global: mon warn on pool no app: false ms die on skipped message: false mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) sha1: 086c737fa75c22d521a3932c59b31c5e1d1c9ef4 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 086c737fa75c22d521a3932c59b31c5e1d1c9ef4 extra_system_packages: - pv thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: squid sha1: 086c737fa75c22d521a3932c59b31c5e1d1c9ef4 owner: scheduled_teuthology@teuthology priority: 100 repo: https://git.ceph.com/ceph.git roles: - - cluster1.mon.a - cluster1.mgr.x - cluster1.osd.0 - cluster1.osd.1 - cluster1.osd.2 - - cluster2.mon.a - cluster2.mgr.x - cluster2.osd.0 - cluster2.osd.1 - cluster2.osd.2 - cluster1.client.mirror - cluster1.client.mirror.0 - cluster2.client.mirror - cluster2.client.mirror.0 seed: 6642 sha1: 086c737fa75c22d521a3932c59b31c5e1d1c9ef4 sleep_before_teardown: 0 subset: 2/4 suite: krbd suite_branch: squid suite_path: /home/teuthworker/src/git.ceph.com_ceph_086c737fa75c22d521a3932c59b31c5e1d1c9ef4/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: 086c737fa75c22d521a3932c59b31c5e1d1c9ef4 targets: smithi060.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIFOvl+hCSxBzLOW+W0g8RdwY9HvLI1Bc6w8kIMW64qum4rkb3wrve2G3MNnVVKdcGrJpzRuajn03zqmcHad4HA= smithi130.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAP6S2zoN+SHzbLXCy7LSlxt94/nNlTvcxp6VIom6mCS2EdmiV6WkzCK7pkJ6lvDtvlYwqfk7CacbxjINJQq9RU= tasks: - install: extra_packages: - rbd-mirror - ceph: cluster: cluster1 - ceph: cluster: cluster2 - rbd-mirror: client: cluster1.client.mirror.0 thrash: false - rbd-mirror: client: cluster2.client.mirror.0 thrash: false - workunit: clients: cluster1.client.mirror: - rbd/compare_mirror_images.sh env: RBD_DEVICE_TYPE: krbd RBD_MIRROR_USE_RBD_MIRROR: '1' timeout: 3h teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-11_21:48:03 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-12T12:44:37.023 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_086c737fa75c22d521a3932c59b31c5e1d1c9ef4/qa; will attempt to use it 2024-09-12T12:44:37.024 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_086c737fa75c22d521a3932c59b31c5e1d1c9ef4/qa/tasks 2024-09-12T12:44:37.024 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-12T12:44:37.025 INFO:teuthology.task.internal:Checking packages... 2024-09-12T12:44:37.046 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '086c737fa75c22d521a3932c59b31c5e1d1c9ef4' 2024-09-12T12:44:37.046 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-12T12:44:37.046 INFO:teuthology.packaging:ref: None 2024-09-12T12:44:37.046 INFO:teuthology.packaging:tag: None 2024-09-12T12:44:37.046 INFO:teuthology.packaging:branch: squid 2024-09-12T12:44:37.046 INFO:teuthology.packaging:sha1: 086c737fa75c22d521a3932c59b31c5e1d1c9ef4 2024-09-12T12:44:37.046 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=squid 2024-09-12T12:44:37.257 INFO:teuthology.task.internal:Found packages for ceph version 19.1.1-314.g086c737f 2024-09-12T12:44:37.259 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-12T12:44:37.266 INFO:teuthology.task.internal:no buildpackages task found 2024-09-12T12:44:37.266 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-12T12:44:37.283 INFO:teuthology.task.internal:Saving configuration 2024-09-12T12:44:37.295 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-12T12:44:37.302 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-12T12:44:37.334 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi060.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-11_21:48:03-krbd-squid-testing-default-smithi/7901765', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-12 12:39:45.659299', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIFOvl+hCSxBzLOW+W0g8RdwY9HvLI1Bc6w8kIMW64qum4rkb3wrve2G3MNnVVKdcGrJpzRuajn03zqmcHad4HA='} 2024-09-12T12:44:37.361 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi130.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-11_21:48:03-krbd-squid-testing-default-smithi/7901765', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-12 12:39:45.657868', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAP6S2zoN+SHzbLXCy7LSlxt94/nNlTvcxp6VIom6mCS2EdmiV6WkzCK7pkJ6lvDtvlYwqfk7CacbxjINJQq9RU='} 2024-09-12T12:44:37.361 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-12T12:44:37.368 INFO:teuthology.task.internal:roles: ubuntu@smithi060.front.sepia.ceph.com - ['cluster1.mon.a', 'cluster1.mgr.x', 'cluster1.osd.0', 'cluster1.osd.1', 'cluster1.osd.2'] 2024-09-12T12:44:37.368 INFO:teuthology.task.internal:roles: ubuntu@smithi130.front.sepia.ceph.com - ['cluster2.mon.a', 'cluster2.mgr.x', 'cluster2.osd.0', 'cluster2.osd.1', 'cluster2.osd.2', 'cluster1.client.mirror', 'cluster1.client.mirror.0', 'cluster2.client.mirror', 'cluster2.client.mirror.0'] 2024-09-12T12:44:37.368 INFO:teuthology.run_tasks:Running task console_log... 2024-09-12T12:44:37.447 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fb278197eb0>, signals=[15]) 2024-09-12T12:44:37.447 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-12T12:44:37.454 INFO:teuthology.task.internal:Opening connections... 2024-09-12T12:44:37.454 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi060.front.sepia.ceph.com 2024-09-12T12:44:37.455 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi060.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-12T12:44:37.540 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi130.front.sepia.ceph.com 2024-09-12T12:44:37.541 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi130.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-12T12:44:37.621 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-12T12:44:37.628 DEBUG:teuthology.orchestra.run.smithi060:> uname -m 2024-09-12T12:44:37.646 INFO:teuthology.orchestra.run.smithi060.stdout:x86_64 2024-09-12T12:44:37.646 DEBUG:teuthology.orchestra.run.smithi060:> cat /etc/os-release 2024-09-12T12:44:37.703 INFO:teuthology.orchestra.run.smithi060.stdout:NAME="CentOS Stream" 2024-09-12T12:44:37.703 INFO:teuthology.orchestra.run.smithi060.stdout:VERSION="9" 2024-09-12T12:44:37.703 INFO:teuthology.orchestra.run.smithi060.stdout:ID="centos" 2024-09-12T12:44:37.703 INFO:teuthology.orchestra.run.smithi060.stdout:ID_LIKE="rhel fedora" 2024-09-12T12:44:37.704 INFO:teuthology.orchestra.run.smithi060.stdout:VERSION_ID="9" 2024-09-12T12:44:37.704 INFO:teuthology.orchestra.run.smithi060.stdout:PLATFORM_ID="platform:el9" 2024-09-12T12:44:37.704 INFO:teuthology.orchestra.run.smithi060.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-12T12:44:37.704 INFO:teuthology.orchestra.run.smithi060.stdout:ANSI_COLOR="0;31" 2024-09-12T12:44:37.704 INFO:teuthology.orchestra.run.smithi060.stdout:LOGO="fedora-logo-icon" 2024-09-12T12:44:37.704 INFO:teuthology.orchestra.run.smithi060.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-12T12:44:37.704 INFO:teuthology.orchestra.run.smithi060.stdout:HOME_URL="https://centos.org/" 2024-09-12T12:44:37.704 INFO:teuthology.orchestra.run.smithi060.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-12T12:44:37.704 INFO:teuthology.orchestra.run.smithi060.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-12T12:44:37.704 INFO:teuthology.orchestra.run.smithi060.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-12T12:44:37.705 INFO:teuthology.lock.ops:Updating smithi060.front.sepia.ceph.com on lock server 2024-09-12T12:44:37.733 DEBUG:teuthology.orchestra.run.smithi130:> uname -m 2024-09-12T12:44:37.752 INFO:teuthology.orchestra.run.smithi130.stdout:x86_64 2024-09-12T12:44:37.752 DEBUG:teuthology.orchestra.run.smithi130:> cat /etc/os-release 2024-09-12T12:44:37.811 INFO:teuthology.orchestra.run.smithi130.stdout:NAME="CentOS Stream" 2024-09-12T12:44:37.811 INFO:teuthology.orchestra.run.smithi130.stdout:VERSION="9" 2024-09-12T12:44:37.811 INFO:teuthology.orchestra.run.smithi130.stdout:ID="centos" 2024-09-12T12:44:37.811 INFO:teuthology.orchestra.run.smithi130.stdout:ID_LIKE="rhel fedora" 2024-09-12T12:44:37.811 INFO:teuthology.orchestra.run.smithi130.stdout:VERSION_ID="9" 2024-09-12T12:44:37.811 INFO:teuthology.orchestra.run.smithi130.stdout:PLATFORM_ID="platform:el9" 2024-09-12T12:44:37.811 INFO:teuthology.orchestra.run.smithi130.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-12T12:44:37.812 INFO:teuthology.orchestra.run.smithi130.stdout:ANSI_COLOR="0;31" 2024-09-12T12:44:37.812 INFO:teuthology.orchestra.run.smithi130.stdout:LOGO="fedora-logo-icon" 2024-09-12T12:44:37.812 INFO:teuthology.orchestra.run.smithi130.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-12T12:44:37.812 INFO:teuthology.orchestra.run.smithi130.stdout:HOME_URL="https://centos.org/" 2024-09-12T12:44:37.812 INFO:teuthology.orchestra.run.smithi130.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-12T12:44:37.812 INFO:teuthology.orchestra.run.smithi130.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-12T12:44:37.812 INFO:teuthology.orchestra.run.smithi130.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-12T12:44:37.812 INFO:teuthology.lock.ops:Updating smithi130.front.sepia.ceph.com on lock server 2024-09-12T12:44:37.841 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-12T12:44:37.850 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-12T12:44:37.855 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-12T12:44:37.856 DEBUG:teuthology.orchestra.run.smithi060:> test '!' -e /home/ubuntu/cephtest 2024-09-12T12:44:37.858 DEBUG:teuthology.orchestra.run.smithi130:> test '!' -e /home/ubuntu/cephtest 2024-09-12T12:44:37.875 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-12T12:44:37.883 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-12T12:44:37.883 DEBUG:teuthology.orchestra.run.smithi060:> test -z $(ls -A /var/lib/ceph) 2024-09-12T12:44:37.915 DEBUG:teuthology.orchestra.run.smithi130:> test -z $(ls -A /var/lib/ceph) 2024-09-12T12:44:37.943 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-12T12:44:38.013 INFO:teuthology.run_tasks:Running task kernel... 2024-09-12T12:44:38.027 INFO:teuthology.task.kernel:normalize config orig: {'flavor': 'default', 'kdb': 1, 'sha1': '2ca44a2f35ca8f36983aaaa46d49312239ef519a'} 2024-09-12T12:44:38.027 INFO:teuthology.task.kernel:config {'cluster1.mon.a': {'flavor': 'default', 'kdb': 1, 'sha1': '2ca44a2f35ca8f36983aaaa46d49312239ef519a'}, 'cluster2.mon.a': {'flavor': 'default', 'kdb': 1, 'sha1': '2ca44a2f35ca8f36983aaaa46d49312239ef519a'}}, timeout 300 2024-09-12T12:44:38.028 DEBUG:teuthology.orchestra.run.smithi060:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-12T12:44:38.028 DEBUG:teuthology.orchestra.run.smithi130:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-12T12:44:38.045 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T12:44:38.045 DEBUG:teuthology.task.kernel:sha1 for cluster1.mon.a is 2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-12T12:44:38.046 INFO:teuthology.task.kernel:Checking kernel version of cluster1.mon.a, want "2ca44a2f35ca8f36983aaaa46d49312239ef519a"... 2024-09-12T12:44:38.046 DEBUG:teuthology.orchestra.run.smithi060:> uname -r 2024-09-12T12:44:38.046 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T12:44:38.047 DEBUG:teuthology.task.kernel:sha1 for cluster2.mon.a is 2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-12T12:44:38.047 INFO:teuthology.task.kernel:Checking kernel version of cluster2.mon.a, want "2ca44a2f35ca8f36983aaaa46d49312239ef519a"... 2024-09-12T12:44:38.047 DEBUG:teuthology.orchestra.run.smithi130:> uname -r 2024-09-12T12:44:38.104 INFO:teuthology.orchestra.run.smithi060.stdout:5.14.0-503.el9.x86_64 2024-09-12T12:44:38.104 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-503.el9.x86_64 vs 2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-12T12:44:38.105 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-09-12T12:44:38.105 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-12T12:44:38.116 INFO:teuthology.orchestra.run.smithi130.stdout:5.14.0-503.el9.x86_64 2024-09-12T12:44:38.117 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-503.el9.x86_64 vs 2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-12T12:44:38.117 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-09-12T12:44:38.117 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-12T12:44:38.271 DEBUG:teuthology.orchestra.run.smithi130:> sudo yum upgrade -y linux-firmware 2024-09-12T12:44:38.278 DEBUG:teuthology.orchestra.run.smithi060:> sudo yum upgrade -y linux-firmware 2024-09-12T12:44:40.549 INFO:teuthology.orchestra.run.smithi060.stdout:CentOS Stream 9 - BaseOS 16 MB/s | 8.2 MB 00:00 2024-09-12T12:44:41.405 INFO:teuthology.orchestra.run.smithi130.stdout:CentOS Stream 9 - BaseOS 6.0 MB/s | 8.2 MB 00:01 2024-09-12T12:44:42.500 INFO:teuthology.orchestra.run.smithi060.stdout:CentOS Stream 9 - AppStream 26 MB/s | 20 MB 00:00 2024-09-12T12:44:44.565 INFO:teuthology.orchestra.run.smithi130.stdout:CentOS Stream 9 - AppStream 10 MB/s | 20 MB 00:02 2024-09-12T12:44:46.926 INFO:teuthology.orchestra.run.smithi060.stdout:CentOS Stream 9 - CRB 13 MB/s | 6.5 MB 00:00 2024-09-12T12:44:48.618 INFO:teuthology.orchestra.run.smithi060.stdout:CentOS Stream 9 - Extras packages 43 kB/s | 19 kB 00:00 2024-09-12T12:44:48.940 INFO:teuthology.orchestra.run.smithi130.stdout:CentOS Stream 9 - CRB 11 MB/s | 6.5 MB 00:00 2024-09-12T12:44:49.381 INFO:teuthology.orchestra.run.smithi060.stdout:Extra Packages for Enterprise Linux 36 MB/s | 23 MB 00:00 2024-09-12T12:44:50.574 INFO:teuthology.orchestra.run.smithi130.stdout:CentOS Stream 9 - Extras packages 42 kB/s | 19 kB 00:00 2024-09-12T12:44:51.503 INFO:teuthology.orchestra.run.smithi130.stdout:Extra Packages for Enterprise Linux 28 MB/s | 23 MB 00:00 2024-09-12T12:44:54.987 INFO:teuthology.orchestra.run.smithi060.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-09-12T12:44:56.901 INFO:teuthology.orchestra.run.smithi130.stdout:lab-extras 34 kB/s | 1.7 kB 00:00 2024-09-12T12:44:57.147 INFO:teuthology.orchestra.run.smithi060.stdout:Dependencies resolved. 2024-09-12T12:44:57.148 INFO:teuthology.orchestra.run.smithi060.stdout:Nothing to do. 2024-09-12T12:44:57.149 INFO:teuthology.orchestra.run.smithi060.stdout:Complete! 2024-09-12T12:44:57.267 INFO:teuthology.task.kernel:Downloading kernel 2ca44a2f35ca8f36983aaaa46d49312239ef519a on cluster1.mon.a... 2024-09-12T12:44:57.268 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-12T12:44:57.419 INFO:teuthology.task.kernel:fetching, builder baseurl is https://3.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/centos/9/flavors/default/x86_64/ 2024-09-12T12:44:57.420 DEBUG:teuthology.orchestra.run.smithi060:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.11.0_rc5_g2ca44a2f35ca-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://3.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/centos/9/flavors/default/x86_64/ --input-file=- 2024-09-12T12:44:57.422 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi060.front.sepia.ceph.com to complete... 2024-09-12T12:44:58.997 INFO:teuthology.orchestra.run.smithi130.stdout:Dependencies resolved. 2024-09-12T12:44:58.998 INFO:teuthology.orchestra.run.smithi130.stdout:Nothing to do. 2024-09-12T12:44:58.998 INFO:teuthology.orchestra.run.smithi130.stdout:Complete! 2024-09-12T12:44:59.103 INFO:teuthology.task.kernel:Downloading kernel 2ca44a2f35ca8f36983aaaa46d49312239ef519a on cluster2.mon.a... 2024-09-12T12:44:59.104 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-12T12:44:59.253 INFO:teuthology.task.kernel:fetching, builder baseurl is https://3.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/centos/9/flavors/default/x86_64/ 2024-09-12T12:44:59.253 DEBUG:teuthology.orchestra.run.smithi130:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.11.0_rc5_g2ca44a2f35ca-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://3.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/centos/9/flavors/default/x86_64/ --input-file=- 2024-09-12T12:44:59.256 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi130.front.sepia.ceph.com to complete... 2024-09-12T12:45:07.615 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-12 12:45:07 URL:https://3.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/centos/9/flavors/default/x86_64/kernel-6.11.0_rc5_g2ca44a2f35ca-1.x86_64.rpm [359633503/359633503] -> "/tmp/kernel.x86_64.rpm" [1] 2024-09-12T12:45:07.615 INFO:teuthology.orchestra.run.smithi060.stderr:FINISHED --2024-09-12 12:45:07-- 2024-09-12T12:45:07.615 INFO:teuthology.orchestra.run.smithi060.stderr:Total wall clock time: 10s 2024-09-12T12:45:07.615 INFO:teuthology.orchestra.run.smithi060.stderr:Downloaded: 1 files, 343M in 9.9s (34.7 MB/s) 2024-09-12T12:45:07.618 INFO:teuthology.task.kernel:Installing kernel 2ca44a2f35ca8f36983aaaa46d49312239ef519a on cluster1.mon.a... 2024-09-12T12:45:07.618 DEBUG:teuthology.orchestra.run.smithi060:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-09-12T12:45:09.841 INFO:teuthology.orchestra.run.smithi060.stdout:Verifying... ######################################## 2024-09-12T12:45:10.021 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing... ######################################## 2024-09-12T12:45:10.127 INFO:teuthology.orchestra.run.smithi060.stdout:Updating / installing... 2024-09-12T12:45:18.436 INFO:teuthology.orchestra.run.smithi060.stdout:kernel-6.11.0_rc5_g2ca44a2f35ca-1 ######################################## 2024-09-12T12:45:19.477 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-12 12:45:19 URL:https://3.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/centos/9/flavors/default/x86_64/kernel-6.11.0_rc5_g2ca44a2f35ca-1.x86_64.rpm [359633503/359633503] -> "/tmp/kernel.x86_64.rpm" [1] 2024-09-12T12:45:19.477 INFO:teuthology.orchestra.run.smithi130.stderr:FINISHED --2024-09-12 12:45:19-- 2024-09-12T12:45:19.478 INFO:teuthology.orchestra.run.smithi130.stderr:Total wall clock time: 20s 2024-09-12T12:45:19.478 INFO:teuthology.orchestra.run.smithi130.stderr:Downloaded: 1 files, 343M in 20s (17.3 MB/s) 2024-09-12T12:45:19.481 INFO:teuthology.task.kernel:Installing kernel 2ca44a2f35ca8f36983aaaa46d49312239ef519a on cluster2.mon.a... 2024-09-12T12:45:19.481 DEBUG:teuthology.orchestra.run.smithi130:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-09-12T12:45:21.587 INFO:teuthology.orchestra.run.smithi130.stdout:Verifying... ######################################## 2024-09-12T12:45:21.797 INFO:teuthology.orchestra.run.smithi130.stdout:Preparing... ######################################## 2024-09-12T12:45:21.913 INFO:teuthology.orchestra.run.smithi130.stdout:Updating / installing... 2024-09-12T12:45:29.927 INFO:teuthology.orchestra.run.smithi130.stdout:kernel-6.11.0_rc5_g2ca44a2f35ca-1 ######################################## 2024-09-12T12:45:58.547 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi060.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-09-12T12:45:58.548 DEBUG:teuthology.orchestra.run.smithi060:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-09-12T12:45:58.630 INFO:teuthology.orchestra.run.smithi060.stdout:/boot/System.map-6.11.0-rc5-g2ca44a2f35ca 2024-09-12T12:45:58.630 INFO:teuthology.orchestra.run.smithi060.stdout:/boot/config-6.11.0-rc5-g2ca44a2f35ca 2024-09-12T12:45:58.630 INFO:teuthology.orchestra.run.smithi060.stdout:/boot/vmlinuz-6.11.0-rc5-g2ca44a2f35ca 2024-09-12T12:45:58.630 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca 2024-09-12T12:45:58.630 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/System.map 2024-09-12T12:45:58.630 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/config 2024-09-12T12:45:58.631 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel 2024-09-12T12:45:58.631 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch 2024-09-12T12:45:58.631 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86 2024-09-12T12:45:58.631 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto 2024-09-12T12:45:58.631 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-09-12T12:45:58.631 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-09-12T12:45:58.631 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-09-12T12:45:58.631 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-09-12T12:45:58.631 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-09-12T12:45:58.631 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-09-12T12:45:58.631 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-09-12T12:45:58.632 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/crc32c-intel.ko 2024-09-12T12:45:58.632 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-09-12T12:45:58.632 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-09-12T12:45:58.632 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-09-12T12:45:58.632 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-09-12T12:45:58.632 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-avx2.ko 2024-09-12T12:45:58.632 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-09-12T12:45:58.632 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-09-12T12:45:58.632 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-09-12T12:45:58.632 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-09-12T12:45:58.632 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-09-12T12:45:58.632 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-09-12T12:45:58.633 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-09-12T12:45:58.633 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel 2024-09-12T12:45:58.633 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu 2024-09-12T12:45:58.633 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu/mce 2024-09-12T12:45:58.633 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-09-12T12:45:58.633 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm 2024-09-12T12:45:58.633 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm-amd.ko 2024-09-12T12:45:58.633 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm-intel.ko 2024-09-12T12:45:58.633 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm.ko 2024-09-12T12:45:58.633 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto 2024-09-12T12:45:58.633 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ansi_cprng.ko 2024-09-12T12:45:58.633 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/anubis.ko 2024-09-12T12:45:58.633 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/arc4.ko 2024-09-12T12:45:58.634 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx 2024-09-12T12:45:58.634 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_memcpy.ko 2024-09-12T12:45:58.634 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_pq.ko 2024-09-12T12:45:58.634 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_raid6_recov.ko 2024-09-12T12:45:58.634 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_tx.ko 2024-09-12T12:45:58.634 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_xor.ko 2024-09-12T12:45:58.634 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/raid6test.ko 2024-09-12T12:45:58.634 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/authenc.ko 2024-09-12T12:45:58.634 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/authencesn.ko 2024-09-12T12:45:58.634 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blake2b_generic.ko 2024-09-12T12:45:58.634 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blowfish_common.ko 2024-09-12T12:45:58.634 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blowfish_generic.ko 2024-09-12T12:45:58.635 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/camellia_generic.ko 2024-09-12T12:45:58.635 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast5_generic.ko 2024-09-12T12:45:58.635 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast6_generic.ko 2024-09-12T12:45:58.635 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast_common.ko 2024-09-12T12:45:58.635 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ccm.ko 2024-09-12T12:45:58.635 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cmac.ko 2024-09-12T12:45:58.635 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crc32_generic.ko 2024-09-12T12:45:58.635 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crypto_engine.ko 2024-09-12T12:45:58.635 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crypto_user.ko 2024-09-12T12:45:58.635 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cts.ko 2024-09-12T12:45:58.635 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/des_generic.ko 2024-09-12T12:45:58.635 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/dh_generic.ko 2024-09-12T12:45:58.635 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ecc.ko 2024-09-12T12:45:58.636 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ecdh_generic.ko 2024-09-12T12:45:58.636 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/echainiv.ko 2024-09-12T12:45:58.636 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/essiv.ko 2024-09-12T12:45:58.636 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/fcrypt.ko 2024-09-12T12:45:58.636 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/khazad.ko 2024-09-12T12:45:58.636 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/lz4.ko 2024-09-12T12:45:58.636 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/lz4hc.ko 2024-09-12T12:45:58.636 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/md4.ko 2024-09-12T12:45:58.636 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/michael_mic.ko 2024-09-12T12:45:58.636 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/pcbc.ko 2024-09-12T12:45:58.636 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/pcrypt.ko 2024-09-12T12:45:58.636 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/rmd160.ko 2024-09-12T12:45:58.636 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/seed.ko 2024-09-12T12:45:58.637 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/serpent_generic.ko 2024-09-12T12:45:58.637 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/tcrypt.ko 2024-09-12T12:45:58.637 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/tea.ko 2024-09-12T12:45:58.637 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/twofish_common.ko 2024-09-12T12:45:58.637 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/twofish_generic.ko 2024-09-12T12:45:58.637 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/vmac.ko 2024-09-12T12:45:58.637 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/wp512.ko 2024-09-12T12:45:58.637 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xcbc.ko 2024-09-12T12:45:58.637 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xor.ko 2024-09-12T12:45:58.637 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xxhash_generic.ko 2024-09-12T12:45:58.637 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers 2024-09-12T12:45:58.637 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi 2024-09-12T12:45:58.637 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/acpi_ipmi.ko 2024-09-12T12:45:58.638 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/acpi_pad.ko 2024-09-12T12:45:58.638 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/ec_sys.ko 2024-09-12T12:45:58.638 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/platform_profile.ko 2024-09-12T12:45:58.638 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/sbs.ko 2024-09-12T12:45:58.638 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/sbshc.ko 2024-09-12T12:45:58.638 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/video.ko 2024-09-12T12:45:58.638 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata 2024-09-12T12:45:58.638 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/acard-ahci.ko 2024-09-12T12:45:58.638 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/ahci_platform.ko 2024-09-12T12:45:58.638 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/ata_generic.ko 2024-09-12T12:45:58.638 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/libahci_platform.ko 2024-09-12T12:45:58.638 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_acpi.ko 2024-09-12T12:45:58.638 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ali.ko 2024-09-12T12:45:58.639 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_amd.ko 2024-09-12T12:45:58.639 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_artop.ko 2024-09-12T12:45:58.639 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_atiixp.ko 2024-09-12T12:45:58.639 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_atp867x.ko 2024-09-12T12:45:58.639 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cmd640.ko 2024-09-12T12:45:58.639 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cmd64x.ko 2024-09-12T12:45:58.639 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cypress.ko 2024-09-12T12:45:58.639 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_efar.ko 2024-09-12T12:45:58.639 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt366.ko 2024-09-12T12:45:58.639 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt37x.ko 2024-09-12T12:45:58.639 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt3x2n.ko 2024-09-12T12:45:58.639 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt3x3.ko 2024-09-12T12:45:58.639 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_it8213.ko 2024-09-12T12:45:58.640 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_it821x.ko 2024-09-12T12:45:58.640 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_jmicron.ko 2024-09-12T12:45:58.640 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_marvell.ko 2024-09-12T12:45:58.640 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_mpiix.ko 2024-09-12T12:45:58.640 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_netcell.ko 2024-09-12T12:45:58.640 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ninja32.ko 2024-09-12T12:45:58.640 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ns87410.ko 2024-09-12T12:45:58.640 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ns87415.ko 2024-09-12T12:45:58.640 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_oldpiix.ko 2024-09-12T12:45:58.640 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_opti.ko 2024-09-12T12:45:58.640 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_optidma.ko 2024-09-12T12:45:58.640 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pcmcia.ko 2024-09-12T12:45:58.640 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pdc2027x.ko 2024-09-12T12:45:58.641 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-09-12T12:45:58.641 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_piccolo.ko 2024-09-12T12:45:58.641 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_rdc.ko 2024-09-12T12:45:58.641 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sch.ko 2024-09-12T12:45:58.641 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_serverworks.ko 2024-09-12T12:45:58.641 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sil680.ko 2024-09-12T12:45:58.641 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sis.ko 2024-09-12T12:45:58.641 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sl82c105.ko 2024-09-12T12:45:58.641 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_triflex.ko 2024-09-12T12:45:58.641 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_via.ko 2024-09-12T12:45:58.641 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pdc_adma.ko 2024-09-12T12:45:58.641 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_inic162x.ko 2024-09-12T12:45:58.641 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_mv.ko 2024-09-12T12:45:58.642 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_nv.ko 2024-09-12T12:45:58.642 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_promise.ko 2024-09-12T12:45:58.642 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_qstor.ko 2024-09-12T12:45:58.642 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sil.ko 2024-09-12T12:45:58.642 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sil24.ko 2024-09-12T12:45:58.642 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sis.ko 2024-09-12T12:45:58.642 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_svw.ko 2024-09-12T12:45:58.642 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sx4.ko 2024-09-12T12:45:58.642 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_uli.ko 2024-09-12T12:45:58.642 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_via.ko 2024-09-12T12:45:58.642 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_vsc.ko 2024-09-12T12:45:58.642 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/auxdisplay 2024-09-12T12:45:58.642 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/auxdisplay/ks0108.ko 2024-09-12T12:45:58.643 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/base 2024-09-12T12:45:58.643 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/base/regmap 2024-09-12T12:45:58.643 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/base/regmap/regmap-i2c.ko 2024-09-12T12:45:58.643 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/base/regmap/regmap-mmio.ko 2024-09-12T12:45:58.643 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/bcma 2024-09-12T12:45:58.643 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/bcma/bcma.ko 2024-09-12T12:45:58.643 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block 2024-09-12T12:45:58.643 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/aoe 2024-09-12T12:45:58.643 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/aoe/aoe.ko 2024-09-12T12:45:58.643 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/brd.ko 2024-09-12T12:45:58.643 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/drbd 2024-09-12T12:45:58.643 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/drbd/drbd.ko 2024-09-12T12:45:58.643 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/floppy.ko 2024-09-12T12:45:58.644 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/loop.ko 2024-09-12T12:45:58.644 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/mtip32xx 2024-09-12T12:45:58.644 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-09-12T12:45:58.644 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/nbd.ko 2024-09-12T12:45:58.644 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/null_blk 2024-09-12T12:45:58.644 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/null_blk/null_blk.ko 2024-09-12T12:45:58.644 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/pktcdvd.ko 2024-09-12T12:45:58.644 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/rbd.ko 2024-09-12T12:45:58.644 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/virtio_blk.ko 2024-09-12T12:45:58.644 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/xen-blkback 2024-09-12T12:45:58.644 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-09-12T12:45:58.644 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/xen-blkfront.ko 2024-09-12T12:45:58.644 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/zram 2024-09-12T12:45:58.645 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/zram/zram.ko 2024-09-12T12:45:58.645 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char 2024-09-12T12:45:58.645 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hangcheck-timer.ko 2024-09-12T12:45:58.645 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random 2024-09-12T12:45:58.645 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/amd-rng.ko 2024-09-12T12:45:58.645 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/intel-rng.ko 2024-09-12T12:45:58.645 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-09-12T12:45:58.645 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/via-rng.ko 2024-09-12T12:45:58.645 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/virtio-rng.ko 2024-09-12T12:45:58.645 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi 2024-09-12T12:45:58.645 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-09-12T12:45:58.645 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-09-12T12:45:58.645 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-09-12T12:45:58.646 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_si.ko 2024-09-12T12:45:58.646 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-09-12T12:45:58.646 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-09-12T12:45:58.646 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/lp.ko 2024-09-12T12:45:58.646 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/mwave 2024-09-12T12:45:58.646 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/mwave/mwave.ko 2024-09-12T12:45:58.646 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ppdev.ko 2024-09-12T12:45:58.646 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tlclk.ko 2024-09-12T12:45:58.646 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm 2024-09-12T12:45:58.646 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm.ko 2024-09-12T12:45:58.646 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_atmel.ko 2024-09-12T12:45:58.646 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_infineon.ko 2024-09-12T12:45:58.646 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_nsc.ko 2024-09-12T12:45:58.646 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_tis.ko 2024-09-12T12:45:58.647 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-09-12T12:45:58.647 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/uv_mmtimer.ko 2024-09-12T12:45:58.647 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/virtio_console.ko 2024-09-12T12:45:58.647 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq 2024-09-12T12:45:58.647 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-09-12T12:45:58.647 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-09-12T12:45:58.647 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/p4-clockmod.ko 2024-09-12T12:45:58.647 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-09-12T12:45:58.647 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/powernow-k8.ko 2024-09-12T12:45:58.647 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/speedstep-lib.ko 2024-09-12T12:45:58.647 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto 2024-09-12T12:45:58.647 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp 2024-09-12T12:45:58.647 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-09-12T12:45:58.648 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp/ccp.ko 2024-09-12T12:45:58.648 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel 2024-09-12T12:45:58.648 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat 2024-09-12T12:45:58.648 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_common 2024-09-12T12:45:58.648 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-09-12T12:45:58.648 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-09-12T12:45:58.648 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-09-12T12:45:58.648 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/padlock-aes.ko 2024-09-12T12:45:58.648 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/padlock-sha.ko 2024-09-12T12:45:58.648 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/virtio 2024-09-12T12:45:58.648 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-09-12T12:45:58.648 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dca 2024-09-12T12:45:58.648 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dca/dca.ko 2024-09-12T12:45:58.649 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/devfreq 2024-09-12T12:45:58.649 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-09-12T12:45:58.649 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma 2024-09-12T12:45:58.649 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/dw 2024-09-12T12:45:58.649 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/dw/dw_dmac.ko 2024-09-12T12:45:58.649 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/ioat 2024-09-12T12:45:58.649 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/ioat/ioatdma.ko 2024-09-12T12:45:58.649 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac 2024-09-12T12:45:58.649 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/amd64_edac.ko 2024-09-12T12:45:58.649 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/e752x_edac.ko 2024-09-12T12:45:58.649 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/edac_mce_amd.ko 2024-09-12T12:45:58.649 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i3000_edac.ko 2024-09-12T12:45:58.649 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i3200_edac.ko 2024-09-12T12:45:58.650 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i5100_edac.ko 2024-09-12T12:45:58.650 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i5400_edac.ko 2024-09-12T12:45:58.650 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i7300_edac.ko 2024-09-12T12:45:58.650 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i7core_edac.ko 2024-09-12T12:45:58.650 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i82975x_edac.ko 2024-09-12T12:45:58.650 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/ie31200_edac.ko 2024-09-12T12:45:58.650 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/sb_edac.ko 2024-09-12T12:45:58.650 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/x38_edac.ko 2024-09-12T12:45:58.650 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware 2024-09-12T12:45:58.650 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/edd.ko 2024-09-12T12:45:58.650 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/iscsi_ibft.ko 2024-09-12T12:45:58.650 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio 2024-09-12T12:45:58.650 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-ich.ko 2024-09-12T12:45:58.651 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-viperboard.ko 2024-09-12T12:45:58.651 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid 2024-09-12T12:45:58.651 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-appleir.ko 2024-09-12T12:45:58.651 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-aureal.ko 2024-09-12T12:45:58.651 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-axff.ko 2024-09-12T12:45:58.651 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-dr.ko 2024-09-12T12:45:58.651 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-elecom.ko 2024-09-12T12:45:58.651 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-elo.ko 2024-09-12T12:45:58.651 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-emsff.ko 2024-09-12T12:45:58.651 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gaff.ko 2024-09-12T12:45:58.651 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gt683r.ko 2024-09-12T12:45:58.651 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gyration.ko 2024-09-12T12:45:58.651 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtek-kbd.ko 2024-09-12T12:45:58.652 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtek-mouse.ko 2024-09-12T12:45:58.652 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtekff.ko 2024-09-12T12:45:58.652 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-hyperv.ko 2024-09-12T12:45:58.652 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-icade.ko 2024-09-12T12:45:58.652 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-keytouch.ko 2024-09-12T12:45:58.652 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-kye.ko 2024-09-12T12:45:58.652 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-lcpower.ko 2024-09-12T12:45:58.652 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-led.ko 2024-09-12T12:45:58.652 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-lenovo.ko 2024-09-12T12:45:58.652 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-logitech-dj.ko 2024-09-12T12:45:58.652 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-09-12T12:45:58.652 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-multitouch.ko 2024-09-12T12:45:58.653 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-ortek.ko 2024-09-12T12:45:58.653 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-penmount.ko 2024-09-12T12:45:58.653 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-petalynx.ko 2024-09-12T12:45:58.653 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-picolcd.ko 2024-09-12T12:45:58.653 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-pl.ko 2024-09-12T12:45:58.653 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-plantronics.ko 2024-09-12T12:45:58.653 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-primax.ko 2024-09-12T12:45:58.653 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-rmi.ko 2024-09-12T12:45:58.653 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-arvo.ko 2024-09-12T12:45:58.653 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-common.ko 2024-09-12T12:45:58.653 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-isku.ko 2024-09-12T12:45:58.653 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-kone.ko 2024-09-12T12:45:58.653 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-09-12T12:45:58.654 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-konepure.ko 2024-09-12T12:45:58.654 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-09-12T12:45:58.654 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-lua.ko 2024-09-12T12:45:58.654 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-pyra.ko 2024-09-12T12:45:58.654 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-ryos.ko 2024-09-12T12:45:58.654 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-savu.ko 2024-09-12T12:45:58.654 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat.ko 2024-09-12T12:45:58.654 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-saitek.ko 2024-09-12T12:45:58.654 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-samsung.ko 2024-09-12T12:45:58.654 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sensor-hub.ko 2024-09-12T12:45:58.654 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sjoy.ko 2024-09-12T12:45:58.654 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sony.ko 2024-09-12T12:45:58.654 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-speedlink.ko 2024-09-12T12:45:58.655 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-steelseries.ko 2024-09-12T12:45:58.655 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sunplus.ko 2024-09-12T12:45:58.655 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-thrustmaster.ko 2024-09-12T12:45:58.655 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-tivo.ko 2024-09-12T12:45:58.655 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-tmff.ko 2024-09-12T12:45:58.655 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-topseed.ko 2024-09-12T12:45:58.655 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-twinhan.ko 2024-09-12T12:45:58.655 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-uclogic.ko 2024-09-12T12:45:58.655 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-waltop.ko 2024-09-12T12:45:58.655 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-wiimote.ko 2024-09-12T12:45:58.655 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-xinmo.ko 2024-09-12T12:45:58.655 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-zpff.ko 2024-09-12T12:45:58.655 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-zydacron.ko 2024-09-12T12:45:58.655 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/uhid.ko 2024-09-12T12:45:58.656 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/wacom.ko 2024-09-12T12:45:58.656 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv 2024-09-12T12:45:58.656 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_balloon.ko 2024-09-12T12:45:58.656 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_utils.ko 2024-09-12T12:45:58.656 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_vmbus.ko 2024-09-12T12:45:58.656 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon 2024-09-12T12:45:58.656 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/abituguru.ko 2024-09-12T12:45:58.656 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/abituguru3.ko 2024-09-12T12:45:58.656 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/acpi_power_meter.ko 2024-09-12T12:45:58.656 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ad7414.ko 2024-09-12T12:45:58.656 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ad7418.ko 2024-09-12T12:45:58.656 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adc128d818.ko 2024-09-12T12:45:58.656 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1025.ko 2024-09-12T12:45:58.657 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1026.ko 2024-09-12T12:45:58.657 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1029.ko 2024-09-12T12:45:58.657 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1031.ko 2024-09-12T12:45:58.657 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm9240.ko 2024-09-12T12:45:58.657 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ads7828.ko 2024-09-12T12:45:58.657 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7410.ko 2024-09-12T12:45:58.657 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7411.ko 2024-09-12T12:45:58.657 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7462.ko 2024-09-12T12:45:58.657 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7470.ko 2024-09-12T12:45:58.657 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7475.ko 2024-09-12T12:45:58.657 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7x10.ko 2024-09-12T12:45:58.657 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/amc6821.ko 2024-09-12T12:45:58.657 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/applesmc.ko 2024-09-12T12:45:58.658 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asb100.ko 2024-09-12T12:45:58.658 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asc7621.ko 2024-09-12T12:45:58.658 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asus_atk0110.ko 2024-09-12T12:45:58.658 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/atxp1.ko 2024-09-12T12:45:58.658 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/coretemp.ko 2024-09-12T12:45:58.658 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-09-12T12:45:58.658 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/dme1737.ko 2024-09-12T12:45:58.658 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ds1621.ko 2024-09-12T12:45:58.658 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ds620.ko 2024-09-12T12:45:58.658 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/emc1403.ko 2024-09-12T12:45:58.658 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/emc6w201.ko 2024-09-12T12:45:58.658 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f71805f.ko 2024-09-12T12:45:58.658 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f71882fg.ko 2024-09-12T12:45:58.659 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f75375s.ko 2024-09-12T12:45:58.659 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/fam15h_power.ko 2024-09-12T12:45:58.659 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/fschmd.ko 2024-09-12T12:45:58.659 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/g760a.ko 2024-09-12T12:45:58.659 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/g762.ko 2024-09-12T12:45:58.659 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/gl518sm.ko 2024-09-12T12:45:58.659 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/gl520sm.ko 2024-09-12T12:45:58.659 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/hwmon-vid.ko 2024-09-12T12:45:58.659 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/i5500_temp.ko 2024-09-12T12:45:58.659 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/i5k_amb.ko 2024-09-12T12:45:58.659 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ibmaem.ko 2024-09-12T12:45:58.659 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ibmpex.ko 2024-09-12T12:45:58.659 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ina209.ko 2024-09-12T12:45:58.659 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ina2xx.ko 2024-09-12T12:45:58.660 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/it87.ko 2024-09-12T12:45:58.660 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/k10temp.ko 2024-09-12T12:45:58.660 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/k8temp.ko 2024-09-12T12:45:58.660 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lineage-pem.ko 2024-09-12T12:45:58.660 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm63.ko 2024-09-12T12:45:58.660 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm73.ko 2024-09-12T12:45:58.660 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm75.ko 2024-09-12T12:45:58.660 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm77.ko 2024-09-12T12:45:58.660 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm78.ko 2024-09-12T12:45:58.660 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm80.ko 2024-09-12T12:45:58.660 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm83.ko 2024-09-12T12:45:58.660 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm85.ko 2024-09-12T12:45:58.660 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm87.ko 2024-09-12T12:45:58.661 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm90.ko 2024-09-12T12:45:58.661 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm92.ko 2024-09-12T12:45:58.661 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm93.ko 2024-09-12T12:45:58.661 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95234.ko 2024-09-12T12:45:58.661 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95241.ko 2024-09-12T12:45:58.661 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95245.ko 2024-09-12T12:45:58.661 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc2945.ko 2024-09-12T12:45:58.661 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4151.ko 2024-09-12T12:45:58.661 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4215.ko 2024-09-12T12:45:58.661 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4222.ko 2024-09-12T12:45:58.661 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4245.ko 2024-09-12T12:45:58.661 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4260.ko 2024-09-12T12:45:58.661 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4261.ko 2024-09-12T12:45:58.662 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max16065.ko 2024-09-12T12:45:58.662 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max1619.ko 2024-09-12T12:45:58.662 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max1668.ko 2024-09-12T12:45:58.662 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max197.ko 2024-09-12T12:45:58.662 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6639.ko 2024-09-12T12:45:58.662 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6650.ko 2024-09-12T12:45:58.662 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6697.ko 2024-09-12T12:45:58.662 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/mcp3021.ko 2024-09-12T12:45:58.662 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6683.ko 2024-09-12T12:45:58.662 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6775-core.ko 2024-09-12T12:45:58.662 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6775.ko 2024-09-12T12:45:58.662 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct7802.ko 2024-09-12T12:45:58.662 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pc87360.ko 2024-09-12T12:45:58.663 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pc87427.ko 2024-09-12T12:45:58.663 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pcf8591.ko 2024-09-12T12:45:58.663 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus 2024-09-12T12:45:58.663 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-09-12T12:45:58.663 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-09-12T12:45:58.663 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-09-12T12:45:58.663 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max16064.ko 2024-09-12T12:45:58.663 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max34440.ko 2024-09-12T12:45:58.663 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max8688.ko 2024-09-12T12:45:58.663 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-09-12T12:45:58.663 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-09-12T12:45:58.663 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-09-12T12:45:58.663 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-09-12T12:45:58.664 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-09-12T12:45:58.664 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-09-12T12:45:58.664 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/powr1220.ko 2024-09-12T12:45:58.664 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch5627.ko 2024-09-12T12:45:58.664 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch5636.ko 2024-09-12T12:45:58.664 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch56xx-common.ko 2024-09-12T12:45:58.664 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sht15.ko 2024-09-12T12:45:58.664 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sht21.ko 2024-09-12T12:45:58.664 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/shtc1.ko 2024-09-12T12:45:58.664 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sis5595.ko 2024-09-12T12:45:58.664 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47b397.ko 2024-09-12T12:45:58.664 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47m1.ko 2024-09-12T12:45:58.665 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47m192.ko 2024-09-12T12:45:58.665 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/thmc50.ko 2024-09-12T12:45:58.665 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp102.ko 2024-09-12T12:45:58.665 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp103.ko 2024-09-12T12:45:58.665 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp401.ko 2024-09-12T12:45:58.665 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp421.ko 2024-09-12T12:45:58.665 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/via-cputemp.ko 2024-09-12T12:45:58.665 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/via686a.ko 2024-09-12T12:45:58.665 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/vt1211.ko 2024-09-12T12:45:58.665 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/vt8231.ko 2024-09-12T12:45:58.665 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83627ehf.ko 2024-09-12T12:45:58.665 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83627hf.ko 2024-09-12T12:45:58.665 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83781d.ko 2024-09-12T12:45:58.666 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83791d.ko 2024-09-12T12:45:58.666 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83792d.ko 2024-09-12T12:45:58.666 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83793.ko 2024-09-12T12:45:58.666 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83795.ko 2024-09-12T12:45:58.666 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83l785ts.ko 2024-09-12T12:45:58.666 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83l786ng.ko 2024-09-12T12:45:58.666 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c 2024-09-12T12:45:58.666 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos 2024-09-12T12:45:58.666 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-09-12T12:45:58.666 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-09-12T12:45:58.666 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses 2024-09-12T12:45:58.666 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-09-12T12:45:58.667 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-09-12T12:45:58.667 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-09-12T12:45:58.667 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-09-12T12:45:58.667 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-09-12T12:45:58.667 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-09-12T12:45:58.667 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-09-12T12:45:58.667 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-09-12T12:45:58.667 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-i801.ko 2024-09-12T12:45:58.667 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-isch.ko 2024-09-12T12:45:58.667 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-09-12T12:45:58.667 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-09-12T12:45:58.667 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-09-12T12:45:58.667 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-parport.ko 2024-09-12T12:45:58.667 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-09-12T12:45:58.668 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-09-12T12:45:58.668 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-09-12T12:45:58.668 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-09-12T12:45:58.668 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-09-12T12:45:58.668 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-09-12T12:45:58.668 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-via.ko 2024-09-12T12:45:58.668 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-09-12T12:45:58.668 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-09-12T12:45:58.668 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-dev.ko 2024-09-12T12:45:58.668 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-mux.ko 2024-09-12T12:45:58.668 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-09-12T12:45:58.668 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-smbus.ko 2024-09-12T12:45:58.668 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-stub.ko 2024-09-12T12:45:58.669 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband 2024-09-12T12:45:58.669 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core 2024-09-12T12:45:58.669 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_cm.ko 2024-09-12T12:45:58.669 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_core.ko 2024-09-12T12:45:58.669 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_umad.ko 2024-09-12T12:45:58.669 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-09-12T12:45:58.669 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/iw_cm.ko 2024-09-12T12:45:58.669 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/rdma_cm.ko 2024-09-12T12:45:58.669 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-09-12T12:45:58.669 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw 2024-09-12T12:45:58.669 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/cxgb4 2024-09-12T12:45:58.669 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-09-12T12:45:58.670 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx4 2024-09-12T12:45:58.670 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-09-12T12:45:58.670 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx5 2024-09-12T12:45:58.670 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-09-12T12:45:58.670 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mthca 2024-09-12T12:45:58.670 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-09-12T12:45:58.670 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp 2024-09-12T12:45:58.670 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/ipoib 2024-09-12T12:45:58.670 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-09-12T12:45:58.670 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/iser 2024-09-12T12:45:58.670 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-09-12T12:45:58.670 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/isert 2024-09-12T12:45:58.670 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-09-12T12:45:58.671 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srp 2024-09-12T12:45:58.671 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-09-12T12:45:58.671 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srpt 2024-09-12T12:45:58.671 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-09-12T12:45:58.671 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input 2024-09-12T12:45:58.671 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport 2024-09-12T12:45:58.671 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-09-12T12:45:58.671 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/fm801-gp.ko 2024-09-12T12:45:58.671 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/gameport.ko 2024-09-12T12:45:58.671 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/joydev.ko 2024-09-12T12:45:58.671 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard 2024-09-12T12:45:58.671 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/gpio_keys.ko 2024-09-12T12:45:58.671 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse 2024-09-12T12:45:58.671 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/appletouch.ko 2024-09-12T12:45:58.672 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/bcm5974.ko 2024-09-12T12:45:58.672 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/cyapatp.ko 2024-09-12T12:45:58.672 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/elan_i2c.ko 2024-09-12T12:45:58.672 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/sermouse.ko 2024-09-12T12:45:58.672 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-09-12T12:45:58.672 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/synaptics_usb.ko 2024-09-12T12:45:58.672 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/vsxxxaa.ko 2024-09-12T12:45:58.672 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/rmi4 2024-09-12T12:45:58.672 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/rmi4/rmi_core.ko 2024-09-12T12:45:58.672 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio 2024-09-12T12:45:58.672 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/altera_ps2.ko 2024-09-12T12:45:58.672 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/arc_ps2.ko 2024-09-12T12:45:58.673 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-09-12T12:45:58.673 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/serio_raw.ko 2024-09-12T12:45:58.673 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/sparse-keymap.ko 2024-09-12T12:45:58.673 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds 2024-09-12T12:45:58.673 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-blinkm.ko 2024-09-12T12:45:58.673 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lm3530.ko 2024-09-12T12:45:58.673 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lp3944.ko 2024-09-12T12:45:58.673 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-ss4200.ko 2024-09-12T12:45:58.673 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger 2024-09-12T12:45:58.673 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-09-12T12:45:58.673 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-09-12T12:45:58.673 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-09-12T12:45:58.673 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-09-12T12:45:58.674 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-09-12T12:45:58.674 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-09-12T12:45:58.674 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-09-12T12:45:58.674 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-09-12T12:45:58.674 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md 2024-09-12T12:45:58.674 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/bcache 2024-09-12T12:45:58.674 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/bcache/bcache.ko 2024-09-12T12:45:58.674 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-bio-prison.ko 2024-09-12T12:45:58.674 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-cache-smq.ko 2024-09-12T12:45:58.674 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-cache.ko 2024-09-12T12:45:58.674 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-crypt.ko 2024-09-12T12:45:58.674 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-delay.ko 2024-09-12T12:45:58.674 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-flakey.ko 2024-09-12T12:45:58.675 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-log-userspace.ko 2024-09-12T12:45:58.675 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-multipath.ko 2024-09-12T12:45:58.675 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-queue-length.ko 2024-09-12T12:45:58.675 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-raid.ko 2024-09-12T12:45:58.675 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-round-robin.ko 2024-09-12T12:45:58.675 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-service-time.ko 2024-09-12T12:45:58.675 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-switch.ko 2024-09-12T12:45:58.676 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-thin-pool.ko 2024-09-12T12:45:58.676 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-verity.ko 2024-09-12T12:45:58.676 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/persistent-data 2024-09-12T12:45:58.676 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-09-12T12:45:58.676 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid0.ko 2024-09-12T12:45:58.676 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid1.ko 2024-09-12T12:45:58.676 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid10.ko 2024-09-12T12:45:58.676 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid456.ko 2024-09-12T12:45:58.676 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message 2024-09-12T12:45:58.676 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion 2024-09-12T12:45:58.676 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptbase.ko 2024-09-12T12:45:58.676 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptctl.ko 2024-09-12T12:45:58.676 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptfc.ko 2024-09-12T12:45:58.676 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptlan.ko 2024-09-12T12:45:58.677 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptsas.ko 2024-09-12T12:45:58.677 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptscsih.ko 2024-09-12T12:45:58.677 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptspi.ko 2024-09-12T12:45:58.677 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd 2024-09-12T12:45:58.677 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lpc_ich.ko 2024-09-12T12:45:58.677 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lpc_sch.ko 2024-09-12T12:45:58.677 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/mfd-core.ko 2024-09-12T12:45:58.677 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/sm501.ko 2024-09-12T12:45:58.677 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/viperboard.ko 2024-09-12T12:45:58.677 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/vx855.ko 2024-09-12T12:45:58.677 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/wl1273-core.ko 2024-09-12T12:45:58.677 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc 2024-09-12T12:45:58.677 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/altera-stapl 2024-09-12T12:45:58.678 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-09-12T12:45:58.678 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/apds9802als.ko 2024-09-12T12:45:58.678 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/apds990x.ko 2024-09-12T12:45:58.678 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/bh1770glc.ko 2024-09-12T12:45:58.678 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/cb710 2024-09-12T12:45:58.678 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/cb710/cb710.ko 2024-09-12T12:45:58.678 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom 2024-09-12T12:45:58.678 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/at24.ko 2024-09-12T12:45:58.678 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-09-12T12:45:58.678 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/max6875.ko 2024-09-12T12:45:58.678 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/enclosure.ko 2024-09-12T12:45:58.678 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/hpilo.ko 2024-09-12T12:45:58.678 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/isl29003.ko 2024-09-12T12:45:58.678 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/isl29020.ko 2024-09-12T12:45:58.679 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d 2024-09-12T12:45:58.679 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-09-12T12:45:58.679 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-09-12T12:45:58.679 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei 2024-09-12T12:45:58.679 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei-me.ko 2024-09-12T12:45:58.679 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei-txe.ko 2024-09-12T12:45:58.679 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei.ko 2024-09-12T12:45:58.679 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-gru 2024-09-12T12:45:58.679 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-gru/gru.ko 2024-09-12T12:45:58.679 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-xp 2024-09-12T12:45:58.679 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-xp/xp.ko 2024-09-12T12:45:58.679 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-xp/xpc.ko 2024-09-12T12:45:58.679 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-09-12T12:45:58.680 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tifm_7xx1.ko 2024-09-12T12:45:58.680 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tifm_core.ko 2024-09-12T12:45:58.680 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tsl2550.ko 2024-09-12T12:45:58.680 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_balloon.ko 2024-09-12T12:45:58.680 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_vmci 2024-09-12T12:45:58.680 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-09-12T12:45:58.680 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net 2024-09-12T12:45:58.680 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/bonding 2024-09-12T12:45:58.680 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/bonding/bonding.ko 2024-09-12T12:45:58.680 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa 2024-09-12T12:45:58.680 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/b53 2024-09-12T12:45:58.680 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/b53/b53_common.ko 2024-09-12T12:45:58.680 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/bcm-sf2.ko 2024-09-12T12:45:58.681 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6060.ko 2024-09-12T12:45:58.681 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6xxx 2024-09-12T12:45:58.681 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-09-12T12:45:58.681 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dummy.ko 2024-09-12T12:45:58.681 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/eql.ko 2024-09-12T12:45:58.681 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet 2024-09-12T12:45:58.681 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com 2024-09-12T12:45:58.681 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-09-12T12:45:58.681 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-09-12T12:45:58.681 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-09-12T12:45:58.681 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-09-12T12:45:58.681 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390 2024-09-12T12:45:58.681 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/8390.ko 2024-09-12T12:45:58.682 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-09-12T12:45:58.682 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-09-12T12:45:58.682 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-09-12T12:45:58.682 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/adaptec 2024-09-12T12:45:58.682 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-09-12T12:45:58.682 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/agere 2024-09-12T12:45:58.682 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/agere/et131x.ko 2024-09-12T12:45:58.682 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/alteon 2024-09-12T12:45:58.682 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-09-12T12:45:58.682 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/altera 2024-09-12T12:45:58.682 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-09-12T12:45:58.682 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd 2024-09-12T12:45:58.682 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-09-12T12:45:58.683 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-09-12T12:45:58.683 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-09-12T12:45:58.683 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros 2024-09-12T12:45:58.683 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/alx 2024-09-12T12:45:58.683 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-09-12T12:45:58.683 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1c 2024-09-12T12:45:58.683 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-09-12T12:45:58.683 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1e 2024-09-12T12:45:58.683 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-09-12T12:45:58.683 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx 2024-09-12T12:45:58.683 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-09-12T12:45:58.683 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-09-12T12:45:58.684 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom 2024-09-12T12:45:58.684 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-09-12T12:45:58.684 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-09-12T12:45:58.684 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-09-12T12:45:58.684 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-09-12T12:45:58.684 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-09-12T12:45:58.684 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/genet 2024-09-12T12:45:58.684 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-09-12T12:45:58.684 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-09-12T12:45:58.684 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade 2024-09-12T12:45:58.684 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade/bna 2024-09-12T12:45:58.684 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-09-12T12:45:58.685 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio 2024-09-12T12:45:58.685 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb 2024-09-12T12:45:58.685 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-09-12T12:45:58.685 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-09-12T12:45:58.685 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-09-12T12:45:58.685 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-09-12T12:45:58.685 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-09-12T12:45:58.685 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-09-12T12:45:58.685 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-09-12T12:45:58.685 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-09-12T12:45:58.685 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-09-12T12:45:58.685 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco 2024-09-12T12:45:58.685 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco/enic 2024-09-12T12:45:58.686 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-09-12T12:45:58.686 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec 2024-09-12T12:45:58.686 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip 2024-09-12T12:45:58.686 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-09-12T12:45:58.686 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-09-12T12:45:58.686 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-09-12T12:45:58.686 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-09-12T12:45:58.686 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-09-12T12:45:58.686 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-09-12T12:45:58.686 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink 2024-09-12T12:45:58.686 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-09-12T12:45:58.686 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-09-12T12:45:58.686 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dnet.ko 2024-09-12T12:45:58.686 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex 2024-09-12T12:45:58.687 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex/benet 2024-09-12T12:45:58.687 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-09-12T12:45:58.687 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ethoc.ko 2024-09-12T12:45:58.687 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/fealnx.ko 2024-09-12T12:45:58.687 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel 2024-09-12T12:45:58.687 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e100.ko 2024-09-12T12:45:58.687 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000 2024-09-12T12:45:58.687 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-09-12T12:45:58.687 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000e 2024-09-12T12:45:58.687 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-09-12T12:45:58.687 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/fm10k 2024-09-12T12:45:58.687 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-09-12T12:45:58.688 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/i40e 2024-09-12T12:45:58.688 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-09-12T12:45:58.688 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/iavf 2024-09-12T12:45:58.688 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-09-12T12:45:58.688 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igb 2024-09-12T12:45:58.688 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-09-12T12:45:58.688 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igbvf 2024-09-12T12:45:58.688 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-09-12T12:45:58.688 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbe 2024-09-12T12:45:58.688 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-09-12T12:45:58.688 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbevf 2024-09-12T12:45:58.688 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-09-12T12:45:58.689 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libeth 2024-09-12T12:45:58.689 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libeth/libeth.ko 2024-09-12T12:45:58.689 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libie 2024-09-12T12:45:58.689 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libie/libie.ko 2024-09-12T12:45:58.689 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/jme.ko 2024-09-12T12:45:58.689 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell 2024-09-12T12:45:58.689 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-09-12T12:45:58.689 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/skge.ko 2024-09-12T12:45:58.689 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-09-12T12:45:58.689 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox 2024-09-12T12:45:58.689 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4 2024-09-12T12:45:58.689 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-09-12T12:45:58.689 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-09-12T12:45:58.690 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5 2024-09-12T12:45:58.690 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-09-12T12:45:58.690 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-09-12T12:45:58.690 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel 2024-09-12T12:45:58.690 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-09-12T12:45:58.690 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom 2024-09-12T12:45:58.690 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom/myri10ge 2024-09-12T12:45:58.690 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-09-12T12:45:58.690 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi 2024-09-12T12:45:58.690 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-09-12T12:45:58.690 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-09-12T12:45:58.691 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/neterion 2024-09-12T12:45:58.691 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-09-12T12:45:58.691 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/nvidia 2024-09-12T12:45:58.691 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-09-12T12:45:58.691 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines 2024-09-12T12:45:58.691 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-09-12T12:45:58.691 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-09-12T12:45:58.691 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic 2024-09-12T12:45:58.691 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/netxen 2024-09-12T12:45:58.691 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-09-12T12:45:58.691 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-09-12T12:45:58.691 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-09-12T12:45:58.692 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-09-12T12:45:58.692 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rdc 2024-09-12T12:45:58.692 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-09-12T12:45:58.692 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek 2024-09-12T12:45:58.692 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-09-12T12:45:58.692 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-09-12T12:45:58.692 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/atp.ko 2024-09-12T12:45:58.692 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-09-12T12:45:58.692 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rocker 2024-09-12T12:45:58.692 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-09-12T12:45:58.692 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sfc 2024-09-12T12:45:58.692 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-09-12T12:45:58.692 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/silan 2024-09-12T12:45:58.693 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-09-12T12:45:58.693 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis 2024-09-12T12:45:58.693 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis/sis190.ko 2024-09-12T12:45:58.693 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis/sis900.ko 2024-09-12T12:45:58.693 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc 2024-09-12T12:45:58.693 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-09-12T12:45:58.693 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-09-12T12:45:58.693 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-09-12T12:45:58.693 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-09-12T12:45:58.693 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro 2024-09-12T12:45:58.693 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac 2024-09-12T12:45:58.693 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-09-12T12:45:58.693 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-09-12T12:45:58.693 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun 2024-09-12T12:45:58.694 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/cassini.ko 2024-09-12T12:45:58.694 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/niu.ko 2024-09-12T12:45:58.694 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/sungem.ko 2024-09-12T12:45:58.694 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-09-12T12:45:58.694 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/tehuti 2024-09-12T12:45:58.694 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-09-12T12:45:58.694 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ti 2024-09-12T12:45:58.694 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ti/tlan.ko 2024-09-12T12:45:58.694 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via 2024-09-12T12:45:58.694 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-09-12T12:45:58.694 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-09-12T12:45:58.694 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet 2024-09-12T12:45:58.694 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-09-12T12:45:58.695 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-09-12T12:45:58.695 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/xircom 2024-09-12T12:45:58.695 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-09-12T12:45:58.695 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/geneve.ko 2024-09-12T12:45:58.695 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/hyperv 2024-09-12T12:45:58.695 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-09-12T12:45:58.695 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ieee802154 2024-09-12T12:45:58.695 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ieee802154/fakelb.ko 2024-09-12T12:45:58.695 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ifb.ko 2024-09-12T12:45:58.695 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ipvlan 2024-09-12T12:45:58.695 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ipvlan/ipvlan.ko 2024-09-12T12:45:58.695 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/macvlan.ko 2024-09-12T12:45:58.695 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/macvtap.ko 2024-09-12T12:45:58.696 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio 2024-09-12T12:45:58.696 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio.ko 2024-09-12T12:45:58.696 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-09-12T12:45:58.696 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-09-12T12:45:58.696 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-regmap.ko 2024-09-12T12:45:58.696 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mii.ko 2024-09-12T12:45:58.696 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/net_failover.ko 2024-09-12T12:45:58.696 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/netconsole.ko 2024-09-12T12:45:58.696 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/nlmon.ko 2024-09-12T12:45:58.696 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs 2024-09-12T12:45:58.696 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs/pcs-lynx.ko 2024-09-12T12:45:58.696 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-09-12T12:45:58.696 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy 2024-09-12T12:45:58.697 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/amd.ko 2024-09-12T12:45:58.697 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-09-12T12:45:58.697 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-09-12T12:45:58.697 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm7xxx.ko 2024-09-12T12:45:58.697 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm87xx.ko 2024-09-12T12:45:58.697 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/broadcom.ko 2024-09-12T12:45:58.697 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/cicada.ko 2024-09-12T12:45:58.697 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/davicom.ko 2024-09-12T12:45:58.697 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/dp83640.ko 2024-09-12T12:45:58.697 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/et1011c.ko 2024-09-12T12:45:58.697 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/icplus.ko 2024-09-12T12:45:58.697 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/lxt.ko 2024-09-12T12:45:58.697 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/marvell.ko 2024-09-12T12:45:58.698 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/micrel.ko 2024-09-12T12:45:58.698 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/national.ko 2024-09-12T12:45:58.698 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/phylink.ko 2024-09-12T12:45:58.698 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/qsemi.ko 2024-09-12T12:45:58.698 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/realtek.ko 2024-09-12T12:45:58.698 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/smsc.ko 2024-09-12T12:45:58.698 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/ste10Xp.ko 2024-09-12T12:45:58.698 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/vitesse.ko 2024-09-12T12:45:58.698 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp 2024-09-12T12:45:58.698 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/bsd_comp.ko 2024-09-12T12:45:58.698 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_async.ko 2024-09-12T12:45:58.698 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_deflate.ko 2024-09-12T12:45:58.699 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_generic.ko 2024-09-12T12:45:58.699 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_mppe.ko 2024-09-12T12:45:58.699 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_synctty.ko 2024-09-12T12:45:58.699 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pppoe.ko 2024-09-12T12:45:58.699 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pppox.ko 2024-09-12T12:45:58.699 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pptp.ko 2024-09-12T12:45:58.699 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/slip 2024-09-12T12:45:58.699 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/slip/slhc.ko 2024-09-12T12:45:58.699 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/slip/slip.ko 2024-09-12T12:45:58.699 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/sungem_phy.ko 2024-09-12T12:45:58.699 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/tap.ko 2024-09-12T12:45:58.699 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team 2024-09-12T12:45:58.699 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team.ko 2024-09-12T12:45:58.700 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_activebackup.ko 2024-09-12T12:45:58.700 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_broadcast.ko 2024-09-12T12:45:58.700 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-09-12T12:45:58.700 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_random.ko 2024-09-12T12:45:58.700 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-09-12T12:45:58.700 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/tun.ko 2024-09-12T12:45:58.700 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/veth.ko 2024-09-12T12:45:58.700 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/virtio_net.ko 2024-09-12T12:45:58.700 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vmxnet3 2024-09-12T12:45:58.700 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-09-12T12:45:58.700 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vxlan 2024-09-12T12:45:58.700 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vxlan/vxlan.ko 2024-09-12T12:45:58.700 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/xen-netback 2024-09-12T12:45:58.701 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/xen-netback/xen-netback.ko 2024-09-12T12:45:58.701 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/xen-netfront.ko 2024-09-12T12:45:58.701 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ntb 2024-09-12T12:45:58.701 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ntb/ntb.ko 2024-09-12T12:45:58.701 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme 2024-09-12T12:45:58.701 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host 2024-09-12T12:45:58.701 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host/nvme-core.ko 2024-09-12T12:45:58.701 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host/nvme.ko 2024-09-12T12:45:58.701 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport 2024-09-12T12:45:58.701 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport.ko 2024-09-12T12:45:58.701 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_cs.ko 2024-09-12T12:45:58.701 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_pc.ko 2024-09-12T12:45:58.701 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_serial.ko 2024-09-12T12:45:58.702 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci 2024-09-12T12:45:58.702 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug 2024-09-12T12:45:58.702 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-09-12T12:45:58.702 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/pcie 2024-09-12T12:45:58.702 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/pcie/aer_inject.ko 2024-09-12T12:45:58.702 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/xen-pcifront.ko 2024-09-12T12:45:58.702 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia 2024-09-12T12:45:58.702 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/i82092.ko 2024-09-12T12:45:58.702 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/pd6729.ko 2024-09-12T12:45:58.702 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/yenta_socket.ko 2024-09-12T12:45:58.702 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl 2024-09-12T12:45:58.702 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl/intel 2024-09-12T12:45:58.702 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-09-12T12:45:58.703 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform 2024-09-12T12:45:58.703 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86 2024-09-12T12:45:58.703 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/acer-wmi.ko 2024-09-12T12:45:58.703 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/acerhdf.ko 2024-09-12T12:45:58.703 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/apple-gmux.ko 2024-09-12T12:45:58.703 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-laptop.ko 2024-09-12T12:45:58.703 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-09-12T12:45:58.703 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-wmi.ko 2024-09-12T12:45:58.703 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/classmate-laptop.ko 2024-09-12T12:45:58.703 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-09-12T12:45:58.703 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-09-12T12:45:58.703 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-09-12T12:45:58.703 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-09-12T12:45:58.704 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/hdaps.ko 2024-09-12T12:45:58.704 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel 2024-09-12T12:45:58.704 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-09-12T12:45:58.704 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel_ips.ko 2024-09-12T12:45:58.704 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/msi-wmi.ko 2024-09-12T12:45:58.704 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/mxm-wmi.ko 2024-09-12T12:45:58.704 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-09-12T12:45:58.704 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/samsung-laptop.ko 2024-09-12T12:45:58.704 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/samsung-q10.ko 2024-09-12T12:45:58.704 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/topstar-laptop.ko 2024-09-12T12:45:58.704 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-09-12T12:45:58.704 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/toshiba_haps.ko 2024-09-12T12:45:58.705 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/wmi-bmof.ko 2024-09-12T12:45:58.705 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/wmi.ko 2024-09-12T12:45:58.705 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap 2024-09-12T12:45:58.705 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap/intel_rapl_common.ko 2024-09-12T12:45:58.705 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap/intel_rapl_msr.ko 2024-09-12T12:45:58.705 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps 2024-09-12T12:45:58.705 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients 2024-09-12T12:45:58.705 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps-gpio.ko 2024-09-12T12:45:58.705 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps-ldisc.ko 2024-09-12T12:45:58.705 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps_parport.ko 2024-09-12T12:45:58.705 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/pps_core.ko 2024-09-12T12:45:58.705 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp 2024-09-12T12:45:58.705 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp/ptp.ko 2024-09-12T12:45:58.706 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp/ptp_kvm.ko 2024-09-12T12:45:58.706 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras 2024-09-12T12:45:58.706 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd 2024-09-12T12:45:58.706 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/atl 2024-09-12T12:45:58.706 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-09-12T12:45:58.706 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/fmpm.ko 2024-09-12T12:45:58.706 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc 2024-09-12T12:45:58.706 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-bq32k.ko 2024-09-12T12:45:58.706 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1286.ko 2024-09-12T12:45:58.706 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1307.ko 2024-09-12T12:45:58.706 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1374.ko 2024-09-12T12:45:58.706 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1511.ko 2024-09-12T12:45:58.706 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1553.ko 2024-09-12T12:45:58.707 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1672.ko 2024-09-12T12:45:58.707 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1742.ko 2024-09-12T12:45:58.707 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds2404.ko 2024-09-12T12:45:58.707 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds3232.ko 2024-09-12T12:45:58.707 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-em3027.ko 2024-09-12T12:45:58.707 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-fm3130.ko 2024-09-12T12:45:58.707 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-isl12022.ko 2024-09-12T12:45:58.707 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-isl1208.ko 2024-09-12T12:45:58.707 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m41t80.ko 2024-09-12T12:45:58.707 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m48t35.ko 2024-09-12T12:45:58.707 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m48t59.ko 2024-09-12T12:45:58.707 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-max6900.ko 2024-09-12T12:45:58.707 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-msm6242.ko 2024-09-12T12:45:58.708 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf2127.ko 2024-09-12T12:45:58.708 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf85063.ko 2024-09-12T12:45:58.708 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8523.ko 2024-09-12T12:45:58.708 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8563.ko 2024-09-12T12:45:58.708 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8583.ko 2024-09-12T12:45:58.708 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rp5c01.ko 2024-09-12T12:45:58.708 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rs5c372.ko 2024-09-12T12:45:58.708 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-09-12T12:45:58.708 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rx8025.ko 2024-09-12T12:45:58.708 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rx8581.ko 2024-09-12T12:45:58.708 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-09-12T12:45:58.708 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-x1205.ko 2024-09-12T12:45:58.708 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi 2024-09-12T12:45:58.709 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-9xxx.ko 2024-09-12T12:45:58.709 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-sas.ko 2024-09-12T12:45:58.709 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-xxxx.ko 2024-09-12T12:45:58.709 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/BusLogic.ko 2024-09-12T12:45:58.709 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/a100u2w.ko 2024-09-12T12:45:58.709 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aacraid 2024-09-12T12:45:58.709 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aacraid/aacraid.ko 2024-09-12T12:45:58.709 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/advansys.ko 2024-09-12T12:45:58.709 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx 2024-09-12T12:45:58.709 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-09-12T12:45:58.709 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-09-12T12:45:58.709 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/am53c974.ko 2024-09-12T12:45:58.710 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/arcmsr 2024-09-12T12:45:58.710 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-09-12T12:45:58.710 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/atp870u.ko 2024-09-12T12:45:58.710 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/be2iscsi 2024-09-12T12:45:58.710 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-09-12T12:45:58.710 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bfa 2024-09-12T12:45:58.710 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bfa/bfa.ko 2024-09-12T12:45:58.710 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2fc 2024-09-12T12:45:58.710 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-09-12T12:45:58.710 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2i 2024-09-12T12:45:58.710 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-09-12T12:45:58.710 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ch.ko 2024-09-12T12:45:58.710 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/csiostor 2024-09-12T12:45:58.710 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/csiostor/csiostor.ko 2024-09-12T12:45:58.711 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi 2024-09-12T12:45:58.711 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb3i 2024-09-12T12:45:58.711 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-09-12T12:45:58.711 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb4i 2024-09-12T12:45:58.711 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-09-12T12:45:58.711 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-09-12T12:45:58.711 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/dc395x.ko 2024-09-12T12:45:58.711 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler 2024-09-12T12:45:58.711 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-09-12T12:45:58.711 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-09-12T12:45:58.711 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-09-12T12:45:58.711 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-09-12T12:45:58.711 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/dmx3191d.ko 2024-09-12T12:45:58.712 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esas2r 2024-09-12T12:45:58.712 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esas2r/esas2r.ko 2024-09-12T12:45:58.712 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esp_scsi.ko 2024-09-12T12:45:58.712 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe 2024-09-12T12:45:58.712 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe/fcoe.ko 2024-09-12T12:45:58.712 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-09-12T12:45:58.712 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fnic 2024-09-12T12:45:58.712 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fnic/fnic.ko 2024-09-12T12:45:58.712 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hpsa.ko 2024-09-12T12:45:58.712 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hptiop.ko 2024-09-12T12:45:58.712 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hv_storvsc.ko 2024-09-12T12:45:58.712 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/initio.ko 2024-09-12T12:45:58.713 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ipr.ko 2024-09-12T12:45:58.713 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ips.ko 2024-09-12T12:45:58.713 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/isci 2024-09-12T12:45:58.713 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/isci/isci.ko 2024-09-12T12:45:58.713 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-09-12T12:45:58.713 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/iscsi_tcp.ko 2024-09-12T12:45:58.713 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libfc 2024-09-12T12:45:58.713 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libfc/libfc.ko 2024-09-12T12:45:58.713 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libiscsi.ko 2024-09-12T12:45:58.713 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libiscsi_tcp.ko 2024-09-12T12:45:58.713 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libsas 2024-09-12T12:45:58.713 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libsas/libsas.ko 2024-09-12T12:45:58.713 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/lpfc 2024-09-12T12:45:58.714 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/lpfc/lpfc.ko 2024-09-12T12:45:58.714 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid 2024-09-12T12:45:58.714 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid.ko 2024-09-12T12:45:58.714 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-09-12T12:45:58.714 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-09-12T12:45:58.714 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-09-12T12:45:58.714 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mpt3sas 2024-09-12T12:45:58.714 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-09-12T12:45:58.714 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvsas 2024-09-12T12:45:58.714 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvsas/mvsas.ko 2024-09-12T12:45:58.714 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvumi.ko 2024-09-12T12:45:58.714 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pm8001 2024-09-12T12:45:58.714 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-09-12T12:45:58.715 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pmcraid.ko 2024-09-12T12:45:58.715 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla1280.ko 2024-09-12T12:45:58.715 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx 2024-09-12T12:45:58.715 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-09-12T12:45:58.715 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-09-12T12:45:58.715 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla4xxx 2024-09-12T12:45:58.715 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-09-12T12:45:58.715 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/raid_class.ko 2024-09-12T12:45:58.715 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_debug.ko 2024-09-12T12:45:58.715 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_fc.ko 2024-09-12T12:45:58.715 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-09-12T12:45:58.715 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_sas.ko 2024-09-12T12:45:58.716 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_spi.ko 2024-09-12T12:45:58.716 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_srp.ko 2024-09-12T12:45:58.716 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ses.ko 2024-09-12T12:45:58.716 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/st.ko 2024-09-12T12:45:58.716 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/stex.ko 2024-09-12T12:45:58.716 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/sym53c8xx_2 2024-09-12T12:45:58.716 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-09-12T12:45:58.716 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/virtio_scsi.ko 2024-09-12T12:45:58.716 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/vmw_pvscsi.ko 2024-09-12T12:45:58.716 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/wd719x.ko 2024-09-12T12:45:58.716 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ssb 2024-09-12T12:45:58.716 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ssb/ssb.ko 2024-09-12T12:45:58.716 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target 2024-09-12T12:45:58.717 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/iscsi 2024-09-12T12:45:58.717 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-09-12T12:45:58.717 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/loopback 2024-09-12T12:45:58.717 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/loopback/tcm_loop.ko 2024-09-12T12:45:58.717 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_file.ko 2024-09-12T12:45:58.717 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_iblock.ko 2024-09-12T12:45:58.717 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_mod.ko 2024-09-12T12:45:58.717 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_pscsi.ko 2024-09-12T12:45:58.717 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_user.ko 2024-09-12T12:45:58.717 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/tcm_fc 2024-09-12T12:45:58.717 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-09-12T12:45:58.717 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal 2024-09-12T12:45:58.717 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel 2024-09-12T12:45:58.718 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal 2024-09-12T12:45:58.718 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-09-12T12:45:58.718 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-09-12T12:45:58.718 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-09-12T12:45:58.718 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-09-12T12:45:58.718 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-09-12T12:45:58.718 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-09-12T12:45:58.718 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-09-12T12:45:58.718 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-09-12T12:45:58.718 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-09-12T12:45:58.718 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-09-12T12:45:58.718 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-09-12T12:45:58.719 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-09-12T12:45:58.719 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-09-12T12:45:58.719 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-09-12T12:45:58.719 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-09-12T12:45:58.719 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-09-12T12:45:58.719 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-09-12T12:45:58.719 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-09-12T12:45:58.719 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty 2024-09-12T12:45:58.719 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/ipwireless 2024-09-12T12:45:58.719 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-09-12T12:45:58.719 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/n_gsm.ko 2024-09-12T12:45:58.720 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/n_hdlc.ko 2024-09-12T12:45:58.720 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/nozomi.ko 2024-09-12T12:45:58.720 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial 2024-09-12T12:45:58.720 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/8250 2024-09-12T12:45:58.720 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-09-12T12:45:58.720 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/arc_uart.ko 2024-09-12T12:45:58.720 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/jsm 2024-09-12T12:45:58.720 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/jsm/jsm.ko 2024-09-12T12:45:58.720 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs 2024-09-12T12:45:58.720 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/core 2024-09-12T12:45:58.720 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/core/ufshcd-core.ko 2024-09-12T12:45:58.720 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/host 2024-09-12T12:45:58.720 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-09-12T12:45:58.721 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio 2024-09-12T12:45:58.721 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio.ko 2024-09-12T12:45:58.721 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_aec.ko 2024-09-12T12:45:58.721 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_cif.ko 2024-09-12T12:45:58.721 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_pci_generic.ko 2024-09-12T12:45:58.721 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_sercos3.ko 2024-09-12T12:45:58.721 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb 2024-09-12T12:45:58.721 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class 2024-09-12T12:45:58.721 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/cdc-acm.ko 2024-09-12T12:45:58.721 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/cdc-wdm.ko 2024-09-12T12:45:58.721 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/usblp.ko 2024-09-12T12:45:58.721 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/usbtmc.ko 2024-09-12T12:45:58.721 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host 2024-09-12T12:45:58.722 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/sl811-hcd.ko 2024-09-12T12:45:58.722 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image 2024-09-12T12:45:58.722 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image/mdc800.ko 2024-09-12T12:45:58.722 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image/microtek.ko 2024-09-12T12:45:58.722 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc 2024-09-12T12:45:58.722 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/adutux.ko 2024-09-12T12:45:58.722 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/appledisplay.ko 2024-09-12T12:45:58.722 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/emi26.ko 2024-09-12T12:45:58.722 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/emi62.ko 2024-09-12T12:45:58.722 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ezusb.ko 2024-09-12T12:45:58.722 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/idmouse.ko 2024-09-12T12:45:58.722 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/iowarrior.ko 2024-09-12T12:45:58.722 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/isight_firmware.ko 2024-09-12T12:45:58.722 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ldusb.ko 2024-09-12T12:45:58.723 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/legousbtower.ko 2024-09-12T12:45:58.723 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/sisusbvga 2024-09-12T12:45:58.723 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-09-12T12:45:58.723 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/trancevibrator.ko 2024-09-12T12:45:58.723 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usb3503.ko 2024-09-12T12:45:58.723 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usblcd.ko 2024-09-12T12:45:58.723 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usbsevseg.ko 2024-09-12T12:45:58.723 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/uss720.ko 2024-09-12T12:45:58.723 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/yurex.ko 2024-09-12T12:45:58.723 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy 2024-09-12T12:45:58.723 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy/phy-generic.ko 2024-09-12T12:45:58.723 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial 2024-09-12T12:45:58.723 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/aircable.ko 2024-09-12T12:45:58.723 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ark3116.ko 2024-09-12T12:45:58.724 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/belkin_sa.ko 2024-09-12T12:45:58.724 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ch341.ko 2024-09-12T12:45:58.724 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cp210x.ko 2024-09-12T12:45:58.724 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cyberjack.ko 2024-09-12T12:45:58.724 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cypress_m8.ko 2024-09-12T12:45:58.724 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/digi_acceleport.ko 2024-09-12T12:45:58.724 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/empeg.ko 2024-09-12T12:45:58.724 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ftdi_sio.ko 2024-09-12T12:45:58.724 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/garmin_gps.ko 2024-09-12T12:45:58.724 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/io_edgeport.ko 2024-09-12T12:45:58.724 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/io_ti.ko 2024-09-12T12:45:58.724 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ipaq.ko 2024-09-12T12:45:58.724 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ipw.ko 2024-09-12T12:45:58.725 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ir-usb.ko 2024-09-12T12:45:58.725 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-09-12T12:45:58.725 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/keyspan.ko 2024-09-12T12:45:58.725 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/keyspan_pda.ko 2024-09-12T12:45:58.725 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/kl5kusb105.ko 2024-09-12T12:45:58.725 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/kobil_sct.ko 2024-09-12T12:45:58.725 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mct_u232.ko 2024-09-12T12:45:58.725 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mos7720.ko 2024-09-12T12:45:58.725 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mos7840.ko 2024-09-12T12:45:58.725 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/navman.ko 2024-09-12T12:45:58.725 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/omninet.ko 2024-09-12T12:45:58.725 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/opticon.ko 2024-09-12T12:45:58.725 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/option.ko 2024-09-12T12:45:58.725 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/oti6858.ko 2024-09-12T12:45:58.726 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/pl2303.ko 2024-09-12T12:45:58.726 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/qcaux.ko 2024-09-12T12:45:58.726 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/qcserial.ko 2024-09-12T12:45:58.726 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/quatech2.ko 2024-09-12T12:45:58.726 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/safe_serial.ko 2024-09-12T12:45:58.726 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/sierra.ko 2024-09-12T12:45:58.726 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/spcp8x5.ko 2024-09-12T12:45:58.726 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ssu100.ko 2024-09-12T12:45:58.726 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/symbolserial.ko 2024-09-12T12:45:58.726 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-09-12T12:45:58.726 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-09-12T12:45:58.726 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb_debug.ko 2024-09-12T12:45:58.726 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb_wwan.ko 2024-09-12T12:45:58.727 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/visor.ko 2024-09-12T12:45:58.727 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/whiteheat.ko 2024-09-12T12:45:58.727 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/xsens_mt.ko 2024-09-12T12:45:58.727 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage 2024-09-12T12:45:58.727 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/uas.ko 2024-09-12T12:45:58.727 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-alauda.ko 2024-09-12T12:45:58.727 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-cypress.ko 2024-09-12T12:45:58.727 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-datafab.ko 2024-09-12T12:45:58.727 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-09-12T12:45:58.727 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-freecom.ko 2024-09-12T12:45:58.727 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-isd200.ko 2024-09-12T12:45:58.727 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-09-12T12:45:58.727 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-karma.ko 2024-09-12T12:45:58.727 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-onetouch.ko 2024-09-12T12:45:58.728 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-realtek.ko 2024-09-12T12:45:58.728 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-sddr09.ko 2024-09-12T12:45:58.728 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-sddr55.ko 2024-09-12T12:45:58.728 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-usbat.ko 2024-09-12T12:45:58.728 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/usb-storage.ko 2024-09-12T12:45:58.728 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip 2024-09-12T12:45:58.728 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/usbip-core.ko 2024-09-12T12:45:58.728 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/usbip-host.ko 2024-09-12T12:45:58.728 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-09-12T12:45:58.728 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio 2024-09-12T12:45:58.728 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci 2024-09-12T12:45:58.728 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-09-12T12:45:58.728 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci/vfio-pci.ko 2024-09-12T12:45:58.729 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/vfio.ko 2024-09-12T12:45:58.729 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-09-12T12:45:58.729 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost 2024-09-12T12:45:58.729 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost.ko 2024-09-12T12:45:58.729 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_iotlb.ko 2024-09-12T12:45:58.729 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_net.ko 2024-09-12T12:45:58.729 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_scsi.ko 2024-09-12T12:45:58.729 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video 2024-09-12T12:45:58.729 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight 2024-09-12T12:45:58.729 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/apple_bl.ko 2024-09-12T12:45:58.729 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/lcd.ko 2024-09-12T12:45:58.729 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/platform_lcd.ko 2024-09-12T12:45:58.729 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio 2024-09-12T12:45:58.730 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio.ko 2024-09-12T12:45:58.730 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_balloon.ko 2024-09-12T12:45:58.730 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_mmio.ko 2024-09-12T12:45:58.730 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_pci.ko 2024-09-12T12:45:58.730 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-09-12T12:45:58.730 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-09-12T12:45:58.730 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_ring.ko 2024-09-12T12:45:58.730 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1 2024-09-12T12:45:58.730 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters 2024-09-12T12:45:58.730 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/ds2482.ko 2024-09-12T12:45:58.730 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/ds2490.ko 2024-09-12T12:45:58.730 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves 2024-09-12T12:45:58.730 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-09-12T12:45:58.731 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-09-12T12:45:58.731 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-09-12T12:45:58.731 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-09-12T12:45:58.731 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-09-12T12:45:58.731 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-09-12T12:45:58.731 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-09-12T12:45:58.731 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-09-12T12:45:58.731 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-09-12T12:45:58.731 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_smem.ko 2024-09-12T12:45:58.731 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_therm.ko 2024-09-12T12:45:58.731 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/wire.ko 2024-09-12T12:45:58.731 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog 2024-09-12T12:45:58.731 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/alim1535_wdt.ko 2024-09-12T12:45:58.731 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/alim7101_wdt.ko 2024-09-12T12:45:58.732 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/f71808e_wdt.ko 2024-09-12T12:45:58.732 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/hpwdt.ko 2024-09-12T12:45:58.732 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/i6300esb.ko 2024-09-12T12:45:58.732 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-09-12T12:45:58.732 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/iTCO_wdt.ko 2024-09-12T12:45:58.732 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ib700wdt.ko 2024-09-12T12:45:58.732 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ibmasr.ko 2024-09-12T12:45:58.732 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-09-12T12:45:58.732 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/it8712f_wdt.ko 2024-09-12T12:45:58.732 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/it87_wdt.ko 2024-09-12T12:45:58.732 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/machzwd.ko 2024-09-12T12:45:58.732 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/nv_tco.ko 2024-09-12T12:45:58.733 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/pcwd_pci.ko 2024-09-12T12:45:58.733 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/pcwd_usb.ko 2024-09-12T12:45:58.733 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-09-12T12:45:58.733 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sch311x_wdt.ko 2024-09-12T12:45:58.733 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/softdog.ko 2024-09-12T12:45:58.733 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sp5100_tco.ko 2024-09-12T12:45:58.733 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/via_wdt.ko 2024-09-12T12:45:58.733 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-09-12T12:45:58.733 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83877f_wdt.ko 2024-09-12T12:45:58.733 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83977f_wdt.ko 2024-09-12T12:45:58.733 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/wdt_pci.ko 2024-09-12T12:45:58.733 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/xen_wdt.ko 2024-09-12T12:45:58.733 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen 2024-09-12T12:45:58.733 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-acpi-processor.ko 2024-09-12T12:45:58.734 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-evtchn.ko 2024-09-12T12:45:58.734 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-gntalloc.ko 2024-09-12T12:45:58.734 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-gntdev.ko 2024-09-12T12:45:58.734 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-pciback 2024-09-12T12:45:58.734 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-09-12T12:45:58.734 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-privcmd.ko 2024-09-12T12:45:58.734 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xenfs 2024-09-12T12:45:58.734 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xenfs/xenfs.ko 2024-09-12T12:45:58.734 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs 2024-09-12T12:45:58.734 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/9p 2024-09-12T12:45:58.734 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/9p/9p.ko 2024-09-12T12:45:58.734 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/affs 2024-09-12T12:45:58.734 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/affs/affs.ko 2024-09-12T12:45:58.735 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/befs 2024-09-12T12:45:58.735 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/befs/befs.ko 2024-09-12T12:45:58.735 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/binfmt_misc.ko 2024-09-12T12:45:58.735 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/btrfs 2024-09-12T12:45:58.735 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/btrfs/btrfs.ko 2024-09-12T12:45:58.735 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cachefiles 2024-09-12T12:45:58.735 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cachefiles/cachefiles.ko 2024-09-12T12:45:58.735 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ceph 2024-09-12T12:45:58.735 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ceph/ceph.ko 2024-09-12T12:45:58.735 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/coda 2024-09-12T12:45:58.735 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/coda/coda.ko 2024-09-12T12:45:58.735 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cramfs 2024-09-12T12:45:58.735 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cramfs/cramfs.ko 2024-09-12T12:45:58.735 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/dlm 2024-09-12T12:45:58.736 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/dlm/dlm.ko 2024-09-12T12:45:58.736 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ecryptfs 2024-09-12T12:45:58.736 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ecryptfs/ecryptfs.ko 2024-09-12T12:45:58.736 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/f2fs 2024-09-12T12:45:58.736 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/f2fs/f2fs.ko 2024-09-12T12:45:58.736 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat 2024-09-12T12:45:58.736 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat/fat.ko 2024-09-12T12:45:58.736 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat/msdos.ko 2024-09-12T12:45:58.736 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat/vfat.ko 2024-09-12T12:45:58.736 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fuse 2024-09-12T12:45:58.736 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fuse/cuse.ko 2024-09-12T12:45:58.736 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fuse/fuse.ko 2024-09-12T12:45:58.736 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/gfs2 2024-09-12T12:45:58.737 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/gfs2/gfs2.ko 2024-09-12T12:45:58.737 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfs 2024-09-12T12:45:58.737 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfs/hfs.ko 2024-09-12T12:45:58.737 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfsplus 2024-09-12T12:45:58.737 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfsplus/hfsplus.ko 2024-09-12T12:45:58.737 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/isofs 2024-09-12T12:45:58.737 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/isofs/isofs.ko 2024-09-12T12:45:58.737 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/jfs 2024-09-12T12:45:58.737 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/jfs/jfs.ko 2024-09-12T12:45:58.737 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/lockd 2024-09-12T12:45:58.737 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/lockd/lockd.ko 2024-09-12T12:45:58.737 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/minix 2024-09-12T12:45:58.737 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/minix/minix.ko 2024-09-12T12:45:58.738 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/netfs 2024-09-12T12:45:58.738 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/netfs/netfs.ko 2024-09-12T12:45:58.738 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs 2024-09-12T12:45:58.738 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/blocklayout 2024-09-12T12:45:58.738 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-09-12T12:45:58.738 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/filelayout 2024-09-12T12:45:58.738 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-09-12T12:45:58.738 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/flexfilelayout 2024-09-12T12:45:58.738 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-09-12T12:45:58.738 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfs.ko 2024-09-12T12:45:58.738 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfsv3.ko 2024-09-12T12:45:58.738 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfsv4.ko 2024-09-12T12:45:58.738 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common 2024-09-12T12:45:58.738 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common/grace.ko 2024-09-12T12:45:58.739 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common/nfs_acl.ko 2024-09-12T12:45:58.739 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfsd 2024-09-12T12:45:58.739 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfsd/nfsd.ko 2024-09-12T12:45:58.739 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nilfs2 2024-09-12T12:45:58.739 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nilfs2/nilfs2.ko 2024-09-12T12:45:58.739 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls 2024-09-12T12:45:58.739 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-celtic.ko 2024-09-12T12:45:58.739 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-centeuro.ko 2024-09-12T12:45:58.739 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-croatian.ko 2024-09-12T12:45:58.739 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-cyrillic.ko 2024-09-12T12:45:58.739 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-gaelic.ko 2024-09-12T12:45:58.739 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-greek.ko 2024-09-12T12:45:58.739 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-iceland.ko 2024-09-12T12:45:58.740 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-inuit.ko 2024-09-12T12:45:58.740 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-roman.ko 2024-09-12T12:45:58.740 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-romanian.ko 2024-09-12T12:45:58.740 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-turkish.ko 2024-09-12T12:45:58.740 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1250.ko 2024-09-12T12:45:58.740 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1251.ko 2024-09-12T12:45:58.740 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1255.ko 2024-09-12T12:45:58.740 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp737.ko 2024-09-12T12:45:58.740 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp775.ko 2024-09-12T12:45:58.740 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp850.ko 2024-09-12T12:45:58.740 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp852.ko 2024-09-12T12:45:58.740 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp855.ko 2024-09-12T12:45:58.740 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp857.ko 2024-09-12T12:45:58.741 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp860.ko 2024-09-12T12:45:58.741 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp861.ko 2024-09-12T12:45:58.741 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp862.ko 2024-09-12T12:45:58.741 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp863.ko 2024-09-12T12:45:58.741 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp864.ko 2024-09-12T12:45:58.741 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp865.ko 2024-09-12T12:45:58.741 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp866.ko 2024-09-12T12:45:58.741 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp869.ko 2024-09-12T12:45:58.741 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp874.ko 2024-09-12T12:45:58.741 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp932.ko 2024-09-12T12:45:58.741 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp936.ko 2024-09-12T12:45:58.741 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp949.ko 2024-09-12T12:45:58.742 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp950.ko 2024-09-12T12:45:58.742 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_euc-jp.ko 2024-09-12T12:45:58.742 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-1.ko 2024-09-12T12:45:58.742 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-13.ko 2024-09-12T12:45:58.742 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-14.ko 2024-09-12T12:45:58.742 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-15.ko 2024-09-12T12:45:58.742 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-2.ko 2024-09-12T12:45:58.742 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-3.ko 2024-09-12T12:45:58.742 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-4.ko 2024-09-12T12:45:58.742 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-5.ko 2024-09-12T12:45:58.742 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-6.ko 2024-09-12T12:45:58.742 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-7.ko 2024-09-12T12:45:58.742 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-9.ko 2024-09-12T12:45:58.743 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-r.ko 2024-09-12T12:45:58.743 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-ru.ko 2024-09-12T12:45:58.743 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-u.ko 2024-09-12T12:45:58.743 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_ucs2_utils.ko 2024-09-12T12:45:58.743 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_utf8.ko 2024-09-12T12:45:58.743 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2 2024-09-12T12:45:58.743 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/cluster 2024-09-12T12:45:58.743 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-09-12T12:45:58.743 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlm 2024-09-12T12:45:58.743 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-09-12T12:45:58.743 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlmfs 2024-09-12T12:45:58.743 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-09-12T12:45:58.743 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2.ko 2024-09-12T12:45:58.744 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-09-12T12:45:58.744 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-09-12T12:45:58.744 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-09-12T12:45:58.744 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/overlayfs 2024-09-12T12:45:58.744 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/overlayfs/overlay.ko 2024-09-12T12:45:58.744 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/pstore 2024-09-12T12:45:58.744 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/pstore/ramoops.ko 2024-09-12T12:45:58.744 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/reiserfs 2024-09-12T12:45:58.744 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/reiserfs/reiserfs.ko 2024-09-12T12:45:58.744 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/romfs 2024-09-12T12:45:58.744 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/romfs/romfs.ko 2024-09-12T12:45:58.744 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb 2024-09-12T12:45:58.745 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/client 2024-09-12T12:45:58.745 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/client/cifs.ko 2024-09-12T12:45:58.745 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common 2024-09-12T12:45:58.745 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common/cifs_arc4.ko 2024-09-12T12:45:58.745 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common/cifs_md4.ko 2024-09-12T12:45:58.745 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/squashfs 2024-09-12T12:45:58.745 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/squashfs/squashfs.ko 2024-09-12T12:45:58.745 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/sysv 2024-09-12T12:45:58.745 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/sysv/sysv.ko 2024-09-12T12:45:58.745 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/udf 2024-09-12T12:45:58.745 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/udf/udf.ko 2024-09-12T12:45:58.745 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ufs 2024-09-12T12:45:58.745 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ufs/ufs.ko 2024-09-12T12:45:58.746 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/xfs 2024-09-12T12:45:58.746 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/xfs/xfs.ko 2024-09-12T12:45:58.746 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel 2024-09-12T12:45:58.746 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/rcu 2024-09-12T12:45:58.746 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/rcu/rcutorture.ko 2024-09-12T12:45:58.746 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/torture.ko 2024-09-12T12:45:58.746 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/trace 2024-09-12T12:45:58.746 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-09-12T12:45:58.746 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib 2024-09-12T12:45:58.746 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/asn1_encoder.ko 2024-09-12T12:45:58.746 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crc-itu-t.ko 2024-09-12T12:45:58.746 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crc8.ko 2024-09-12T12:45:58.747 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto 2024-09-12T12:45:58.747 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/libaescfb.ko 2024-09-12T12:45:58.747 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/libarc4.ko 2024-09-12T12:45:58.747 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/libdes.ko 2024-09-12T12:45:58.747 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/libcrc32c.ko 2024-09-12T12:45:58.747 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lru_cache.ko 2024-09-12T12:45:58.747 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4 2024-09-12T12:45:58.747 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4/lz4_compress.ko 2024-09-12T12:45:58.747 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4/lz4hc_compress.ko 2024-09-12T12:45:58.747 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/math 2024-09-12T12:45:58.747 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/math/cordic.ko 2024-09-12T12:45:58.747 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/raid6 2024-09-12T12:45:58.747 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/raid6/raid6_pq.ko 2024-09-12T12:45:58.747 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/reed_solomon 2024-09-12T12:45:58.748 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/reed_solomon/reed_solomon.ko 2024-09-12T12:45:58.748 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_bm.ko 2024-09-12T12:45:58.748 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_fsm.ko 2024-09-12T12:45:58.748 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_kmp.ko 2024-09-12T12:45:58.748 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/zstd 2024-09-12T12:45:58.748 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/zstd/zstd_compress.ko 2024-09-12T12:45:58.748 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/mm 2024-09-12T12:45:58.748 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/mm/hwpoison-inject.ko 2024-09-12T12:45:58.748 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net 2024-09-12T12:45:58.748 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan 2024-09-12T12:45:58.748 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/6lowpan.ko 2024-09-12T12:45:58.748 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_dest.ko 2024-09-12T12:45:58.748 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_fragment.ko 2024-09-12T12:45:58.749 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_hop.ko 2024-09-12T12:45:58.749 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_ipv6.ko 2024-09-12T12:45:58.749 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_mobility.ko 2024-09-12T12:45:58.749 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_routing.ko 2024-09-12T12:45:58.749 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_udp.ko 2024-09-12T12:45:58.749 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802 2024-09-12T12:45:58.749 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/garp.ko 2024-09-12T12:45:58.749 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/mrp.ko 2024-09-12T12:45:58.749 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/p8022.ko 2024-09-12T12:45:58.749 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/psnap.ko 2024-09-12T12:45:58.749 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/stp.ko 2024-09-12T12:45:58.749 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/8021q 2024-09-12T12:45:58.750 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/8021q/8021q.ko 2024-09-12T12:45:58.750 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p 2024-09-12T12:45:58.750 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet.ko 2024-09-12T12:45:58.750 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_fd.ko 2024-09-12T12:45:58.750 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_rdma.ko 2024-09-12T12:45:58.750 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_virtio.ko 2024-09-12T12:45:58.750 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/batman-adv 2024-09-12T12:45:58.750 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/batman-adv/batman-adv.ko 2024-09-12T12:45:58.750 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge 2024-09-12T12:45:58.750 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/br_netfilter.ko 2024-09-12T12:45:58.750 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/bridge.ko 2024-09-12T12:45:58.750 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter 2024-09-12T12:45:58.750 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-09-12T12:45:58.751 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_among.ko 2024-09-12T12:45:58.751 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_arp.ko 2024-09-12T12:45:58.751 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-09-12T12:45:58.751 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-09-12T12:45:58.751 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_ip.ko 2024-09-12T12:45:58.751 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-09-12T12:45:58.751 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_limit.ko 2024-09-12T12:45:58.751 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_log.ko 2024-09-12T12:45:58.751 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_mark.ko 2024-09-12T12:45:58.751 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-09-12T12:45:58.751 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-09-12T12:45:58.751 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-09-12T12:45:58.751 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-09-12T12:45:58.751 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_snat.ko 2024-09-12T12:45:58.752 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_stp.ko 2024-09-12T12:45:58.752 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-09-12T12:45:58.752 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-09-12T12:45:58.752 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-09-12T12:45:58.752 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-09-12T12:45:58.752 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtables.ko 2024-09-12T12:45:58.752 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-09-12T12:45:58.752 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-09-12T12:45:58.752 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-09-12T12:45:58.752 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ceph 2024-09-12T12:45:58.752 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ceph/libceph.ko 2024-09-12T12:45:58.752 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/core 2024-09-12T12:45:58.752 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/core/failover.ko 2024-09-12T12:45:58.753 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/core/pktgen.ko 2024-09-12T12:45:58.753 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp 2024-09-12T12:45:58.753 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp.ko 2024-09-12T12:45:58.753 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_diag.ko 2024-09-12T12:45:58.753 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_ipv4.ko 2024-09-12T12:45:58.753 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_ipv6.ko 2024-09-12T12:45:58.753 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dns_resolver 2024-09-12T12:45:58.753 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dns_resolver/dns_resolver.ko 2024-09-12T12:45:58.753 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa 2024-09-12T12:45:58.753 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/dsa_core.ko 2024-09-12T12:45:58.753 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_brcm.ko 2024-09-12T12:45:58.753 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_dsa.ko 2024-09-12T12:45:58.753 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_none.ko 2024-09-12T12:45:58.754 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_trailer.ko 2024-09-12T12:45:58.754 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154 2024-09-12T12:45:58.754 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/6lowpan 2024-09-12T12:45:58.754 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-09-12T12:45:58.754 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/ieee802154.ko 2024-09-12T12:45:58.754 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/ieee802154_socket.ko 2024-09-12T12:45:58.754 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4 2024-09-12T12:45:58.754 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ah4.ko 2024-09-12T12:45:58.754 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/esp4.ko 2024-09-12T12:45:58.754 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/fou.ko 2024-09-12T12:45:58.754 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/gre.ko 2024-09-12T12:45:58.754 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/inet_diag.ko 2024-09-12T12:45:58.754 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_gre.ko 2024-09-12T12:45:58.755 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_tunnel.ko 2024-09-12T12:45:58.755 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_vti.ko 2024-09-12T12:45:58.755 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ipcomp.ko 2024-09-12T12:45:58.755 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ipip.ko 2024-09-12T12:45:58.755 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter 2024-09-12T12:45:58.755 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arp_tables.ko 2024-09-12T12:45:58.755 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-09-12T12:45:58.755 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-09-12T12:45:58.755 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-09-12T12:45:58.755 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-09-12T12:45:58.755 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-09-12T12:45:58.755 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-09-12T12:45:58.756 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-09-12T12:45:58.756 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-09-12T12:45:58.756 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-09-12T12:45:58.756 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_security.ko 2024-09-12T12:45:58.756 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-09-12T12:45:58.756 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-09-12T12:45:58.756 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-09-12T12:45:58.756 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-09-12T12:45:58.756 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-09-12T12:45:58.756 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-09-12T12:45:58.756 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-09-12T12:45:58.756 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-09-12T12:45:58.756 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-09-12T12:45:58.757 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-09-12T12:45:58.757 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_bic.ko 2024-09-12T12:45:58.757 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_dctcp.ko 2024-09-12T12:45:58.757 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_diag.ko 2024-09-12T12:45:58.757 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_highspeed.ko 2024-09-12T12:45:58.757 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_htcp.ko 2024-09-12T12:45:58.757 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_hybla.ko 2024-09-12T12:45:58.757 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_illinois.ko 2024-09-12T12:45:58.757 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_lp.ko 2024-09-12T12:45:58.757 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_scalable.ko 2024-09-12T12:45:58.757 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_vegas.ko 2024-09-12T12:45:58.757 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_veno.ko 2024-09-12T12:45:58.757 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_westwood.ko 2024-09-12T12:45:58.758 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_yeah.ko 2024-09-12T12:45:58.758 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tunnel4.ko 2024-09-12T12:45:58.758 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/udp_diag.ko 2024-09-12T12:45:58.758 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/udp_tunnel.ko 2024-09-12T12:45:58.758 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/xfrm4_tunnel.ko 2024-09-12T12:45:58.758 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6 2024-09-12T12:45:58.758 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ah6.ko 2024-09-12T12:45:58.758 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/esp6.ko 2024-09-12T12:45:58.758 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/fou6.ko 2024-09-12T12:45:58.758 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_tunnel.ko 2024-09-12T12:45:58.758 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-09-12T12:45:58.758 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_vti.ko 2024-09-12T12:45:58.758 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ipcomp6.ko 2024-09-12T12:45:58.758 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter 2024-09-12T12:45:58.759 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-09-12T12:45:58.759 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-09-12T12:45:58.759 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-09-12T12:45:58.759 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-09-12T12:45:58.759 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-09-12T12:45:58.759 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-09-12T12:45:58.759 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-09-12T12:45:58.759 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-09-12T12:45:58.759 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-09-12T12:45:58.759 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-09-12T12:45:58.759 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-09-12T12:45:58.759 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-09-12T12:45:58.760 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-09-12T12:45:58.760 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-09-12T12:45:58.760 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-09-12T12:45:58.760 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-09-12T12:45:58.760 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-09-12T12:45:58.760 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-09-12T12:45:58.760 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-09-12T12:45:58.760 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-09-12T12:45:58.760 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-09-12T12:45:58.760 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-09-12T12:45:58.760 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-09-12T12:45:58.760 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-09-12T12:45:58.760 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/sit.ko 2024-09-12T12:45:58.761 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/tunnel6.ko 2024-09-12T12:45:58.761 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/xfrm6_tunnel.ko 2024-09-12T12:45:58.761 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/key 2024-09-12T12:45:58.761 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/key/af_key.ko 2024-09-12T12:45:58.761 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp 2024-09-12T12:45:58.761 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_core.ko 2024-09-12T12:45:58.761 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_debugfs.ko 2024-09-12T12:45:58.761 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_eth.ko 2024-09-12T12:45:58.761 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ip.ko 2024-09-12T12:45:58.761 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ip6.ko 2024-09-12T12:45:58.761 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_netlink.ko 2024-09-12T12:45:58.761 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ppp.ko 2024-09-12T12:45:58.761 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/llc 2024-09-12T12:45:58.762 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/llc/llc.ko 2024-09-12T12:45:58.762 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mac802154 2024-09-12T12:45:58.762 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mac802154/mac802154.ko 2024-09-12T12:45:58.762 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mpls 2024-09-12T12:45:58.762 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mpls/mpls_gso.ko 2024-09-12T12:45:58.762 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter 2024-09-12T12:45:58.762 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset 2024-09-12T12:45:58.762 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set.ko 2024-09-12T12:45:58.762 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-09-12T12:45:58.762 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-09-12T12:45:58.762 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-09-12T12:45:58.762 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-09-12T12:45:58.763 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-09-12T12:45:58.763 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-09-12T12:45:58.763 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-09-12T12:45:58.763 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-09-12T12:45:58.763 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-09-12T12:45:58.763 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-09-12T12:45:58.763 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-09-12T12:45:58.763 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-09-12T12:45:58.763 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-09-12T12:45:58.763 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-09-12T12:45:58.763 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-09-12T12:45:58.763 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs 2024-09-12T12:45:58.763 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs.ko 2024-09-12T12:45:58.764 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-09-12T12:45:58.764 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-09-12T12:45:58.764 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-09-12T12:45:58.764 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-09-12T12:45:58.764 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-09-12T12:45:58.764 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-09-12T12:45:58.764 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-09-12T12:45:58.764 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-09-12T12:45:58.764 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-09-12T12:45:58.764 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-09-12T12:45:58.764 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-09-12T12:45:58.764 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-09-12T12:45:58.764 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-09-12T12:45:58.765 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conncount.ko 2024-09-12T12:45:58.765 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack.ko 2024-09-12T12:45:58.765 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-09-12T12:45:58.765 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-09-12T12:45:58.765 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-09-12T12:45:58.765 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_h323.ko 2024-09-12T12:45:58.765 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_irc.ko 2024-09-12T12:45:58.765 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-09-12T12:45:58.765 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-09-12T12:45:58.765 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-09-12T12:45:58.765 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_sane.ko 2024-09-12T12:45:58.765 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_sip.ko 2024-09-12T12:45:58.765 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-09-12T12:45:58.766 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-09-12T12:45:58.766 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_dup_netdev.ko 2024-09-12T12:45:58.766 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_flow_table.ko 2024-09-12T12:45:58.766 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_flow_table_inet.ko 2024-09-12T12:45:58.766 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_log_syslog.ko 2024-09-12T12:45:58.766 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat.ko 2024-09-12T12:45:58.766 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_amanda.ko 2024-09-12T12:45:58.766 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_ftp.ko 2024-09-12T12:45:58.766 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_irc.ko 2024-09-12T12:45:58.766 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_sip.ko 2024-09-12T12:45:58.766 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_tftp.ko 2024-09-12T12:45:58.766 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_synproxy_core.ko 2024-09-12T12:45:58.766 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_tables.ko 2024-09-12T12:45:58.767 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink.ko 2024-09-12T12:45:58.767 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_acct.ko 2024-09-12T12:45:58.767 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_log.ko 2024-09-12T12:45:58.767 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_osf.ko 2024-09-12T12:45:58.767 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_queue.ko 2024-09-12T12:45:58.767 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_chain_nat.ko 2024-09-12T12:45:58.767 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_compat.ko 2024-09-12T12:45:58.767 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_ct.ko 2024-09-12T12:45:58.767 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_dup_netdev.ko 2024-09-12T12:45:58.767 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_fib.ko 2024-09-12T12:45:58.767 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_fwd_netdev.ko 2024-09-12T12:45:58.767 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_hash.ko 2024-09-12T12:45:58.767 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_limit.ko 2024-09-12T12:45:58.768 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_log.ko 2024-09-12T12:45:58.768 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_masq.ko 2024-09-12T12:45:58.768 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_nat.ko 2024-09-12T12:45:58.768 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_numgen.ko 2024-09-12T12:45:58.768 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_queue.ko 2024-09-12T12:45:58.768 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_quota.ko 2024-09-12T12:45:58.768 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_redir.ko 2024-09-12T12:45:58.768 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_reject.ko 2024-09-12T12:45:58.768 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_reject_inet.ko 2024-09-12T12:45:58.768 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_socket.ko 2024-09-12T12:45:58.768 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_tunnel.ko 2024-09-12T12:45:58.768 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_AUDIT.ko 2024-09-12T12:45:58.768 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CHECKSUM.ko 2024-09-12T12:45:58.768 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CLASSIFY.ko 2024-09-12T12:45:58.769 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-09-12T12:45:58.769 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CT.ko 2024-09-12T12:45:58.769 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_DSCP.ko 2024-09-12T12:45:58.769 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_HL.ko 2024-09-12T12:45:58.769 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_HMARK.ko 2024-09-12T12:45:58.769 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_IDLETIMER.ko 2024-09-12T12:45:58.769 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_LED.ko 2024-09-12T12:45:58.769 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_LOG.ko 2024-09-12T12:45:58.769 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_MASQUERADE.ko 2024-09-12T12:45:58.769 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NETMAP.ko 2024-09-12T12:45:58.769 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NFLOG.ko 2024-09-12T12:45:58.769 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NFQUEUE.ko 2024-09-12T12:45:58.769 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_RATEEST.ko 2024-09-12T12:45:58.770 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_REDIRECT.ko 2024-09-12T12:45:58.770 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_SECMARK.ko 2024-09-12T12:45:58.770 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TCPMSS.ko 2024-09-12T12:45:58.770 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-09-12T12:45:58.770 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TEE.ko 2024-09-12T12:45:58.770 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TPROXY.ko 2024-09-12T12:45:58.770 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TRACE.ko 2024-09-12T12:45:58.770 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_addrtype.ko 2024-09-12T12:45:58.770 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_bpf.ko 2024-09-12T12:45:58.770 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cgroup.ko 2024-09-12T12:45:58.770 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cluster.ko 2024-09-12T12:45:58.770 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_comment.ko 2024-09-12T12:45:58.770 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connbytes.ko 2024-09-12T12:45:58.770 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connlabel.ko 2024-09-12T12:45:58.771 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connlimit.ko 2024-09-12T12:45:58.771 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connmark.ko 2024-09-12T12:45:58.771 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_conntrack.ko 2024-09-12T12:45:58.771 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cpu.ko 2024-09-12T12:45:58.771 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_dccp.ko 2024-09-12T12:45:58.771 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_devgroup.ko 2024-09-12T12:45:58.771 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_dscp.ko 2024-09-12T12:45:58.771 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ecn.ko 2024-09-12T12:45:58.771 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_esp.ko 2024-09-12T12:45:58.771 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_hashlimit.ko 2024-09-12T12:45:58.771 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_helper.ko 2024-09-12T12:45:58.771 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_hl.ko 2024-09-12T12:45:58.771 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ipcomp.ko 2024-09-12T12:45:58.771 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_iprange.ko 2024-09-12T12:45:58.772 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ipvs.ko 2024-09-12T12:45:58.772 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_l2tp.ko 2024-09-12T12:45:58.772 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_length.ko 2024-09-12T12:45:58.772 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_limit.ko 2024-09-12T12:45:58.772 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_mac.ko 2024-09-12T12:45:58.772 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_mark.ko 2024-09-12T12:45:58.772 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_multiport.ko 2024-09-12T12:45:58.772 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_nat.ko 2024-09-12T12:45:58.772 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_nfacct.ko 2024-09-12T12:45:58.772 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_osf.ko 2024-09-12T12:45:58.772 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_owner.ko 2024-09-12T12:45:58.772 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_physdev.ko 2024-09-12T12:45:58.773 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_pkttype.ko 2024-09-12T12:45:58.773 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_policy.ko 2024-09-12T12:45:58.773 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_quota.ko 2024-09-12T12:45:58.773 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_rateest.ko 2024-09-12T12:45:58.773 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_realm.ko 2024-09-12T12:45:58.773 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_recent.ko 2024-09-12T12:45:58.773 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_sctp.ko 2024-09-12T12:45:58.773 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_set.ko 2024-09-12T12:45:58.773 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_socket.ko 2024-09-12T12:45:58.773 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_state.ko 2024-09-12T12:45:58.773 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_statistic.ko 2024-09-12T12:45:58.773 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_string.ko 2024-09-12T12:45:58.774 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_tcpmss.ko 2024-09-12T12:45:58.774 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_time.ko 2024-09-12T12:45:58.774 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_u32.ko 2024-09-12T12:45:58.774 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netlink 2024-09-12T12:45:58.774 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netlink/netlink_diag.ko 2024-09-12T12:45:58.774 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/nsh 2024-09-12T12:45:58.774 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/nsh/nsh.ko 2024-09-12T12:45:58.774 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch 2024-09-12T12:45:58.774 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/openvswitch.ko 2024-09-12T12:45:58.774 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-geneve.ko 2024-09-12T12:45:58.774 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-gre.ko 2024-09-12T12:45:58.774 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-vxlan.ko 2024-09-12T12:45:58.774 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/packet 2024-09-12T12:45:58.774 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/packet/af_packet_diag.ko 2024-09-12T12:45:58.775 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds 2024-09-12T12:45:58.775 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds.ko 2024-09-12T12:45:58.775 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds_rdma.ko 2024-09-12T12:45:58.775 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds_tcp.ko 2024-09-12T12:45:58.775 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched 2024-09-12T12:45:58.775 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_csum.ko 2024-09-12T12:45:58.775 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_gact.ko 2024-09-12T12:45:58.775 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_mirred.ko 2024-09-12T12:45:58.775 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_nat.ko 2024-09-12T12:45:58.775 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_pedit.ko 2024-09-12T12:45:58.775 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_police.ko 2024-09-12T12:45:58.775 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_simple.ko 2024-09-12T12:45:58.775 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_skbedit.ko 2024-09-12T12:45:58.775 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_vlan.ko 2024-09-12T12:45:58.776 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_basic.ko 2024-09-12T12:45:58.776 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_bpf.ko 2024-09-12T12:45:58.776 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_flow.ko 2024-09-12T12:45:58.776 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_fw.ko 2024-09-12T12:45:58.776 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_route.ko 2024-09-12T12:45:58.776 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_u32.ko 2024-09-12T12:45:58.776 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_cmp.ko 2024-09-12T12:45:58.776 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_ipset.ko 2024-09-12T12:45:58.776 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_meta.ko 2024-09-12T12:45:58.776 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_nbyte.ko 2024-09-12T12:45:58.776 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_text.ko 2024-09-12T12:45:58.776 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_u32.ko 2024-09-12T12:45:58.776 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_choke.ko 2024-09-12T12:45:58.776 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_codel.ko 2024-09-12T12:45:58.777 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_drr.ko 2024-09-12T12:45:58.777 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_fq.ko 2024-09-12T12:45:58.777 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_gred.ko 2024-09-12T12:45:58.777 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_hfsc.ko 2024-09-12T12:45:58.777 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_hhf.ko 2024-09-12T12:45:58.777 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_htb.ko 2024-09-12T12:45:58.777 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_ingress.ko 2024-09-12T12:45:58.777 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_mqprio.ko 2024-09-12T12:45:58.777 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_mqprio_lib.ko 2024-09-12T12:45:58.777 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_multiq.ko 2024-09-12T12:45:58.777 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_netem.ko 2024-09-12T12:45:58.777 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_pie.ko 2024-09-12T12:45:58.778 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_plug.ko 2024-09-12T12:45:58.778 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_prio.ko 2024-09-12T12:45:58.778 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_qfq.ko 2024-09-12T12:45:58.778 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_red.ko 2024-09-12T12:45:58.778 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_sfb.ko 2024-09-12T12:45:58.778 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_sfq.ko 2024-09-12T12:45:58.778 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_tbf.ko 2024-09-12T12:45:58.778 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_teql.ko 2024-09-12T12:45:58.778 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp 2024-09-12T12:45:58.778 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp/sctp.ko 2024-09-12T12:45:58.778 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp/sctp_diag.ko 2024-09-12T12:45:58.778 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc 2024-09-12T12:45:58.778 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss 2024-09-12T12:45:58.778 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-09-12T12:45:58.779 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-09-12T12:45:58.779 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/sunrpc.ko 2024-09-12T12:45:58.779 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/xprtrdma 2024-09-12T12:45:58.779 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-09-12T12:45:58.779 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc 2024-09-12T12:45:58.779 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc/tipc.ko 2024-09-12T12:45:58.779 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc/tipc_diag.ko 2024-09-12T12:45:58.779 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/unix 2024-09-12T12:45:58.779 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/unix/unix_diag.ko 2024-09-12T12:45:58.779 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock 2024-09-12T12:45:58.779 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-09-12T12:45:58.779 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-09-12T12:45:58.779 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock.ko 2024-09-12T12:45:58.779 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock_diag.ko 2024-09-12T12:45:58.780 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock_loopback.ko 2024-09-12T12:45:58.780 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm 2024-09-12T12:45:58.780 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm/xfrm_ipcomp.ko 2024-09-12T12:45:58.780 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security 2024-09-12T12:45:58.780 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys 2024-09-12T12:45:58.780 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys/encrypted-keys 2024-09-12T12:45:58.780 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-09-12T12:45:58.780 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys/trusted-keys 2024-09-12T12:45:58.780 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys/trusted-keys/trusted.ko 2024-09-12T12:45:58.780 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.alias 2024-09-12T12:45:58.780 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.alias.bin 2024-09-12T12:45:58.780 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin 2024-09-12T12:45:58.780 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.alias.bin 2024-09-12T12:45:58.781 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.bin 2024-09-12T12:45:58.781 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.modinfo 2024-09-12T12:45:58.781 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.dep 2024-09-12T12:45:58.781 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.dep.bin 2024-09-12T12:45:58.781 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.devname 2024-09-12T12:45:58.781 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.order 2024-09-12T12:45:58.781 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.softdep 2024-09-12T12:45:58.781 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.symbols 2024-09-12T12:45:58.781 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.symbols.bin 2024-09-12T12:45:58.781 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.weakdep 2024-09-12T12:45:58.781 INFO:teuthology.orchestra.run.smithi060.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/vmlinuz 2024-09-12T12:45:58.785 DEBUG:teuthology.task.kernel:get_image_version: 6.11.0-rc5-g2ca44a2f35ca 2024-09-12T12:45:58.786 DEBUG:teuthology.orchestra.run.smithi060:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-09-12T12:45:58.809 INFO:teuthology.orchestra.run.smithi060.stdout:postinstall scriptlet (using /bin/sh): 2024-09-12T12:45:58.809 INFO:teuthology.orchestra.run.smithi060.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-09-12T12:45:58.809 INFO:teuthology.orchestra.run.smithi060.stdout: /usr/bin/kernel-install add 6.11.0-rc5-g2ca44a2f35ca /lib/modules/6.11.0-rc5-g2ca44a2f35ca/vmlinuz 2024-09-12T12:45:58.809 INFO:teuthology.orchestra.run.smithi060.stdout:fi 2024-09-12T12:45:58.810 INFO:teuthology.orchestra.run.smithi060.stdout:for file in vmlinuz System.map config; do 2024-09-12T12:45:58.810 INFO:teuthology.orchestra.run.smithi060.stdout: if ! cmp --silent "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/${file}" "/boot/${file}-6.11.0-rc5-g2ca44a2f35ca"; then 2024-09-12T12:45:58.810 INFO:teuthology.orchestra.run.smithi060.stdout: cp "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/${file}" "/boot/${file}-6.11.0-rc5-g2ca44a2f35ca" 2024-09-12T12:45:58.810 INFO:teuthology.orchestra.run.smithi060.stdout: fi 2024-09-12T12:45:58.810 INFO:teuthology.orchestra.run.smithi060.stdout:done 2024-09-12T12:45:58.810 INFO:teuthology.orchestra.run.smithi060.stdout:if [ -d "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/dtb" ] && \ 2024-09-12T12:45:58.810 INFO:teuthology.orchestra.run.smithi060.stdout: ! diff -rq "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/dtb" "/boot/dtb-6.11.0-rc5-g2ca44a2f35ca" >/dev/null 2>&1; then 2024-09-12T12:45:58.810 INFO:teuthology.orchestra.run.smithi060.stdout: rm -rf "/boot/dtb-6.11.0-rc5-g2ca44a2f35ca" 2024-09-12T12:45:58.810 INFO:teuthology.orchestra.run.smithi060.stdout: cp -r "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/dtb" "/boot/dtb-6.11.0-rc5-g2ca44a2f35ca" 2024-09-12T12:45:58.810 INFO:teuthology.orchestra.run.smithi060.stdout:fi 2024-09-12T12:45:58.810 INFO:teuthology.orchestra.run.smithi060.stdout:if [ ! -e "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.dep" ]; then 2024-09-12T12:45:58.810 INFO:teuthology.orchestra.run.smithi060.stdout: /usr/sbin/depmod 6.11.0-rc5-g2ca44a2f35ca 2024-09-12T12:45:58.810 INFO:teuthology.orchestra.run.smithi060.stdout:fi 2024-09-12T12:45:58.811 INFO:teuthology.orchestra.run.smithi060.stdout:preuninstall scriptlet (using /bin/sh): 2024-09-12T12:45:58.811 INFO:teuthology.orchestra.run.smithi060.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-09-12T12:45:58.811 INFO:teuthology.orchestra.run.smithi060.stdout:kernel-install remove 6.11.0-rc5-g2ca44a2f35ca 2024-09-12T12:45:58.811 INFO:teuthology.orchestra.run.smithi060.stdout:fi 2024-09-12T12:45:58.812 DEBUG:teuthology.orchestra.run.smithi060:> sudo rpm -qi grub2-tools 2024-09-12T12:45:58.891 INFO:teuthology.orchestra.run.smithi060.stdout:Name : grub2-tools 2024-09-12T12:45:58.891 INFO:teuthology.orchestra.run.smithi060.stdout:Epoch : 1 2024-09-12T12:45:58.891 INFO:teuthology.orchestra.run.smithi060.stdout:Version : 2.06 2024-09-12T12:45:58.891 INFO:teuthology.orchestra.run.smithi060.stdout:Release : 82.el9 2024-09-12T12:45:58.891 INFO:teuthology.orchestra.run.smithi060.stdout:Architecture: x86_64 2024-09-12T12:45:58.891 INFO:teuthology.orchestra.run.smithi060.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-12T12:45:58.892 INFO:teuthology.orchestra.run.smithi060.stdout:Group : Unspecified 2024-09-12T12:45:58.892 INFO:teuthology.orchestra.run.smithi060.stdout:Size : 8274670 2024-09-12T12:45:58.892 INFO:teuthology.orchestra.run.smithi060.stdout:License : GPLv3+ 2024-09-12T12:45:58.892 INFO:teuthology.orchestra.run.smithi060.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-12T12:45:58.892 INFO:teuthology.orchestra.run.smithi060.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-12T12:45:58.892 INFO:teuthology.orchestra.run.smithi060.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-12T12:45:58.892 INFO:teuthology.orchestra.run.smithi060.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-12T12:45:58.892 INFO:teuthology.orchestra.run.smithi060.stdout:Packager : builder@centos.org 2024-09-12T12:45:58.892 INFO:teuthology.orchestra.run.smithi060.stdout:Vendor : CentOS 2024-09-12T12:45:58.892 INFO:teuthology.orchestra.run.smithi060.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-12T12:45:58.892 INFO:teuthology.orchestra.run.smithi060.stdout:Summary : Support tools for GRUB. 2024-09-12T12:45:58.892 INFO:teuthology.orchestra.run.smithi060.stdout:Description : 2024-09-12T12:45:58.892 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-12T12:45:58.893 INFO:teuthology.orchestra.run.smithi060.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-12T12:45:58.893 INFO:teuthology.orchestra.run.smithi060.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-12T12:45:58.893 INFO:teuthology.orchestra.run.smithi060.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-12T12:45:58.893 INFO:teuthology.orchestra.run.smithi060.stdout:hardware devices. 2024-09-12T12:45:58.893 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-12T12:45:58.893 INFO:teuthology.orchestra.run.smithi060.stdout:This subpackage provides tools for support of all platforms. 2024-09-12T12:45:58.894 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-12T12:45:58.894 INFO:teuthology.task.kernel:Updating grub on smithi060 to boot 6.11.0-rc5-g2ca44a2f35ca 2024-09-12T12:45:58.894 DEBUG:teuthology.orchestra.run.smithi060:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-12T12:45:59.507 INFO:teuthology.orchestra.run.smithi060.stderr:Generating grub configuration file ... 2024-09-12T12:46:00.869 INFO:teuthology.orchestra.run.smithi060.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-12T12:46:00.897 INFO:teuthology.orchestra.run.smithi060.stderr:done 2024-09-12T12:46:00.899 DEBUG:teuthology.orchestra.run.smithi060:> mktemp 2024-09-12T12:46:00.914 INFO:teuthology.orchestra.run.smithi060.stdout:/tmp/tmp.XVANNMd62D 2024-09-12T12:46:00.915 DEBUG:teuthology.orchestra.run.smithi060:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.XVANNMd62D 2024-09-12T12:46:00.980 DEBUG:teuthology.orchestra.run.smithi060:> sudo chmod 0666 /tmp/tmp.XVANNMd62D 2024-09-12T12:46:01.117 DEBUG:teuthology.orchestra.remote:smithi060:/tmp/tmp.XVANNMd62D is 6KB 2024-09-12T12:46:01.166 DEBUG:teuthology.orchestra.run.smithi060:> rm -fr /tmp/tmp.XVANNMd62D 2024-09-12T12:46:01.180 DEBUG:teuthology.orchestra.run.smithi060:> sudo /bin/ls /boot/loader/entries || true 2024-09-12T12:46:01.243 INFO:teuthology.orchestra.run.smithi060.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-12T12:46:01.243 INFO:teuthology.orchestra.run.smithi060.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-12T12:46:01.243 INFO:teuthology.orchestra.run.smithi060.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-12T12:46:01.243 INFO:teuthology.orchestra.run.smithi060.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-6.11.0-rc5-g2ca44a2f35ca.conf 2024-09-12T12:46:01.244 DEBUG:teuthology.orchestra.run.smithi060:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-6.11.0-rc5-g2ca44a2f35ca 2024-09-12T12:46:01.430 DEBUG:teuthology.orchestra.run.smithi060:> sudo shutdown -r now 2024-09-12T12:46:09.333 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi130.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-09-12T12:46:09.334 DEBUG:teuthology.orchestra.run.smithi130:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-09-12T12:46:09.421 INFO:teuthology.orchestra.run.smithi130.stdout:/boot/System.map-6.11.0-rc5-g2ca44a2f35ca 2024-09-12T12:46:09.421 INFO:teuthology.orchestra.run.smithi130.stdout:/boot/config-6.11.0-rc5-g2ca44a2f35ca 2024-09-12T12:46:09.421 INFO:teuthology.orchestra.run.smithi130.stdout:/boot/vmlinuz-6.11.0-rc5-g2ca44a2f35ca 2024-09-12T12:46:09.421 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca 2024-09-12T12:46:09.421 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/System.map 2024-09-12T12:46:09.421 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/config 2024-09-12T12:46:09.421 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel 2024-09-12T12:46:09.421 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch 2024-09-12T12:46:09.421 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86 2024-09-12T12:46:09.421 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto 2024-09-12T12:46:09.422 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-09-12T12:46:09.422 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-09-12T12:46:09.422 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-09-12T12:46:09.422 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-09-12T12:46:09.422 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-09-12T12:46:09.422 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-09-12T12:46:09.422 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-09-12T12:46:09.422 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/crc32c-intel.ko 2024-09-12T12:46:09.422 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-09-12T12:46:09.422 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-09-12T12:46:09.422 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-09-12T12:46:09.423 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-09-12T12:46:09.423 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-avx2.ko 2024-09-12T12:46:09.423 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-09-12T12:46:09.423 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-09-12T12:46:09.423 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-09-12T12:46:09.423 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-09-12T12:46:09.423 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-09-12T12:46:09.423 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-09-12T12:46:09.423 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-09-12T12:46:09.423 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel 2024-09-12T12:46:09.424 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu 2024-09-12T12:46:09.424 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu/mce 2024-09-12T12:46:09.424 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-09-12T12:46:09.424 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm 2024-09-12T12:46:09.424 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm-amd.ko 2024-09-12T12:46:09.424 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm-intel.ko 2024-09-12T12:46:09.424 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm.ko 2024-09-12T12:46:09.424 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto 2024-09-12T12:46:09.424 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ansi_cprng.ko 2024-09-12T12:46:09.424 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/anubis.ko 2024-09-12T12:46:09.425 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/arc4.ko 2024-09-12T12:46:09.425 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx 2024-09-12T12:46:09.425 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_memcpy.ko 2024-09-12T12:46:09.425 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_pq.ko 2024-09-12T12:46:09.425 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_raid6_recov.ko 2024-09-12T12:46:09.425 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_tx.ko 2024-09-12T12:46:09.425 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_xor.ko 2024-09-12T12:46:09.425 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/raid6test.ko 2024-09-12T12:46:09.425 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/authenc.ko 2024-09-12T12:46:09.425 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/authencesn.ko 2024-09-12T12:46:09.426 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blake2b_generic.ko 2024-09-12T12:46:09.426 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blowfish_common.ko 2024-09-12T12:46:09.426 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blowfish_generic.ko 2024-09-12T12:46:09.426 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/camellia_generic.ko 2024-09-12T12:46:09.426 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast5_generic.ko 2024-09-12T12:46:09.426 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast6_generic.ko 2024-09-12T12:46:09.426 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast_common.ko 2024-09-12T12:46:09.426 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ccm.ko 2024-09-12T12:46:09.426 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cmac.ko 2024-09-12T12:46:09.426 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crc32_generic.ko 2024-09-12T12:46:09.426 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crypto_engine.ko 2024-09-12T12:46:09.427 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crypto_user.ko 2024-09-12T12:46:09.427 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cts.ko 2024-09-12T12:46:09.427 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/des_generic.ko 2024-09-12T12:46:09.427 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/dh_generic.ko 2024-09-12T12:46:09.427 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ecc.ko 2024-09-12T12:46:09.427 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ecdh_generic.ko 2024-09-12T12:46:09.427 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/echainiv.ko 2024-09-12T12:46:09.427 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/essiv.ko 2024-09-12T12:46:09.427 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/fcrypt.ko 2024-09-12T12:46:09.427 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/khazad.ko 2024-09-12T12:46:09.428 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/lz4.ko 2024-09-12T12:46:09.428 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/lz4hc.ko 2024-09-12T12:46:09.428 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/md4.ko 2024-09-12T12:46:09.428 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/michael_mic.ko 2024-09-12T12:46:09.428 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/pcbc.ko 2024-09-12T12:46:09.428 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/pcrypt.ko 2024-09-12T12:46:09.428 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/rmd160.ko 2024-09-12T12:46:09.428 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/seed.ko 2024-09-12T12:46:09.428 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/serpent_generic.ko 2024-09-12T12:46:09.428 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/tcrypt.ko 2024-09-12T12:46:09.428 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/tea.ko 2024-09-12T12:46:09.429 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/twofish_common.ko 2024-09-12T12:46:09.429 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/twofish_generic.ko 2024-09-12T12:46:09.429 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/vmac.ko 2024-09-12T12:46:09.429 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/wp512.ko 2024-09-12T12:46:09.429 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xcbc.ko 2024-09-12T12:46:09.429 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xor.ko 2024-09-12T12:46:09.429 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xxhash_generic.ko 2024-09-12T12:46:09.429 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers 2024-09-12T12:46:09.429 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi 2024-09-12T12:46:09.429 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/acpi_ipmi.ko 2024-09-12T12:46:09.429 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/acpi_pad.ko 2024-09-12T12:46:09.430 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/ec_sys.ko 2024-09-12T12:46:09.430 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/platform_profile.ko 2024-09-12T12:46:09.430 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/sbs.ko 2024-09-12T12:46:09.430 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/sbshc.ko 2024-09-12T12:46:09.430 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/video.ko 2024-09-12T12:46:09.430 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata 2024-09-12T12:46:09.430 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/acard-ahci.ko 2024-09-12T12:46:09.430 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/ahci_platform.ko 2024-09-12T12:46:09.430 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/ata_generic.ko 2024-09-12T12:46:09.430 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/libahci_platform.ko 2024-09-12T12:46:09.430 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_acpi.ko 2024-09-12T12:46:09.430 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ali.ko 2024-09-12T12:46:09.431 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_amd.ko 2024-09-12T12:46:09.431 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_artop.ko 2024-09-12T12:46:09.431 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_atiixp.ko 2024-09-12T12:46:09.431 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_atp867x.ko 2024-09-12T12:46:09.431 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cmd640.ko 2024-09-12T12:46:09.431 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cmd64x.ko 2024-09-12T12:46:09.431 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cypress.ko 2024-09-12T12:46:09.431 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_efar.ko 2024-09-12T12:46:09.431 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt366.ko 2024-09-12T12:46:09.431 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt37x.ko 2024-09-12T12:46:09.432 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt3x2n.ko 2024-09-12T12:46:09.432 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt3x3.ko 2024-09-12T12:46:09.432 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_it8213.ko 2024-09-12T12:46:09.432 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_it821x.ko 2024-09-12T12:46:09.432 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_jmicron.ko 2024-09-12T12:46:09.432 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_marvell.ko 2024-09-12T12:46:09.432 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_mpiix.ko 2024-09-12T12:46:09.432 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_netcell.ko 2024-09-12T12:46:09.432 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ninja32.ko 2024-09-12T12:46:09.432 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ns87410.ko 2024-09-12T12:46:09.433 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ns87415.ko 2024-09-12T12:46:09.433 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_oldpiix.ko 2024-09-12T12:46:09.433 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_opti.ko 2024-09-12T12:46:09.433 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_optidma.ko 2024-09-12T12:46:09.433 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pcmcia.ko 2024-09-12T12:46:09.433 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pdc2027x.ko 2024-09-12T12:46:09.433 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-09-12T12:46:09.433 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_piccolo.ko 2024-09-12T12:46:09.433 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_rdc.ko 2024-09-12T12:46:09.433 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sch.ko 2024-09-12T12:46:09.433 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_serverworks.ko 2024-09-12T12:46:09.434 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sil680.ko 2024-09-12T12:46:09.434 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sis.ko 2024-09-12T12:46:09.434 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sl82c105.ko 2024-09-12T12:46:09.434 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_triflex.ko 2024-09-12T12:46:09.434 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_via.ko 2024-09-12T12:46:09.434 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pdc_adma.ko 2024-09-12T12:46:09.434 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_inic162x.ko 2024-09-12T12:46:09.434 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_mv.ko 2024-09-12T12:46:09.434 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_nv.ko 2024-09-12T12:46:09.434 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_promise.ko 2024-09-12T12:46:09.434 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_qstor.ko 2024-09-12T12:46:09.435 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sil.ko 2024-09-12T12:46:09.435 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sil24.ko 2024-09-12T12:46:09.435 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sis.ko 2024-09-12T12:46:09.435 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_svw.ko 2024-09-12T12:46:09.435 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sx4.ko 2024-09-12T12:46:09.435 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_uli.ko 2024-09-12T12:46:09.435 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_via.ko 2024-09-12T12:46:09.435 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_vsc.ko 2024-09-12T12:46:09.435 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/auxdisplay 2024-09-12T12:46:09.435 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/auxdisplay/ks0108.ko 2024-09-12T12:46:09.436 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/base 2024-09-12T12:46:09.436 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/base/regmap 2024-09-12T12:46:09.436 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/base/regmap/regmap-i2c.ko 2024-09-12T12:46:09.436 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/base/regmap/regmap-mmio.ko 2024-09-12T12:46:09.436 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/bcma 2024-09-12T12:46:09.436 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/bcma/bcma.ko 2024-09-12T12:46:09.436 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block 2024-09-12T12:46:09.436 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/aoe 2024-09-12T12:46:09.436 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/aoe/aoe.ko 2024-09-12T12:46:09.436 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/brd.ko 2024-09-12T12:46:09.436 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/drbd 2024-09-12T12:46:09.437 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/drbd/drbd.ko 2024-09-12T12:46:09.437 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/floppy.ko 2024-09-12T12:46:09.437 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/loop.ko 2024-09-12T12:46:09.437 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/mtip32xx 2024-09-12T12:46:09.437 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-09-12T12:46:09.437 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/nbd.ko 2024-09-12T12:46:09.437 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/null_blk 2024-09-12T12:46:09.437 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/null_blk/null_blk.ko 2024-09-12T12:46:09.437 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/pktcdvd.ko 2024-09-12T12:46:09.437 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/rbd.ko 2024-09-12T12:46:09.437 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/virtio_blk.ko 2024-09-12T12:46:09.438 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/xen-blkback 2024-09-12T12:46:09.438 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-09-12T12:46:09.438 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/xen-blkfront.ko 2024-09-12T12:46:09.438 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/zram 2024-09-12T12:46:09.438 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/zram/zram.ko 2024-09-12T12:46:09.438 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char 2024-09-12T12:46:09.438 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hangcheck-timer.ko 2024-09-12T12:46:09.438 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random 2024-09-12T12:46:09.438 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/amd-rng.ko 2024-09-12T12:46:09.438 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/intel-rng.ko 2024-09-12T12:46:09.438 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-09-12T12:46:09.438 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/via-rng.ko 2024-09-12T12:46:09.439 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/virtio-rng.ko 2024-09-12T12:46:09.439 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi 2024-09-12T12:46:09.439 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-09-12T12:46:09.439 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-09-12T12:46:09.439 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-09-12T12:46:09.439 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_si.ko 2024-09-12T12:46:09.439 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-09-12T12:46:09.439 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-09-12T12:46:09.439 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/lp.ko 2024-09-12T12:46:09.439 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/mwave 2024-09-12T12:46:09.439 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/mwave/mwave.ko 2024-09-12T12:46:09.440 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ppdev.ko 2024-09-12T12:46:09.440 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tlclk.ko 2024-09-12T12:46:09.440 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm 2024-09-12T12:46:09.440 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm.ko 2024-09-12T12:46:09.440 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_atmel.ko 2024-09-12T12:46:09.440 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_infineon.ko 2024-09-12T12:46:09.440 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_nsc.ko 2024-09-12T12:46:09.440 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_tis.ko 2024-09-12T12:46:09.440 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-09-12T12:46:09.440 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/uv_mmtimer.ko 2024-09-12T12:46:09.440 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/virtio_console.ko 2024-09-12T12:46:09.440 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq 2024-09-12T12:46:09.441 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-09-12T12:46:09.441 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-09-12T12:46:09.441 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/p4-clockmod.ko 2024-09-12T12:46:09.441 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-09-12T12:46:09.441 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/powernow-k8.ko 2024-09-12T12:46:09.441 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/speedstep-lib.ko 2024-09-12T12:46:09.441 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto 2024-09-12T12:46:09.441 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp 2024-09-12T12:46:09.441 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-09-12T12:46:09.441 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp/ccp.ko 2024-09-12T12:46:09.442 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel 2024-09-12T12:46:09.442 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat 2024-09-12T12:46:09.442 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_common 2024-09-12T12:46:09.442 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-09-12T12:46:09.442 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-09-12T12:46:09.442 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-09-12T12:46:09.442 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/padlock-aes.ko 2024-09-12T12:46:09.442 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/padlock-sha.ko 2024-09-12T12:46:09.442 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/virtio 2024-09-12T12:46:09.442 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-09-12T12:46:09.442 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dca 2024-09-12T12:46:09.442 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dca/dca.ko 2024-09-12T12:46:09.443 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/devfreq 2024-09-12T12:46:09.443 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-09-12T12:46:09.443 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma 2024-09-12T12:46:09.443 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/dw 2024-09-12T12:46:09.443 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/dw/dw_dmac.ko 2024-09-12T12:46:09.443 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/ioat 2024-09-12T12:46:09.443 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/ioat/ioatdma.ko 2024-09-12T12:46:09.443 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac 2024-09-12T12:46:09.443 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/amd64_edac.ko 2024-09-12T12:46:09.443 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/e752x_edac.ko 2024-09-12T12:46:09.443 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/edac_mce_amd.ko 2024-09-12T12:46:09.444 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i3000_edac.ko 2024-09-12T12:46:09.444 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i3200_edac.ko 2024-09-12T12:46:09.444 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i5100_edac.ko 2024-09-12T12:46:09.444 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i5400_edac.ko 2024-09-12T12:46:09.444 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i7300_edac.ko 2024-09-12T12:46:09.444 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i7core_edac.ko 2024-09-12T12:46:09.444 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i82975x_edac.ko 2024-09-12T12:46:09.444 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/ie31200_edac.ko 2024-09-12T12:46:09.444 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/sb_edac.ko 2024-09-12T12:46:09.444 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/x38_edac.ko 2024-09-12T12:46:09.444 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware 2024-09-12T12:46:09.445 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/edd.ko 2024-09-12T12:46:09.445 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/iscsi_ibft.ko 2024-09-12T12:46:09.445 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio 2024-09-12T12:46:09.445 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-ich.ko 2024-09-12T12:46:09.445 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-viperboard.ko 2024-09-12T12:46:09.445 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid 2024-09-12T12:46:09.445 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-appleir.ko 2024-09-12T12:46:09.445 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-aureal.ko 2024-09-12T12:46:09.445 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-axff.ko 2024-09-12T12:46:09.446 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-dr.ko 2024-09-12T12:46:09.446 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-elecom.ko 2024-09-12T12:46:09.446 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-elo.ko 2024-09-12T12:46:09.446 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-emsff.ko 2024-09-12T12:46:09.446 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gaff.ko 2024-09-12T12:46:09.446 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gt683r.ko 2024-09-12T12:46:09.446 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gyration.ko 2024-09-12T12:46:09.446 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtek-kbd.ko 2024-09-12T12:46:09.446 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtek-mouse.ko 2024-09-12T12:46:09.446 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtekff.ko 2024-09-12T12:46:09.446 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-hyperv.ko 2024-09-12T12:46:09.447 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-icade.ko 2024-09-12T12:46:09.447 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-keytouch.ko 2024-09-12T12:46:09.447 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-kye.ko 2024-09-12T12:46:09.447 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-lcpower.ko 2024-09-12T12:46:09.447 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-led.ko 2024-09-12T12:46:09.447 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-lenovo.ko 2024-09-12T12:46:09.447 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-logitech-dj.ko 2024-09-12T12:46:09.447 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-09-12T12:46:09.447 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-multitouch.ko 2024-09-12T12:46:09.447 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-ortek.ko 2024-09-12T12:46:09.448 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-penmount.ko 2024-09-12T12:46:09.448 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-petalynx.ko 2024-09-12T12:46:09.448 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-picolcd.ko 2024-09-12T12:46:09.448 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-pl.ko 2024-09-12T12:46:09.448 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-plantronics.ko 2024-09-12T12:46:09.448 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-primax.ko 2024-09-12T12:46:09.448 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-rmi.ko 2024-09-12T12:46:09.448 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-arvo.ko 2024-09-12T12:46:09.448 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-common.ko 2024-09-12T12:46:09.448 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-isku.ko 2024-09-12T12:46:09.448 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-kone.ko 2024-09-12T12:46:09.449 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-09-12T12:46:09.449 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-konepure.ko 2024-09-12T12:46:09.449 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-09-12T12:46:09.449 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-lua.ko 2024-09-12T12:46:09.449 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-pyra.ko 2024-09-12T12:46:09.449 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-ryos.ko 2024-09-12T12:46:09.449 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-savu.ko 2024-09-12T12:46:09.449 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat.ko 2024-09-12T12:46:09.449 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-saitek.ko 2024-09-12T12:46:09.449 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-samsung.ko 2024-09-12T12:46:09.449 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sensor-hub.ko 2024-09-12T12:46:09.450 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sjoy.ko 2024-09-12T12:46:09.450 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sony.ko 2024-09-12T12:46:09.450 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-speedlink.ko 2024-09-12T12:46:09.450 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-steelseries.ko 2024-09-12T12:46:09.450 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sunplus.ko 2024-09-12T12:46:09.450 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-thrustmaster.ko 2024-09-12T12:46:09.450 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-tivo.ko 2024-09-12T12:46:09.450 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-tmff.ko 2024-09-12T12:46:09.450 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-topseed.ko 2024-09-12T12:46:09.450 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-twinhan.ko 2024-09-12T12:46:09.450 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-uclogic.ko 2024-09-12T12:46:09.451 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-waltop.ko 2024-09-12T12:46:09.451 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-wiimote.ko 2024-09-12T12:46:09.451 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-xinmo.ko 2024-09-12T12:46:09.451 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-zpff.ko 2024-09-12T12:46:09.451 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-zydacron.ko 2024-09-12T12:46:09.451 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/uhid.ko 2024-09-12T12:46:09.451 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/wacom.ko 2024-09-12T12:46:09.451 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv 2024-09-12T12:46:09.451 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_balloon.ko 2024-09-12T12:46:09.451 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_utils.ko 2024-09-12T12:46:09.451 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_vmbus.ko 2024-09-12T12:46:09.451 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon 2024-09-12T12:46:09.452 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/abituguru.ko 2024-09-12T12:46:09.452 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/abituguru3.ko 2024-09-12T12:46:09.452 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/acpi_power_meter.ko 2024-09-12T12:46:09.452 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ad7414.ko 2024-09-12T12:46:09.452 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ad7418.ko 2024-09-12T12:46:09.452 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adc128d818.ko 2024-09-12T12:46:09.452 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1025.ko 2024-09-12T12:46:09.452 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1026.ko 2024-09-12T12:46:09.452 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1029.ko 2024-09-12T12:46:09.452 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1031.ko 2024-09-12T12:46:09.453 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm9240.ko 2024-09-12T12:46:09.453 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ads7828.ko 2024-09-12T12:46:09.453 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7410.ko 2024-09-12T12:46:09.453 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7411.ko 2024-09-12T12:46:09.453 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7462.ko 2024-09-12T12:46:09.453 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7470.ko 2024-09-12T12:46:09.453 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7475.ko 2024-09-12T12:46:09.453 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7x10.ko 2024-09-12T12:46:09.453 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/amc6821.ko 2024-09-12T12:46:09.453 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/applesmc.ko 2024-09-12T12:46:09.454 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asb100.ko 2024-09-12T12:46:09.454 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asc7621.ko 2024-09-12T12:46:09.454 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asus_atk0110.ko 2024-09-12T12:46:09.454 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/atxp1.ko 2024-09-12T12:46:09.454 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/coretemp.ko 2024-09-12T12:46:09.454 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-09-12T12:46:09.454 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/dme1737.ko 2024-09-12T12:46:09.454 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ds1621.ko 2024-09-12T12:46:09.454 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ds620.ko 2024-09-12T12:46:09.454 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/emc1403.ko 2024-09-12T12:46:09.454 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/emc6w201.ko 2024-09-12T12:46:09.454 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f71805f.ko 2024-09-12T12:46:09.455 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f71882fg.ko 2024-09-12T12:46:09.455 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f75375s.ko 2024-09-12T12:46:09.455 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/fam15h_power.ko 2024-09-12T12:46:09.455 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/fschmd.ko 2024-09-12T12:46:09.455 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/g760a.ko 2024-09-12T12:46:09.455 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/g762.ko 2024-09-12T12:46:09.455 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/gl518sm.ko 2024-09-12T12:46:09.455 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/gl520sm.ko 2024-09-12T12:46:09.455 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/hwmon-vid.ko 2024-09-12T12:46:09.455 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/i5500_temp.ko 2024-09-12T12:46:09.455 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/i5k_amb.ko 2024-09-12T12:46:09.455 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ibmaem.ko 2024-09-12T12:46:09.456 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ibmpex.ko 2024-09-12T12:46:09.456 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ina209.ko 2024-09-12T12:46:09.456 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ina2xx.ko 2024-09-12T12:46:09.456 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/it87.ko 2024-09-12T12:46:09.456 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/k10temp.ko 2024-09-12T12:46:09.456 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/k8temp.ko 2024-09-12T12:46:09.456 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lineage-pem.ko 2024-09-12T12:46:09.456 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm63.ko 2024-09-12T12:46:09.456 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm73.ko 2024-09-12T12:46:09.456 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm75.ko 2024-09-12T12:46:09.456 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm77.ko 2024-09-12T12:46:09.456 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm78.ko 2024-09-12T12:46:09.457 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm80.ko 2024-09-12T12:46:09.457 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm83.ko 2024-09-12T12:46:09.457 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm85.ko 2024-09-12T12:46:09.457 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm87.ko 2024-09-12T12:46:09.457 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm90.ko 2024-09-12T12:46:09.457 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm92.ko 2024-09-12T12:46:09.457 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm93.ko 2024-09-12T12:46:09.457 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95234.ko 2024-09-12T12:46:09.457 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95241.ko 2024-09-12T12:46:09.457 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95245.ko 2024-09-12T12:46:09.457 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc2945.ko 2024-09-12T12:46:09.457 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4151.ko 2024-09-12T12:46:09.458 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4215.ko 2024-09-12T12:46:09.458 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4222.ko 2024-09-12T12:46:09.458 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4245.ko 2024-09-12T12:46:09.458 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4260.ko 2024-09-12T12:46:09.458 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4261.ko 2024-09-12T12:46:09.458 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max16065.ko 2024-09-12T12:46:09.458 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max1619.ko 2024-09-12T12:46:09.458 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max1668.ko 2024-09-12T12:46:09.458 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max197.ko 2024-09-12T12:46:09.458 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6639.ko 2024-09-12T12:46:09.458 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6650.ko 2024-09-12T12:46:09.458 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6697.ko 2024-09-12T12:46:09.459 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/mcp3021.ko 2024-09-12T12:46:09.459 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6683.ko 2024-09-12T12:46:09.459 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6775-core.ko 2024-09-12T12:46:09.459 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6775.ko 2024-09-12T12:46:09.459 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct7802.ko 2024-09-12T12:46:09.459 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pc87360.ko 2024-09-12T12:46:09.459 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pc87427.ko 2024-09-12T12:46:09.459 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pcf8591.ko 2024-09-12T12:46:09.459 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus 2024-09-12T12:46:09.459 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-09-12T12:46:09.459 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-09-12T12:46:09.460 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-09-12T12:46:09.460 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max16064.ko 2024-09-12T12:46:09.460 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max34440.ko 2024-09-12T12:46:09.460 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max8688.ko 2024-09-12T12:46:09.460 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-09-12T12:46:09.460 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-09-12T12:46:09.460 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-09-12T12:46:09.460 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-09-12T12:46:09.460 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-09-12T12:46:09.460 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-09-12T12:46:09.460 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/powr1220.ko 2024-09-12T12:46:09.461 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch5627.ko 2024-09-12T12:46:09.461 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch5636.ko 2024-09-12T12:46:09.461 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch56xx-common.ko 2024-09-12T12:46:09.461 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sht15.ko 2024-09-12T12:46:09.461 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sht21.ko 2024-09-12T12:46:09.461 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/shtc1.ko 2024-09-12T12:46:09.461 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sis5595.ko 2024-09-12T12:46:09.461 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47b397.ko 2024-09-12T12:46:09.461 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47m1.ko 2024-09-12T12:46:09.461 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47m192.ko 2024-09-12T12:46:09.461 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/thmc50.ko 2024-09-12T12:46:09.461 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp102.ko 2024-09-12T12:46:09.462 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp103.ko 2024-09-12T12:46:09.462 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp401.ko 2024-09-12T12:46:09.462 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp421.ko 2024-09-12T12:46:09.462 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/via-cputemp.ko 2024-09-12T12:46:09.462 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/via686a.ko 2024-09-12T12:46:09.462 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/vt1211.ko 2024-09-12T12:46:09.462 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/vt8231.ko 2024-09-12T12:46:09.462 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83627ehf.ko 2024-09-12T12:46:09.462 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83627hf.ko 2024-09-12T12:46:09.462 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83781d.ko 2024-09-12T12:46:09.462 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83791d.ko 2024-09-12T12:46:09.462 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83792d.ko 2024-09-12T12:46:09.463 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83793.ko 2024-09-12T12:46:09.463 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83795.ko 2024-09-12T12:46:09.463 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83l785ts.ko 2024-09-12T12:46:09.463 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83l786ng.ko 2024-09-12T12:46:09.463 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c 2024-09-12T12:46:09.463 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos 2024-09-12T12:46:09.463 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-09-12T12:46:09.463 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-09-12T12:46:09.463 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses 2024-09-12T12:46:09.463 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-09-12T12:46:09.463 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-09-12T12:46:09.463 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-09-12T12:46:09.464 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-09-12T12:46:09.464 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-09-12T12:46:09.464 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-09-12T12:46:09.464 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-09-12T12:46:09.464 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-09-12T12:46:09.464 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-i801.ko 2024-09-12T12:46:09.464 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-isch.ko 2024-09-12T12:46:09.464 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-09-12T12:46:09.464 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-09-12T12:46:09.464 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-09-12T12:46:09.464 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-parport.ko 2024-09-12T12:46:09.464 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-09-12T12:46:09.464 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-09-12T12:46:09.465 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-09-12T12:46:09.465 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-09-12T12:46:09.465 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-09-12T12:46:09.465 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-09-12T12:46:09.465 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-via.ko 2024-09-12T12:46:09.465 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-09-12T12:46:09.465 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-09-12T12:46:09.465 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-dev.ko 2024-09-12T12:46:09.465 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-mux.ko 2024-09-12T12:46:09.465 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-09-12T12:46:09.465 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-smbus.ko 2024-09-12T12:46:09.466 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-stub.ko 2024-09-12T12:46:09.466 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband 2024-09-12T12:46:09.466 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core 2024-09-12T12:46:09.466 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_cm.ko 2024-09-12T12:46:09.466 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_core.ko 2024-09-12T12:46:09.466 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_umad.ko 2024-09-12T12:46:09.466 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-09-12T12:46:09.466 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/iw_cm.ko 2024-09-12T12:46:09.466 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/rdma_cm.ko 2024-09-12T12:46:09.466 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-09-12T12:46:09.466 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw 2024-09-12T12:46:09.466 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/cxgb4 2024-09-12T12:46:09.467 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-09-12T12:46:09.467 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx4 2024-09-12T12:46:09.467 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-09-12T12:46:09.467 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx5 2024-09-12T12:46:09.467 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-09-12T12:46:09.467 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mthca 2024-09-12T12:46:09.467 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-09-12T12:46:09.467 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp 2024-09-12T12:46:09.467 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/ipoib 2024-09-12T12:46:09.467 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-09-12T12:46:09.467 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/iser 2024-09-12T12:46:09.468 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-09-12T12:46:09.468 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/isert 2024-09-12T12:46:09.468 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-09-12T12:46:09.468 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srp 2024-09-12T12:46:09.468 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-09-12T12:46:09.468 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srpt 2024-09-12T12:46:09.468 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-09-12T12:46:09.468 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input 2024-09-12T12:46:09.468 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport 2024-09-12T12:46:09.468 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-09-12T12:46:09.468 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/fm801-gp.ko 2024-09-12T12:46:09.468 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/gameport.ko 2024-09-12T12:46:09.469 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/joydev.ko 2024-09-12T12:46:09.469 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard 2024-09-12T12:46:09.469 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/gpio_keys.ko 2024-09-12T12:46:09.469 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse 2024-09-12T12:46:09.469 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/appletouch.ko 2024-09-12T12:46:09.469 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/bcm5974.ko 2024-09-12T12:46:09.469 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/cyapatp.ko 2024-09-12T12:46:09.469 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/elan_i2c.ko 2024-09-12T12:46:09.469 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/sermouse.ko 2024-09-12T12:46:09.469 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-09-12T12:46:09.469 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/synaptics_usb.ko 2024-09-12T12:46:09.470 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/vsxxxaa.ko 2024-09-12T12:46:09.470 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/rmi4 2024-09-12T12:46:09.470 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/rmi4/rmi_core.ko 2024-09-12T12:46:09.470 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio 2024-09-12T12:46:09.470 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/altera_ps2.ko 2024-09-12T12:46:09.470 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/arc_ps2.ko 2024-09-12T12:46:09.470 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-09-12T12:46:09.470 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/serio_raw.ko 2024-09-12T12:46:09.470 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/sparse-keymap.ko 2024-09-12T12:46:09.470 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds 2024-09-12T12:46:09.470 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-blinkm.ko 2024-09-12T12:46:09.470 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lm3530.ko 2024-09-12T12:46:09.471 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lp3944.ko 2024-09-12T12:46:09.471 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-ss4200.ko 2024-09-12T12:46:09.471 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger 2024-09-12T12:46:09.471 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-09-12T12:46:09.471 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-09-12T12:46:09.471 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-09-12T12:46:09.471 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-09-12T12:46:09.471 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-09-12T12:46:09.471 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-09-12T12:46:09.471 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-09-12T12:46:09.471 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-09-12T12:46:09.471 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md 2024-09-12T12:46:09.471 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/bcache 2024-09-12T12:46:09.471 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/bcache/bcache.ko 2024-09-12T12:46:09.472 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-bio-prison.ko 2024-09-12T12:46:09.472 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-cache-smq.ko 2024-09-12T12:46:09.472 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-cache.ko 2024-09-12T12:46:09.472 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-crypt.ko 2024-09-12T12:46:09.472 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-delay.ko 2024-09-12T12:46:09.472 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-flakey.ko 2024-09-12T12:46:09.472 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-log-userspace.ko 2024-09-12T12:46:09.472 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-multipath.ko 2024-09-12T12:46:09.472 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-queue-length.ko 2024-09-12T12:46:09.472 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-raid.ko 2024-09-12T12:46:09.472 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-round-robin.ko 2024-09-12T12:46:09.473 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-service-time.ko 2024-09-12T12:46:09.473 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-switch.ko 2024-09-12T12:46:09.473 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-thin-pool.ko 2024-09-12T12:46:09.473 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-verity.ko 2024-09-12T12:46:09.473 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/persistent-data 2024-09-12T12:46:09.473 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-09-12T12:46:09.473 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid0.ko 2024-09-12T12:46:09.473 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid1.ko 2024-09-12T12:46:09.473 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid10.ko 2024-09-12T12:46:09.473 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid456.ko 2024-09-12T12:46:09.473 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message 2024-09-12T12:46:09.474 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion 2024-09-12T12:46:09.474 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptbase.ko 2024-09-12T12:46:09.474 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptctl.ko 2024-09-12T12:46:09.474 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptfc.ko 2024-09-12T12:46:09.474 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptlan.ko 2024-09-12T12:46:09.474 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptsas.ko 2024-09-12T12:46:09.474 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptscsih.ko 2024-09-12T12:46:09.474 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptspi.ko 2024-09-12T12:46:09.474 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd 2024-09-12T12:46:09.474 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lpc_ich.ko 2024-09-12T12:46:09.474 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lpc_sch.ko 2024-09-12T12:46:09.474 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/mfd-core.ko 2024-09-12T12:46:09.475 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/sm501.ko 2024-09-12T12:46:09.475 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/viperboard.ko 2024-09-12T12:46:09.475 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/vx855.ko 2024-09-12T12:46:09.475 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/wl1273-core.ko 2024-09-12T12:46:09.475 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc 2024-09-12T12:46:09.475 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/altera-stapl 2024-09-12T12:46:09.475 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-09-12T12:46:09.475 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/apds9802als.ko 2024-09-12T12:46:09.475 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/apds990x.ko 2024-09-12T12:46:09.475 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/bh1770glc.ko 2024-09-12T12:46:09.475 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/cb710 2024-09-12T12:46:09.476 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/cb710/cb710.ko 2024-09-12T12:46:09.476 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom 2024-09-12T12:46:09.476 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/at24.ko 2024-09-12T12:46:09.476 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-09-12T12:46:09.476 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/max6875.ko 2024-09-12T12:46:09.476 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/enclosure.ko 2024-09-12T12:46:09.476 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/hpilo.ko 2024-09-12T12:46:09.476 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/isl29003.ko 2024-09-12T12:46:09.476 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/isl29020.ko 2024-09-12T12:46:09.476 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d 2024-09-12T12:46:09.476 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-09-12T12:46:09.477 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-09-12T12:46:09.477 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei 2024-09-12T12:46:09.477 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei-me.ko 2024-09-12T12:46:09.477 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei-txe.ko 2024-09-12T12:46:09.477 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei.ko 2024-09-12T12:46:09.477 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-gru 2024-09-12T12:46:09.477 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-gru/gru.ko 2024-09-12T12:46:09.477 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-xp 2024-09-12T12:46:09.477 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-xp/xp.ko 2024-09-12T12:46:09.478 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-xp/xpc.ko 2024-09-12T12:46:09.478 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-09-12T12:46:09.478 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tifm_7xx1.ko 2024-09-12T12:46:09.478 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tifm_core.ko 2024-09-12T12:46:09.478 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tsl2550.ko 2024-09-12T12:46:09.478 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_balloon.ko 2024-09-12T12:46:09.478 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_vmci 2024-09-12T12:46:09.478 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-09-12T12:46:09.478 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net 2024-09-12T12:46:09.478 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/bonding 2024-09-12T12:46:09.478 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/bonding/bonding.ko 2024-09-12T12:46:09.478 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa 2024-09-12T12:46:09.479 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/b53 2024-09-12T12:46:09.479 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/b53/b53_common.ko 2024-09-12T12:46:09.479 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/bcm-sf2.ko 2024-09-12T12:46:09.479 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6060.ko 2024-09-12T12:46:09.479 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6xxx 2024-09-12T12:46:09.479 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-09-12T12:46:09.479 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dummy.ko 2024-09-12T12:46:09.479 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/eql.ko 2024-09-12T12:46:09.479 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet 2024-09-12T12:46:09.479 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com 2024-09-12T12:46:09.479 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-09-12T12:46:09.480 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-09-12T12:46:09.480 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-09-12T12:46:09.480 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-09-12T12:46:09.480 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390 2024-09-12T12:46:09.480 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/8390.ko 2024-09-12T12:46:09.480 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-09-12T12:46:09.480 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-09-12T12:46:09.480 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-09-12T12:46:09.480 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/adaptec 2024-09-12T12:46:09.480 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-09-12T12:46:09.480 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/agere 2024-09-12T12:46:09.481 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/agere/et131x.ko 2024-09-12T12:46:09.481 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/alteon 2024-09-12T12:46:09.481 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-09-12T12:46:09.481 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/altera 2024-09-12T12:46:09.481 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-09-12T12:46:09.481 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd 2024-09-12T12:46:09.481 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-09-12T12:46:09.481 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-09-12T12:46:09.481 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-09-12T12:46:09.481 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros 2024-09-12T12:46:09.481 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/alx 2024-09-12T12:46:09.481 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-09-12T12:46:09.482 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1c 2024-09-12T12:46:09.482 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-09-12T12:46:09.482 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1e 2024-09-12T12:46:09.482 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-09-12T12:46:09.482 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx 2024-09-12T12:46:09.482 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-09-12T12:46:09.482 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-09-12T12:46:09.482 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom 2024-09-12T12:46:09.482 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-09-12T12:46:09.482 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-09-12T12:46:09.482 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-09-12T12:46:09.483 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-09-12T12:46:09.483 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-09-12T12:46:09.483 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/genet 2024-09-12T12:46:09.483 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-09-12T12:46:09.483 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-09-12T12:46:09.483 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade 2024-09-12T12:46:09.483 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade/bna 2024-09-12T12:46:09.483 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-09-12T12:46:09.483 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio 2024-09-12T12:46:09.483 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb 2024-09-12T12:46:09.483 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-09-12T12:46:09.483 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-09-12T12:46:09.484 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-09-12T12:46:09.484 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-09-12T12:46:09.484 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-09-12T12:46:09.484 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-09-12T12:46:09.484 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-09-12T12:46:09.484 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-09-12T12:46:09.484 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-09-12T12:46:09.484 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco 2024-09-12T12:46:09.484 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco/enic 2024-09-12T12:46:09.484 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-09-12T12:46:09.484 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec 2024-09-12T12:46:09.484 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip 2024-09-12T12:46:09.485 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-09-12T12:46:09.485 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-09-12T12:46:09.485 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-09-12T12:46:09.485 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-09-12T12:46:09.485 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-09-12T12:46:09.485 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-09-12T12:46:09.485 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink 2024-09-12T12:46:09.485 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-09-12T12:46:09.485 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-09-12T12:46:09.485 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dnet.ko 2024-09-12T12:46:09.485 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex 2024-09-12T12:46:09.486 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex/benet 2024-09-12T12:46:09.486 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-09-12T12:46:09.486 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ethoc.ko 2024-09-12T12:46:09.486 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/fealnx.ko 2024-09-12T12:46:09.486 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel 2024-09-12T12:46:09.486 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e100.ko 2024-09-12T12:46:09.486 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000 2024-09-12T12:46:09.486 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-09-12T12:46:09.486 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000e 2024-09-12T12:46:09.486 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-09-12T12:46:09.486 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/fm10k 2024-09-12T12:46:09.486 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-09-12T12:46:09.487 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/i40e 2024-09-12T12:46:09.487 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-09-12T12:46:09.487 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/iavf 2024-09-12T12:46:09.487 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-09-12T12:46:09.487 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igb 2024-09-12T12:46:09.487 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-09-12T12:46:09.487 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igbvf 2024-09-12T12:46:09.487 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-09-12T12:46:09.487 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbe 2024-09-12T12:46:09.487 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-09-12T12:46:09.487 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbevf 2024-09-12T12:46:09.488 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-09-12T12:46:09.488 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libeth 2024-09-12T12:46:09.488 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libeth/libeth.ko 2024-09-12T12:46:09.488 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libie 2024-09-12T12:46:09.488 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libie/libie.ko 2024-09-12T12:46:09.488 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/jme.ko 2024-09-12T12:46:09.488 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell 2024-09-12T12:46:09.488 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-09-12T12:46:09.488 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/skge.ko 2024-09-12T12:46:09.488 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-09-12T12:46:09.488 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox 2024-09-12T12:46:09.489 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4 2024-09-12T12:46:09.489 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-09-12T12:46:09.489 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-09-12T12:46:09.489 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5 2024-09-12T12:46:09.489 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-09-12T12:46:09.489 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-09-12T12:46:09.489 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel 2024-09-12T12:46:09.489 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-09-12T12:46:09.489 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom 2024-09-12T12:46:09.489 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom/myri10ge 2024-09-12T12:46:09.489 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-09-12T12:46:09.490 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi 2024-09-12T12:46:09.490 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-09-12T12:46:09.490 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-09-12T12:46:09.490 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/neterion 2024-09-12T12:46:09.490 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-09-12T12:46:09.490 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/nvidia 2024-09-12T12:46:09.490 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-09-12T12:46:09.490 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines 2024-09-12T12:46:09.490 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-09-12T12:46:09.490 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-09-12T12:46:09.490 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic 2024-09-12T12:46:09.491 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/netxen 2024-09-12T12:46:09.491 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-09-12T12:46:09.491 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-09-12T12:46:09.491 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-09-12T12:46:09.491 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-09-12T12:46:09.491 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rdc 2024-09-12T12:46:09.491 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-09-12T12:46:09.491 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek 2024-09-12T12:46:09.491 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-09-12T12:46:09.491 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-09-12T12:46:09.491 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/atp.ko 2024-09-12T12:46:09.491 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-09-12T12:46:09.491 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rocker 2024-09-12T12:46:09.492 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-09-12T12:46:09.492 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sfc 2024-09-12T12:46:09.492 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-09-12T12:46:09.492 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/silan 2024-09-12T12:46:09.492 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-09-12T12:46:09.492 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis 2024-09-12T12:46:09.492 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis/sis190.ko 2024-09-12T12:46:09.492 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis/sis900.ko 2024-09-12T12:46:09.492 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc 2024-09-12T12:46:09.492 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-09-12T12:46:09.492 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-09-12T12:46:09.493 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-09-12T12:46:09.493 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-09-12T12:46:09.493 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro 2024-09-12T12:46:09.493 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac 2024-09-12T12:46:09.493 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-09-12T12:46:09.493 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-09-12T12:46:09.493 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun 2024-09-12T12:46:09.493 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/cassini.ko 2024-09-12T12:46:09.493 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/niu.ko 2024-09-12T12:46:09.493 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/sungem.ko 2024-09-12T12:46:09.493 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-09-12T12:46:09.494 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/tehuti 2024-09-12T12:46:09.494 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-09-12T12:46:09.494 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ti 2024-09-12T12:46:09.494 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ti/tlan.ko 2024-09-12T12:46:09.494 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via 2024-09-12T12:46:09.494 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-09-12T12:46:09.494 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-09-12T12:46:09.494 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet 2024-09-12T12:46:09.494 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-09-12T12:46:09.494 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-09-12T12:46:09.494 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/xircom 2024-09-12T12:46:09.495 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-09-12T12:46:09.495 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/geneve.ko 2024-09-12T12:46:09.495 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/hyperv 2024-09-12T12:46:09.495 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-09-12T12:46:09.495 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ieee802154 2024-09-12T12:46:09.495 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ieee802154/fakelb.ko 2024-09-12T12:46:09.495 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ifb.ko 2024-09-12T12:46:09.495 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ipvlan 2024-09-12T12:46:09.495 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ipvlan/ipvlan.ko 2024-09-12T12:46:09.495 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/macvlan.ko 2024-09-12T12:46:09.495 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/macvtap.ko 2024-09-12T12:46:09.496 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio 2024-09-12T12:46:09.496 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio.ko 2024-09-12T12:46:09.496 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-09-12T12:46:09.496 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-09-12T12:46:09.496 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-regmap.ko 2024-09-12T12:46:09.496 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mii.ko 2024-09-12T12:46:09.496 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/net_failover.ko 2024-09-12T12:46:09.496 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/netconsole.ko 2024-09-12T12:46:09.496 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/nlmon.ko 2024-09-12T12:46:09.496 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs 2024-09-12T12:46:09.496 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs/pcs-lynx.ko 2024-09-12T12:46:09.497 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-09-12T12:46:09.497 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy 2024-09-12T12:46:09.497 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/amd.ko 2024-09-12T12:46:09.497 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-09-12T12:46:09.497 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-09-12T12:46:09.497 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm7xxx.ko 2024-09-12T12:46:09.497 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm87xx.ko 2024-09-12T12:46:09.497 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/broadcom.ko 2024-09-12T12:46:09.497 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/cicada.ko 2024-09-12T12:46:09.497 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/davicom.ko 2024-09-12T12:46:09.498 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/dp83640.ko 2024-09-12T12:46:09.498 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/et1011c.ko 2024-09-12T12:46:09.498 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/icplus.ko 2024-09-12T12:46:09.498 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/lxt.ko 2024-09-12T12:46:09.498 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/marvell.ko 2024-09-12T12:46:09.498 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/micrel.ko 2024-09-12T12:46:09.498 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/national.ko 2024-09-12T12:46:09.498 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/phylink.ko 2024-09-12T12:46:09.498 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/qsemi.ko 2024-09-12T12:46:09.498 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/realtek.ko 2024-09-12T12:46:09.498 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/smsc.ko 2024-09-12T12:46:09.498 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/ste10Xp.ko 2024-09-12T12:46:09.499 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/vitesse.ko 2024-09-12T12:46:09.499 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp 2024-09-12T12:46:09.499 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/bsd_comp.ko 2024-09-12T12:46:09.499 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_async.ko 2024-09-12T12:46:09.499 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_deflate.ko 2024-09-12T12:46:09.499 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_generic.ko 2024-09-12T12:46:09.499 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_mppe.ko 2024-09-12T12:46:09.499 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_synctty.ko 2024-09-12T12:46:09.499 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pppoe.ko 2024-09-12T12:46:09.499 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pppox.ko 2024-09-12T12:46:09.499 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pptp.ko 2024-09-12T12:46:09.499 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/slip 2024-09-12T12:46:09.500 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/slip/slhc.ko 2024-09-12T12:46:09.500 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/slip/slip.ko 2024-09-12T12:46:09.500 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/sungem_phy.ko 2024-09-12T12:46:09.500 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/tap.ko 2024-09-12T12:46:09.500 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team 2024-09-12T12:46:09.500 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team.ko 2024-09-12T12:46:09.500 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_activebackup.ko 2024-09-12T12:46:09.500 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_broadcast.ko 2024-09-12T12:46:09.500 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-09-12T12:46:09.500 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_random.ko 2024-09-12T12:46:09.500 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-09-12T12:46:09.501 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/tun.ko 2024-09-12T12:46:09.501 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/veth.ko 2024-09-12T12:46:09.501 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/virtio_net.ko 2024-09-12T12:46:09.501 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vmxnet3 2024-09-12T12:46:09.501 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-09-12T12:46:09.501 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vxlan 2024-09-12T12:46:09.501 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vxlan/vxlan.ko 2024-09-12T12:46:09.501 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/xen-netback 2024-09-12T12:46:09.501 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/xen-netback/xen-netback.ko 2024-09-12T12:46:09.501 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/xen-netfront.ko 2024-09-12T12:46:09.501 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ntb 2024-09-12T12:46:09.502 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ntb/ntb.ko 2024-09-12T12:46:09.502 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme 2024-09-12T12:46:09.502 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host 2024-09-12T12:46:09.502 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host/nvme-core.ko 2024-09-12T12:46:09.502 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host/nvme.ko 2024-09-12T12:46:09.502 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport 2024-09-12T12:46:09.502 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport.ko 2024-09-12T12:46:09.502 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_cs.ko 2024-09-12T12:46:09.502 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_pc.ko 2024-09-12T12:46:09.502 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_serial.ko 2024-09-12T12:46:09.502 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci 2024-09-12T12:46:09.503 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug 2024-09-12T12:46:09.503 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-09-12T12:46:09.503 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/pcie 2024-09-12T12:46:09.503 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/pcie/aer_inject.ko 2024-09-12T12:46:09.503 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/xen-pcifront.ko 2024-09-12T12:46:09.503 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia 2024-09-12T12:46:09.503 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/i82092.ko 2024-09-12T12:46:09.503 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/pd6729.ko 2024-09-12T12:46:09.503 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/yenta_socket.ko 2024-09-12T12:46:09.503 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl 2024-09-12T12:46:09.503 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl/intel 2024-09-12T12:46:09.504 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-09-12T12:46:09.504 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform 2024-09-12T12:46:09.504 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86 2024-09-12T12:46:09.504 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/acer-wmi.ko 2024-09-12T12:46:09.504 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/acerhdf.ko 2024-09-12T12:46:09.504 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/apple-gmux.ko 2024-09-12T12:46:09.504 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-laptop.ko 2024-09-12T12:46:09.504 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-09-12T12:46:09.504 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-wmi.ko 2024-09-12T12:46:09.504 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/classmate-laptop.ko 2024-09-12T12:46:09.505 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-09-12T12:46:09.505 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-09-12T12:46:09.505 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-09-12T12:46:09.505 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-09-12T12:46:09.505 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/hdaps.ko 2024-09-12T12:46:09.505 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel 2024-09-12T12:46:09.505 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-09-12T12:46:09.505 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel_ips.ko 2024-09-12T12:46:09.505 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/msi-wmi.ko 2024-09-12T12:46:09.505 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/mxm-wmi.ko 2024-09-12T12:46:09.505 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-09-12T12:46:09.506 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/samsung-laptop.ko 2024-09-12T12:46:09.506 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/samsung-q10.ko 2024-09-12T12:46:09.506 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/topstar-laptop.ko 2024-09-12T12:46:09.506 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-09-12T12:46:09.506 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/toshiba_haps.ko 2024-09-12T12:46:09.506 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/wmi-bmof.ko 2024-09-12T12:46:09.506 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/wmi.ko 2024-09-12T12:46:09.506 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap 2024-09-12T12:46:09.506 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap/intel_rapl_common.ko 2024-09-12T12:46:09.506 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap/intel_rapl_msr.ko 2024-09-12T12:46:09.506 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps 2024-09-12T12:46:09.507 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients 2024-09-12T12:46:09.507 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps-gpio.ko 2024-09-12T12:46:09.507 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps-ldisc.ko 2024-09-12T12:46:09.507 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps_parport.ko 2024-09-12T12:46:09.507 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/pps_core.ko 2024-09-12T12:46:09.507 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp 2024-09-12T12:46:09.507 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp/ptp.ko 2024-09-12T12:46:09.507 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp/ptp_kvm.ko 2024-09-12T12:46:09.507 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras 2024-09-12T12:46:09.507 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd 2024-09-12T12:46:09.507 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/atl 2024-09-12T12:46:09.507 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-09-12T12:46:09.508 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/fmpm.ko 2024-09-12T12:46:09.508 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc 2024-09-12T12:46:09.508 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-bq32k.ko 2024-09-12T12:46:09.508 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1286.ko 2024-09-12T12:46:09.508 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1307.ko 2024-09-12T12:46:09.508 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1374.ko 2024-09-12T12:46:09.508 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1511.ko 2024-09-12T12:46:09.508 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1553.ko 2024-09-12T12:46:09.508 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1672.ko 2024-09-12T12:46:09.508 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1742.ko 2024-09-12T12:46:09.508 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds2404.ko 2024-09-12T12:46:09.509 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds3232.ko 2024-09-12T12:46:09.509 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-em3027.ko 2024-09-12T12:46:09.509 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-fm3130.ko 2024-09-12T12:46:09.509 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-isl12022.ko 2024-09-12T12:46:09.509 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-isl1208.ko 2024-09-12T12:46:09.509 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m41t80.ko 2024-09-12T12:46:09.509 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m48t35.ko 2024-09-12T12:46:09.509 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m48t59.ko 2024-09-12T12:46:09.509 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-max6900.ko 2024-09-12T12:46:09.509 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-msm6242.ko 2024-09-12T12:46:09.509 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf2127.ko 2024-09-12T12:46:09.509 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf85063.ko 2024-09-12T12:46:09.510 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8523.ko 2024-09-12T12:46:09.510 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8563.ko 2024-09-12T12:46:09.510 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8583.ko 2024-09-12T12:46:09.510 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rp5c01.ko 2024-09-12T12:46:09.510 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rs5c372.ko 2024-09-12T12:46:09.510 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-09-12T12:46:09.510 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rx8025.ko 2024-09-12T12:46:09.510 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rx8581.ko 2024-09-12T12:46:09.510 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-09-12T12:46:09.510 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-x1205.ko 2024-09-12T12:46:09.510 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi 2024-09-12T12:46:09.511 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-9xxx.ko 2024-09-12T12:46:09.511 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-sas.ko 2024-09-12T12:46:09.511 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-xxxx.ko 2024-09-12T12:46:09.511 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/BusLogic.ko 2024-09-12T12:46:09.511 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/a100u2w.ko 2024-09-12T12:46:09.511 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aacraid 2024-09-12T12:46:09.511 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aacraid/aacraid.ko 2024-09-12T12:46:09.511 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/advansys.ko 2024-09-12T12:46:09.511 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx 2024-09-12T12:46:09.511 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-09-12T12:46:09.511 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-09-12T12:46:09.512 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/am53c974.ko 2024-09-12T12:46:09.512 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/arcmsr 2024-09-12T12:46:09.512 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-09-12T12:46:09.512 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/atp870u.ko 2024-09-12T12:46:09.512 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/be2iscsi 2024-09-12T12:46:09.512 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-09-12T12:46:09.512 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bfa 2024-09-12T12:46:09.512 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bfa/bfa.ko 2024-09-12T12:46:09.512 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2fc 2024-09-12T12:46:09.512 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-09-12T12:46:09.512 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2i 2024-09-12T12:46:09.512 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-09-12T12:46:09.513 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ch.ko 2024-09-12T12:46:09.513 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/csiostor 2024-09-12T12:46:09.513 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/csiostor/csiostor.ko 2024-09-12T12:46:09.513 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi 2024-09-12T12:46:09.513 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb3i 2024-09-12T12:46:09.513 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-09-12T12:46:09.513 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb4i 2024-09-12T12:46:09.513 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-09-12T12:46:09.513 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-09-12T12:46:09.513 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/dc395x.ko 2024-09-12T12:46:09.513 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler 2024-09-12T12:46:09.514 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-09-12T12:46:09.514 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-09-12T12:46:09.514 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-09-12T12:46:09.514 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-09-12T12:46:09.514 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/dmx3191d.ko 2024-09-12T12:46:09.514 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esas2r 2024-09-12T12:46:09.514 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esas2r/esas2r.ko 2024-09-12T12:46:09.514 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esp_scsi.ko 2024-09-12T12:46:09.514 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe 2024-09-12T12:46:09.514 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe/fcoe.ko 2024-09-12T12:46:09.514 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-09-12T12:46:09.514 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fnic 2024-09-12T12:46:09.515 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fnic/fnic.ko 2024-09-12T12:46:09.515 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hpsa.ko 2024-09-12T12:46:09.515 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hptiop.ko 2024-09-12T12:46:09.515 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hv_storvsc.ko 2024-09-12T12:46:09.515 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/initio.ko 2024-09-12T12:46:09.515 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ipr.ko 2024-09-12T12:46:09.515 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ips.ko 2024-09-12T12:46:09.515 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/isci 2024-09-12T12:46:09.515 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/isci/isci.ko 2024-09-12T12:46:09.515 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-09-12T12:46:09.515 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/iscsi_tcp.ko 2024-09-12T12:46:09.516 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libfc 2024-09-12T12:46:09.516 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libfc/libfc.ko 2024-09-12T12:46:09.516 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libiscsi.ko 2024-09-12T12:46:09.516 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libiscsi_tcp.ko 2024-09-12T12:46:09.516 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libsas 2024-09-12T12:46:09.516 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libsas/libsas.ko 2024-09-12T12:46:09.516 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/lpfc 2024-09-12T12:46:09.516 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/lpfc/lpfc.ko 2024-09-12T12:46:09.516 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid 2024-09-12T12:46:09.516 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid.ko 2024-09-12T12:46:09.516 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-09-12T12:46:09.516 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-09-12T12:46:09.517 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-09-12T12:46:09.517 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mpt3sas 2024-09-12T12:46:09.517 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-09-12T12:46:09.517 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvsas 2024-09-12T12:46:09.517 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvsas/mvsas.ko 2024-09-12T12:46:09.517 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvumi.ko 2024-09-12T12:46:09.517 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pm8001 2024-09-12T12:46:09.517 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-09-12T12:46:09.518 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pmcraid.ko 2024-09-12T12:46:09.518 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla1280.ko 2024-09-12T12:46:09.518 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx 2024-09-12T12:46:09.518 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-09-12T12:46:09.518 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-09-12T12:46:09.518 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla4xxx 2024-09-12T12:46:09.518 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-09-12T12:46:09.518 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/raid_class.ko 2024-09-12T12:46:09.518 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_debug.ko 2024-09-12T12:46:09.518 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_fc.ko 2024-09-12T12:46:09.518 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-09-12T12:46:09.518 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_sas.ko 2024-09-12T12:46:09.518 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_spi.ko 2024-09-12T12:46:09.519 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_srp.ko 2024-09-12T12:46:09.519 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ses.ko 2024-09-12T12:46:09.519 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/st.ko 2024-09-12T12:46:09.519 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/stex.ko 2024-09-12T12:46:09.519 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/sym53c8xx_2 2024-09-12T12:46:09.519 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-09-12T12:46:09.519 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/virtio_scsi.ko 2024-09-12T12:46:09.519 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/vmw_pvscsi.ko 2024-09-12T12:46:09.519 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/wd719x.ko 2024-09-12T12:46:09.519 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ssb 2024-09-12T12:46:09.519 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ssb/ssb.ko 2024-09-12T12:46:09.519 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target 2024-09-12T12:46:09.520 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/iscsi 2024-09-12T12:46:09.520 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-09-12T12:46:09.520 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/loopback 2024-09-12T12:46:09.520 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/loopback/tcm_loop.ko 2024-09-12T12:46:09.520 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_file.ko 2024-09-12T12:46:09.520 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_iblock.ko 2024-09-12T12:46:09.520 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_mod.ko 2024-09-12T12:46:09.520 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_pscsi.ko 2024-09-12T12:46:09.520 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_user.ko 2024-09-12T12:46:09.520 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/tcm_fc 2024-09-12T12:46:09.521 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-09-12T12:46:09.521 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal 2024-09-12T12:46:09.521 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel 2024-09-12T12:46:09.521 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal 2024-09-12T12:46:09.521 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-09-12T12:46:09.521 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-09-12T12:46:09.521 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-09-12T12:46:09.521 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-09-12T12:46:09.521 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-09-12T12:46:09.521 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-09-12T12:46:09.521 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-09-12T12:46:09.522 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-09-12T12:46:09.522 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-09-12T12:46:09.522 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-09-12T12:46:09.522 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-09-12T12:46:09.522 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-09-12T12:46:09.522 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-09-12T12:46:09.522 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-09-12T12:46:09.522 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-09-12T12:46:09.522 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-09-12T12:46:09.522 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-09-12T12:46:09.522 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-09-12T12:46:09.522 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty 2024-09-12T12:46:09.522 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/ipwireless 2024-09-12T12:46:09.523 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-09-12T12:46:09.523 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/n_gsm.ko 2024-09-12T12:46:09.523 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/n_hdlc.ko 2024-09-12T12:46:09.523 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/nozomi.ko 2024-09-12T12:46:09.523 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial 2024-09-12T12:46:09.523 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/8250 2024-09-12T12:46:09.523 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-09-12T12:46:09.523 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/arc_uart.ko 2024-09-12T12:46:09.523 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/jsm 2024-09-12T12:46:09.523 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/jsm/jsm.ko 2024-09-12T12:46:09.523 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs 2024-09-12T12:46:09.523 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/core 2024-09-12T12:46:09.524 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/core/ufshcd-core.ko 2024-09-12T12:46:09.524 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/host 2024-09-12T12:46:09.524 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-09-12T12:46:09.524 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio 2024-09-12T12:46:09.524 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio.ko 2024-09-12T12:46:09.524 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_aec.ko 2024-09-12T12:46:09.524 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_cif.ko 2024-09-12T12:46:09.524 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_pci_generic.ko 2024-09-12T12:46:09.524 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_sercos3.ko 2024-09-12T12:46:09.524 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb 2024-09-12T12:46:09.524 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class 2024-09-12T12:46:09.525 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/cdc-acm.ko 2024-09-12T12:46:09.525 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/cdc-wdm.ko 2024-09-12T12:46:09.525 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/usblp.ko 2024-09-12T12:46:09.525 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/usbtmc.ko 2024-09-12T12:46:09.525 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host 2024-09-12T12:46:09.525 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/sl811-hcd.ko 2024-09-12T12:46:09.525 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image 2024-09-12T12:46:09.525 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image/mdc800.ko 2024-09-12T12:46:09.525 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image/microtek.ko 2024-09-12T12:46:09.525 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc 2024-09-12T12:46:09.525 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/adutux.ko 2024-09-12T12:46:09.525 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/appledisplay.ko 2024-09-12T12:46:09.526 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/emi26.ko 2024-09-12T12:46:09.526 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/emi62.ko 2024-09-12T12:46:09.526 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ezusb.ko 2024-09-12T12:46:09.526 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/idmouse.ko 2024-09-12T12:46:09.526 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/iowarrior.ko 2024-09-12T12:46:09.526 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/isight_firmware.ko 2024-09-12T12:46:09.526 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ldusb.ko 2024-09-12T12:46:09.526 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/legousbtower.ko 2024-09-12T12:46:09.526 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/sisusbvga 2024-09-12T12:46:09.526 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-09-12T12:46:09.526 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/trancevibrator.ko 2024-09-12T12:46:09.526 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usb3503.ko 2024-09-12T12:46:09.526 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usblcd.ko 2024-09-12T12:46:09.526 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usbsevseg.ko 2024-09-12T12:46:09.527 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/uss720.ko 2024-09-12T12:46:09.527 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/yurex.ko 2024-09-12T12:46:09.527 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy 2024-09-12T12:46:09.527 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy/phy-generic.ko 2024-09-12T12:46:09.527 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial 2024-09-12T12:46:09.527 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/aircable.ko 2024-09-12T12:46:09.527 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ark3116.ko 2024-09-12T12:46:09.527 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/belkin_sa.ko 2024-09-12T12:46:09.527 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ch341.ko 2024-09-12T12:46:09.527 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cp210x.ko 2024-09-12T12:46:09.527 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cyberjack.ko 2024-09-12T12:46:09.528 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cypress_m8.ko 2024-09-12T12:46:09.528 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/digi_acceleport.ko 2024-09-12T12:46:09.528 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/empeg.ko 2024-09-12T12:46:09.528 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ftdi_sio.ko 2024-09-12T12:46:09.528 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/garmin_gps.ko 2024-09-12T12:46:09.528 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/io_edgeport.ko 2024-09-12T12:46:09.528 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/io_ti.ko 2024-09-12T12:46:09.528 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ipaq.ko 2024-09-12T12:46:09.528 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ipw.ko 2024-09-12T12:46:09.528 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ir-usb.ko 2024-09-12T12:46:09.528 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-09-12T12:46:09.528 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/keyspan.ko 2024-09-12T12:46:09.528 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/keyspan_pda.ko 2024-09-12T12:46:09.529 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/kl5kusb105.ko 2024-09-12T12:46:09.529 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/kobil_sct.ko 2024-09-12T12:46:09.529 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mct_u232.ko 2024-09-12T12:46:09.529 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mos7720.ko 2024-09-12T12:46:09.529 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mos7840.ko 2024-09-12T12:46:09.529 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/navman.ko 2024-09-12T12:46:09.529 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/omninet.ko 2024-09-12T12:46:09.529 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/opticon.ko 2024-09-12T12:46:09.529 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/option.ko 2024-09-12T12:46:09.529 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/oti6858.ko 2024-09-12T12:46:09.529 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/pl2303.ko 2024-09-12T12:46:09.529 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/qcaux.ko 2024-09-12T12:46:09.529 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/qcserial.ko 2024-09-12T12:46:09.530 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/quatech2.ko 2024-09-12T12:46:09.530 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/safe_serial.ko 2024-09-12T12:46:09.530 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/sierra.ko 2024-09-12T12:46:09.530 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/spcp8x5.ko 2024-09-12T12:46:09.530 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ssu100.ko 2024-09-12T12:46:09.530 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/symbolserial.ko 2024-09-12T12:46:09.530 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-09-12T12:46:09.530 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-09-12T12:46:09.530 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb_debug.ko 2024-09-12T12:46:09.530 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb_wwan.ko 2024-09-12T12:46:09.530 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/visor.ko 2024-09-12T12:46:09.531 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/whiteheat.ko 2024-09-12T12:46:09.531 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/xsens_mt.ko 2024-09-12T12:46:09.531 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage 2024-09-12T12:46:09.531 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/uas.ko 2024-09-12T12:46:09.531 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-alauda.ko 2024-09-12T12:46:09.531 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-cypress.ko 2024-09-12T12:46:09.531 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-datafab.ko 2024-09-12T12:46:09.531 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-09-12T12:46:09.531 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-freecom.ko 2024-09-12T12:46:09.531 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-isd200.ko 2024-09-12T12:46:09.531 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-09-12T12:46:09.532 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-karma.ko 2024-09-12T12:46:09.532 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-onetouch.ko 2024-09-12T12:46:09.532 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-realtek.ko 2024-09-12T12:46:09.532 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-sddr09.ko 2024-09-12T12:46:09.532 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-sddr55.ko 2024-09-12T12:46:09.532 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-usbat.ko 2024-09-12T12:46:09.532 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/usb-storage.ko 2024-09-12T12:46:09.532 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip 2024-09-12T12:46:09.532 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/usbip-core.ko 2024-09-12T12:46:09.532 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/usbip-host.ko 2024-09-12T12:46:09.532 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-09-12T12:46:09.532 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio 2024-09-12T12:46:09.533 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci 2024-09-12T12:46:09.533 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-09-12T12:46:09.533 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci/vfio-pci.ko 2024-09-12T12:46:09.533 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/vfio.ko 2024-09-12T12:46:09.533 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-09-12T12:46:09.533 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost 2024-09-12T12:46:09.533 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost.ko 2024-09-12T12:46:09.533 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_iotlb.ko 2024-09-12T12:46:09.533 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_net.ko 2024-09-12T12:46:09.533 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_scsi.ko 2024-09-12T12:46:09.533 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video 2024-09-12T12:46:09.533 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight 2024-09-12T12:46:09.533 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/apple_bl.ko 2024-09-12T12:46:09.534 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/lcd.ko 2024-09-12T12:46:09.534 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/platform_lcd.ko 2024-09-12T12:46:09.534 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio 2024-09-12T12:46:09.534 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio.ko 2024-09-12T12:46:09.534 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_balloon.ko 2024-09-12T12:46:09.534 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_mmio.ko 2024-09-12T12:46:09.534 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_pci.ko 2024-09-12T12:46:09.534 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-09-12T12:46:09.534 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-09-12T12:46:09.534 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_ring.ko 2024-09-12T12:46:09.534 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1 2024-09-12T12:46:09.534 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters 2024-09-12T12:46:09.534 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/ds2482.ko 2024-09-12T12:46:09.535 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/ds2490.ko 2024-09-12T12:46:09.535 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves 2024-09-12T12:46:09.535 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-09-12T12:46:09.535 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-09-12T12:46:09.535 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-09-12T12:46:09.535 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-09-12T12:46:09.535 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-09-12T12:46:09.535 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-09-12T12:46:09.535 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-09-12T12:46:09.535 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-09-12T12:46:09.535 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-09-12T12:46:09.536 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_smem.ko 2024-09-12T12:46:09.536 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_therm.ko 2024-09-12T12:46:09.536 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/wire.ko 2024-09-12T12:46:09.536 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog 2024-09-12T12:46:09.536 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/alim1535_wdt.ko 2024-09-12T12:46:09.536 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/alim7101_wdt.ko 2024-09-12T12:46:09.536 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/f71808e_wdt.ko 2024-09-12T12:46:09.536 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/hpwdt.ko 2024-09-12T12:46:09.536 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/i6300esb.ko 2024-09-12T12:46:09.536 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-09-12T12:46:09.536 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/iTCO_wdt.ko 2024-09-12T12:46:09.537 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ib700wdt.ko 2024-09-12T12:46:09.537 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ibmasr.ko 2024-09-12T12:46:09.537 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-09-12T12:46:09.537 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/it8712f_wdt.ko 2024-09-12T12:46:09.537 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/it87_wdt.ko 2024-09-12T12:46:09.537 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/machzwd.ko 2024-09-12T12:46:09.537 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/nv_tco.ko 2024-09-12T12:46:09.537 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/pcwd_pci.ko 2024-09-12T12:46:09.537 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/pcwd_usb.ko 2024-09-12T12:46:09.537 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-09-12T12:46:09.537 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sch311x_wdt.ko 2024-09-12T12:46:09.538 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/softdog.ko 2024-09-12T12:46:09.538 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sp5100_tco.ko 2024-09-12T12:46:09.538 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/via_wdt.ko 2024-09-12T12:46:09.538 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-09-12T12:46:09.538 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83877f_wdt.ko 2024-09-12T12:46:09.538 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83977f_wdt.ko 2024-09-12T12:46:09.538 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/wdt_pci.ko 2024-09-12T12:46:09.538 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/xen_wdt.ko 2024-09-12T12:46:09.538 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen 2024-09-12T12:46:09.538 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-acpi-processor.ko 2024-09-12T12:46:09.538 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-evtchn.ko 2024-09-12T12:46:09.538 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-gntalloc.ko 2024-09-12T12:46:09.538 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-gntdev.ko 2024-09-12T12:46:09.539 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-pciback 2024-09-12T12:46:09.539 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-09-12T12:46:09.539 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-privcmd.ko 2024-09-12T12:46:09.539 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xenfs 2024-09-12T12:46:09.539 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xenfs/xenfs.ko 2024-09-12T12:46:09.539 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs 2024-09-12T12:46:09.539 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/9p 2024-09-12T12:46:09.539 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/9p/9p.ko 2024-09-12T12:46:09.539 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/affs 2024-09-12T12:46:09.539 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/affs/affs.ko 2024-09-12T12:46:09.539 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/befs 2024-09-12T12:46:09.540 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/befs/befs.ko 2024-09-12T12:46:09.540 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/binfmt_misc.ko 2024-09-12T12:46:09.540 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/btrfs 2024-09-12T12:46:09.540 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/btrfs/btrfs.ko 2024-09-12T12:46:09.540 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cachefiles 2024-09-12T12:46:09.540 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cachefiles/cachefiles.ko 2024-09-12T12:46:09.540 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ceph 2024-09-12T12:46:09.540 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ceph/ceph.ko 2024-09-12T12:46:09.540 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/coda 2024-09-12T12:46:09.540 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/coda/coda.ko 2024-09-12T12:46:09.541 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cramfs 2024-09-12T12:46:09.541 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cramfs/cramfs.ko 2024-09-12T12:46:09.541 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/dlm 2024-09-12T12:46:09.541 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/dlm/dlm.ko 2024-09-12T12:46:09.541 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ecryptfs 2024-09-12T12:46:09.541 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ecryptfs/ecryptfs.ko 2024-09-12T12:46:09.541 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/f2fs 2024-09-12T12:46:09.541 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/f2fs/f2fs.ko 2024-09-12T12:46:09.541 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat 2024-09-12T12:46:09.541 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat/fat.ko 2024-09-12T12:46:09.541 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat/msdos.ko 2024-09-12T12:46:09.541 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat/vfat.ko 2024-09-12T12:46:09.542 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fuse 2024-09-12T12:46:09.542 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fuse/cuse.ko 2024-09-12T12:46:09.542 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fuse/fuse.ko 2024-09-12T12:46:09.542 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/gfs2 2024-09-12T12:46:09.542 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/gfs2/gfs2.ko 2024-09-12T12:46:09.542 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfs 2024-09-12T12:46:09.542 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfs/hfs.ko 2024-09-12T12:46:09.542 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfsplus 2024-09-12T12:46:09.542 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfsplus/hfsplus.ko 2024-09-12T12:46:09.542 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/isofs 2024-09-12T12:46:09.542 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/isofs/isofs.ko 2024-09-12T12:46:09.542 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/jfs 2024-09-12T12:46:09.543 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/jfs/jfs.ko 2024-09-12T12:46:09.543 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/lockd 2024-09-12T12:46:09.543 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/lockd/lockd.ko 2024-09-12T12:46:09.543 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/minix 2024-09-12T12:46:09.543 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/minix/minix.ko 2024-09-12T12:46:09.543 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/netfs 2024-09-12T12:46:09.543 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/netfs/netfs.ko 2024-09-12T12:46:09.543 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs 2024-09-12T12:46:09.543 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/blocklayout 2024-09-12T12:46:09.543 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-09-12T12:46:09.543 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/filelayout 2024-09-12T12:46:09.543 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-09-12T12:46:09.544 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/flexfilelayout 2024-09-12T12:46:09.544 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-09-12T12:46:09.544 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfs.ko 2024-09-12T12:46:09.544 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfsv3.ko 2024-09-12T12:46:09.544 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfsv4.ko 2024-09-12T12:46:09.544 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common 2024-09-12T12:46:09.544 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common/grace.ko 2024-09-12T12:46:09.544 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common/nfs_acl.ko 2024-09-12T12:46:09.544 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfsd 2024-09-12T12:46:09.544 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfsd/nfsd.ko 2024-09-12T12:46:09.544 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nilfs2 2024-09-12T12:46:09.544 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nilfs2/nilfs2.ko 2024-09-12T12:46:09.545 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls 2024-09-12T12:46:09.545 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-celtic.ko 2024-09-12T12:46:09.545 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-centeuro.ko 2024-09-12T12:46:09.545 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-croatian.ko 2024-09-12T12:46:09.545 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-cyrillic.ko 2024-09-12T12:46:09.545 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-gaelic.ko 2024-09-12T12:46:09.545 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-greek.ko 2024-09-12T12:46:09.545 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-iceland.ko 2024-09-12T12:46:09.545 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-inuit.ko 2024-09-12T12:46:09.545 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-roman.ko 2024-09-12T12:46:09.545 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-romanian.ko 2024-09-12T12:46:09.545 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-turkish.ko 2024-09-12T12:46:09.546 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1250.ko 2024-09-12T12:46:09.546 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1251.ko 2024-09-12T12:46:09.546 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1255.ko 2024-09-12T12:46:09.546 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp737.ko 2024-09-12T12:46:09.546 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp775.ko 2024-09-12T12:46:09.546 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp850.ko 2024-09-12T12:46:09.546 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp852.ko 2024-09-12T12:46:09.546 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp855.ko 2024-09-12T12:46:09.546 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp857.ko 2024-09-12T12:46:09.546 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp860.ko 2024-09-12T12:46:09.546 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp861.ko 2024-09-12T12:46:09.546 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp862.ko 2024-09-12T12:46:09.546 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp863.ko 2024-09-12T12:46:09.547 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp864.ko 2024-09-12T12:46:09.547 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp865.ko 2024-09-12T12:46:09.547 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp866.ko 2024-09-12T12:46:09.547 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp869.ko 2024-09-12T12:46:09.547 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp874.ko 2024-09-12T12:46:09.547 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp932.ko 2024-09-12T12:46:09.547 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp936.ko 2024-09-12T12:46:09.547 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp949.ko 2024-09-12T12:46:09.547 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp950.ko 2024-09-12T12:46:09.547 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_euc-jp.ko 2024-09-12T12:46:09.548 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-1.ko 2024-09-12T12:46:09.548 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-13.ko 2024-09-12T12:46:09.548 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-14.ko 2024-09-12T12:46:09.548 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-15.ko 2024-09-12T12:46:09.548 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-2.ko 2024-09-12T12:46:09.548 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-3.ko 2024-09-12T12:46:09.548 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-4.ko 2024-09-12T12:46:09.548 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-5.ko 2024-09-12T12:46:09.548 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-6.ko 2024-09-12T12:46:09.548 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-7.ko 2024-09-12T12:46:09.548 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-9.ko 2024-09-12T12:46:09.548 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-r.ko 2024-09-12T12:46:09.548 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-ru.ko 2024-09-12T12:46:09.549 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-u.ko 2024-09-12T12:46:09.549 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_ucs2_utils.ko 2024-09-12T12:46:09.549 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_utf8.ko 2024-09-12T12:46:09.549 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2 2024-09-12T12:46:09.549 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/cluster 2024-09-12T12:46:09.549 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-09-12T12:46:09.549 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlm 2024-09-12T12:46:09.549 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-09-12T12:46:09.549 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlmfs 2024-09-12T12:46:09.549 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-09-12T12:46:09.549 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2.ko 2024-09-12T12:46:09.550 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-09-12T12:46:09.550 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-09-12T12:46:09.550 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-09-12T12:46:09.550 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/overlayfs 2024-09-12T12:46:09.550 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/overlayfs/overlay.ko 2024-09-12T12:46:09.550 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/pstore 2024-09-12T12:46:09.550 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/pstore/ramoops.ko 2024-09-12T12:46:09.550 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/reiserfs 2024-09-12T12:46:09.550 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/reiserfs/reiserfs.ko 2024-09-12T12:46:09.550 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/romfs 2024-09-12T12:46:09.551 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/romfs/romfs.ko 2024-09-12T12:46:09.551 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb 2024-09-12T12:46:09.551 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/client 2024-09-12T12:46:09.551 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/client/cifs.ko 2024-09-12T12:46:09.551 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common 2024-09-12T12:46:09.551 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common/cifs_arc4.ko 2024-09-12T12:46:09.551 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common/cifs_md4.ko 2024-09-12T12:46:09.551 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/squashfs 2024-09-12T12:46:09.551 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/squashfs/squashfs.ko 2024-09-12T12:46:09.551 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/sysv 2024-09-12T12:46:09.551 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/sysv/sysv.ko 2024-09-12T12:46:09.551 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/udf 2024-09-12T12:46:09.552 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/udf/udf.ko 2024-09-12T12:46:09.552 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ufs 2024-09-12T12:46:09.552 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ufs/ufs.ko 2024-09-12T12:46:09.552 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/xfs 2024-09-12T12:46:09.552 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/xfs/xfs.ko 2024-09-12T12:46:09.552 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel 2024-09-12T12:46:09.552 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/rcu 2024-09-12T12:46:09.552 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/rcu/rcutorture.ko 2024-09-12T12:46:09.552 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/torture.ko 2024-09-12T12:46:09.552 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/trace 2024-09-12T12:46:09.552 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-09-12T12:46:09.552 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib 2024-09-12T12:46:09.553 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/asn1_encoder.ko 2024-09-12T12:46:09.553 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crc-itu-t.ko 2024-09-12T12:46:09.553 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crc8.ko 2024-09-12T12:46:09.553 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto 2024-09-12T12:46:09.553 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/libaescfb.ko 2024-09-12T12:46:09.553 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/libarc4.ko 2024-09-12T12:46:09.553 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/libdes.ko 2024-09-12T12:46:09.553 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/libcrc32c.ko 2024-09-12T12:46:09.553 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lru_cache.ko 2024-09-12T12:46:09.553 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4 2024-09-12T12:46:09.553 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4/lz4_compress.ko 2024-09-12T12:46:09.553 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4/lz4hc_compress.ko 2024-09-12T12:46:09.554 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/math 2024-09-12T12:46:09.554 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/math/cordic.ko 2024-09-12T12:46:09.554 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/raid6 2024-09-12T12:46:09.554 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/raid6/raid6_pq.ko 2024-09-12T12:46:09.554 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/reed_solomon 2024-09-12T12:46:09.554 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/reed_solomon/reed_solomon.ko 2024-09-12T12:46:09.554 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_bm.ko 2024-09-12T12:46:09.554 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_fsm.ko 2024-09-12T12:46:09.554 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_kmp.ko 2024-09-12T12:46:09.554 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/zstd 2024-09-12T12:46:09.554 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/zstd/zstd_compress.ko 2024-09-12T12:46:09.555 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/mm 2024-09-12T12:46:09.555 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/mm/hwpoison-inject.ko 2024-09-12T12:46:09.555 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net 2024-09-12T12:46:09.555 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan 2024-09-12T12:46:09.555 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/6lowpan.ko 2024-09-12T12:46:09.555 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_dest.ko 2024-09-12T12:46:09.555 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_fragment.ko 2024-09-12T12:46:09.555 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_hop.ko 2024-09-12T12:46:09.555 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_ipv6.ko 2024-09-12T12:46:09.555 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_mobility.ko 2024-09-12T12:46:09.555 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_routing.ko 2024-09-12T12:46:09.556 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_udp.ko 2024-09-12T12:46:09.556 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802 2024-09-12T12:46:09.556 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/garp.ko 2024-09-12T12:46:09.556 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/mrp.ko 2024-09-12T12:46:09.556 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/p8022.ko 2024-09-12T12:46:09.556 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/psnap.ko 2024-09-12T12:46:09.556 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/stp.ko 2024-09-12T12:46:09.556 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/8021q 2024-09-12T12:46:09.556 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/8021q/8021q.ko 2024-09-12T12:46:09.556 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p 2024-09-12T12:46:09.556 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet.ko 2024-09-12T12:46:09.556 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_fd.ko 2024-09-12T12:46:09.556 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_rdma.ko 2024-09-12T12:46:09.557 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_virtio.ko 2024-09-12T12:46:09.557 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/batman-adv 2024-09-12T12:46:09.557 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/batman-adv/batman-adv.ko 2024-09-12T12:46:09.557 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge 2024-09-12T12:46:09.557 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/br_netfilter.ko 2024-09-12T12:46:09.557 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/bridge.ko 2024-09-12T12:46:09.557 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter 2024-09-12T12:46:09.557 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-09-12T12:46:09.557 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_among.ko 2024-09-12T12:46:09.557 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_arp.ko 2024-09-12T12:46:09.557 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-09-12T12:46:09.557 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-09-12T12:46:09.557 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_ip.ko 2024-09-12T12:46:09.558 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-09-12T12:46:09.558 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_limit.ko 2024-09-12T12:46:09.558 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_log.ko 2024-09-12T12:46:09.558 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_mark.ko 2024-09-12T12:46:09.558 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-09-12T12:46:09.558 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-09-12T12:46:09.558 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-09-12T12:46:09.558 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-09-12T12:46:09.558 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_snat.ko 2024-09-12T12:46:09.558 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_stp.ko 2024-09-12T12:46:09.558 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-09-12T12:46:09.558 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-09-12T12:46:09.559 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-09-12T12:46:09.559 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-09-12T12:46:09.559 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtables.ko 2024-09-12T12:46:09.559 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-09-12T12:46:09.559 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-09-12T12:46:09.559 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-09-12T12:46:09.559 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ceph 2024-09-12T12:46:09.559 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ceph/libceph.ko 2024-09-12T12:46:09.559 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/core 2024-09-12T12:46:09.559 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/core/failover.ko 2024-09-12T12:46:09.559 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/core/pktgen.ko 2024-09-12T12:46:09.560 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp 2024-09-12T12:46:09.560 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp.ko 2024-09-12T12:46:09.560 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_diag.ko 2024-09-12T12:46:09.560 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_ipv4.ko 2024-09-12T12:46:09.560 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_ipv6.ko 2024-09-12T12:46:09.560 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dns_resolver 2024-09-12T12:46:09.560 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dns_resolver/dns_resolver.ko 2024-09-12T12:46:09.560 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa 2024-09-12T12:46:09.560 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/dsa_core.ko 2024-09-12T12:46:09.560 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_brcm.ko 2024-09-12T12:46:09.560 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_dsa.ko 2024-09-12T12:46:09.560 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_none.ko 2024-09-12T12:46:09.560 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_trailer.ko 2024-09-12T12:46:09.561 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154 2024-09-12T12:46:09.561 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/6lowpan 2024-09-12T12:46:09.561 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-09-12T12:46:09.561 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/ieee802154.ko 2024-09-12T12:46:09.561 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/ieee802154_socket.ko 2024-09-12T12:46:09.561 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4 2024-09-12T12:46:09.561 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ah4.ko 2024-09-12T12:46:09.561 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/esp4.ko 2024-09-12T12:46:09.561 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/fou.ko 2024-09-12T12:46:09.561 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/gre.ko 2024-09-12T12:46:09.562 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/inet_diag.ko 2024-09-12T12:46:09.562 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_gre.ko 2024-09-12T12:46:09.562 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_tunnel.ko 2024-09-12T12:46:09.562 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_vti.ko 2024-09-12T12:46:09.562 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ipcomp.ko 2024-09-12T12:46:09.562 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ipip.ko 2024-09-12T12:46:09.562 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter 2024-09-12T12:46:09.562 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arp_tables.ko 2024-09-12T12:46:09.562 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-09-12T12:46:09.562 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-09-12T12:46:09.562 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-09-12T12:46:09.562 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-09-12T12:46:09.562 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-09-12T12:46:09.563 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-09-12T12:46:09.563 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-09-12T12:46:09.563 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-09-12T12:46:09.563 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-09-12T12:46:09.563 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_security.ko 2024-09-12T12:46:09.563 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-09-12T12:46:09.563 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-09-12T12:46:09.563 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-09-12T12:46:09.563 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-09-12T12:46:09.563 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-09-12T12:46:09.563 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-09-12T12:46:09.563 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-09-12T12:46:09.563 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-09-12T12:46:09.564 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-09-12T12:46:09.564 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-09-12T12:46:09.564 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_bic.ko 2024-09-12T12:46:09.564 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_dctcp.ko 2024-09-12T12:46:09.564 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_diag.ko 2024-09-12T12:46:09.564 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_highspeed.ko 2024-09-12T12:46:09.564 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_htcp.ko 2024-09-12T12:46:09.564 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_hybla.ko 2024-09-12T12:46:09.564 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_illinois.ko 2024-09-12T12:46:09.564 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_lp.ko 2024-09-12T12:46:09.564 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_scalable.ko 2024-09-12T12:46:09.565 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_vegas.ko 2024-09-12T12:46:09.565 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_veno.ko 2024-09-12T12:46:09.565 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_westwood.ko 2024-09-12T12:46:09.565 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_yeah.ko 2024-09-12T12:46:09.565 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tunnel4.ko 2024-09-12T12:46:09.565 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/udp_diag.ko 2024-09-12T12:46:09.565 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/udp_tunnel.ko 2024-09-12T12:46:09.565 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/xfrm4_tunnel.ko 2024-09-12T12:46:09.565 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6 2024-09-12T12:46:09.565 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ah6.ko 2024-09-12T12:46:09.565 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/esp6.ko 2024-09-12T12:46:09.565 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/fou6.ko 2024-09-12T12:46:09.565 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_tunnel.ko 2024-09-12T12:46:09.566 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-09-12T12:46:09.566 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_vti.ko 2024-09-12T12:46:09.566 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ipcomp6.ko 2024-09-12T12:46:09.566 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter 2024-09-12T12:46:09.566 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-09-12T12:46:09.566 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-09-12T12:46:09.566 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-09-12T12:46:09.566 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-09-12T12:46:09.566 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-09-12T12:46:09.566 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-09-12T12:46:09.566 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-09-12T12:46:09.566 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-09-12T12:46:09.566 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-09-12T12:46:09.567 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-09-12T12:46:09.567 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-09-12T12:46:09.567 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-09-12T12:46:09.567 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-09-12T12:46:09.567 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-09-12T12:46:09.567 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-09-12T12:46:09.567 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-09-12T12:46:09.567 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-09-12T12:46:09.567 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-09-12T12:46:09.567 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-09-12T12:46:09.567 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-09-12T12:46:09.567 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-09-12T12:46:09.568 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-09-12T12:46:09.568 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-09-12T12:46:09.568 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-09-12T12:46:09.568 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/sit.ko 2024-09-12T12:46:09.568 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/tunnel6.ko 2024-09-12T12:46:09.568 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/xfrm6_tunnel.ko 2024-09-12T12:46:09.568 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/key 2024-09-12T12:46:09.568 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/key/af_key.ko 2024-09-12T12:46:09.568 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp 2024-09-12T12:46:09.568 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_core.ko 2024-09-12T12:46:09.568 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_debugfs.ko 2024-09-12T12:46:09.569 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_eth.ko 2024-09-12T12:46:09.569 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ip.ko 2024-09-12T12:46:09.569 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ip6.ko 2024-09-12T12:46:09.569 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_netlink.ko 2024-09-12T12:46:09.569 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ppp.ko 2024-09-12T12:46:09.569 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/llc 2024-09-12T12:46:09.569 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/llc/llc.ko 2024-09-12T12:46:09.569 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mac802154 2024-09-12T12:46:09.569 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mac802154/mac802154.ko 2024-09-12T12:46:09.569 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mpls 2024-09-12T12:46:09.569 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mpls/mpls_gso.ko 2024-09-12T12:46:09.569 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter 2024-09-12T12:46:09.569 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset 2024-09-12T12:46:09.569 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set.ko 2024-09-12T12:46:09.570 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-09-12T12:46:09.570 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-09-12T12:46:09.570 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-09-12T12:46:09.570 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-09-12T12:46:09.570 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-09-12T12:46:09.570 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-09-12T12:46:09.570 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-09-12T12:46:09.570 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-09-12T12:46:09.570 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-09-12T12:46:09.570 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-09-12T12:46:09.570 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-09-12T12:46:09.570 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-09-12T12:46:09.571 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-09-12T12:46:09.571 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-09-12T12:46:09.571 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-09-12T12:46:09.571 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs 2024-09-12T12:46:09.571 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs.ko 2024-09-12T12:46:09.571 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-09-12T12:46:09.571 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-09-12T12:46:09.571 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-09-12T12:46:09.571 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-09-12T12:46:09.571 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-09-12T12:46:09.571 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-09-12T12:46:09.571 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-09-12T12:46:09.572 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-09-12T12:46:09.572 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-09-12T12:46:09.572 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-09-12T12:46:09.572 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-09-12T12:46:09.572 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-09-12T12:46:09.572 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-09-12T12:46:09.572 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conncount.ko 2024-09-12T12:46:09.572 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack.ko 2024-09-12T12:46:09.572 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-09-12T12:46:09.572 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-09-12T12:46:09.572 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-09-12T12:46:09.572 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_h323.ko 2024-09-12T12:46:09.572 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_irc.ko 2024-09-12T12:46:09.573 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-09-12T12:46:09.573 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-09-12T12:46:09.573 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-09-12T12:46:09.573 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_sane.ko 2024-09-12T12:46:09.573 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_sip.ko 2024-09-12T12:46:09.573 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-09-12T12:46:09.573 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-09-12T12:46:09.573 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_dup_netdev.ko 2024-09-12T12:46:09.573 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_flow_table.ko 2024-09-12T12:46:09.573 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_flow_table_inet.ko 2024-09-12T12:46:09.573 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_log_syslog.ko 2024-09-12T12:46:09.573 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat.ko 2024-09-12T12:46:09.574 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_amanda.ko 2024-09-12T12:46:09.574 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_ftp.ko 2024-09-12T12:46:09.574 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_irc.ko 2024-09-12T12:46:09.574 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_sip.ko 2024-09-12T12:46:09.574 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_tftp.ko 2024-09-12T12:46:09.574 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_synproxy_core.ko 2024-09-12T12:46:09.574 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_tables.ko 2024-09-12T12:46:09.574 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink.ko 2024-09-12T12:46:09.574 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_acct.ko 2024-09-12T12:46:09.574 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_log.ko 2024-09-12T12:46:09.574 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_osf.ko 2024-09-12T12:46:09.574 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_queue.ko 2024-09-12T12:46:09.574 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_chain_nat.ko 2024-09-12T12:46:09.574 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_compat.ko 2024-09-12T12:46:09.575 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_ct.ko 2024-09-12T12:46:09.575 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_dup_netdev.ko 2024-09-12T12:46:09.575 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_fib.ko 2024-09-12T12:46:09.575 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_fwd_netdev.ko 2024-09-12T12:46:09.575 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_hash.ko 2024-09-12T12:46:09.575 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_limit.ko 2024-09-12T12:46:09.575 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_log.ko 2024-09-12T12:46:09.575 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_masq.ko 2024-09-12T12:46:09.575 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_nat.ko 2024-09-12T12:46:09.575 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_numgen.ko 2024-09-12T12:46:09.575 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_queue.ko 2024-09-12T12:46:09.575 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_quota.ko 2024-09-12T12:46:09.575 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_redir.ko 2024-09-12T12:46:09.576 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_reject.ko 2024-09-12T12:46:09.576 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_reject_inet.ko 2024-09-12T12:46:09.576 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_socket.ko 2024-09-12T12:46:09.576 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_tunnel.ko 2024-09-12T12:46:09.576 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_AUDIT.ko 2024-09-12T12:46:09.576 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CHECKSUM.ko 2024-09-12T12:46:09.576 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CLASSIFY.ko 2024-09-12T12:46:09.576 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-09-12T12:46:09.576 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CT.ko 2024-09-12T12:46:09.576 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_DSCP.ko 2024-09-12T12:46:09.576 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_HL.ko 2024-09-12T12:46:09.576 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_HMARK.ko 2024-09-12T12:46:09.577 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_IDLETIMER.ko 2024-09-12T12:46:09.577 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_LED.ko 2024-09-12T12:46:09.577 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_LOG.ko 2024-09-12T12:46:09.577 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_MASQUERADE.ko 2024-09-12T12:46:09.577 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NETMAP.ko 2024-09-12T12:46:09.577 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NFLOG.ko 2024-09-12T12:46:09.577 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NFQUEUE.ko 2024-09-12T12:46:09.577 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_RATEEST.ko 2024-09-12T12:46:09.577 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_REDIRECT.ko 2024-09-12T12:46:09.577 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_SECMARK.ko 2024-09-12T12:46:09.577 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TCPMSS.ko 2024-09-12T12:46:09.578 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-09-12T12:46:09.578 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TEE.ko 2024-09-12T12:46:09.578 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TPROXY.ko 2024-09-12T12:46:09.578 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TRACE.ko 2024-09-12T12:46:09.578 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_addrtype.ko 2024-09-12T12:46:09.578 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_bpf.ko 2024-09-12T12:46:09.578 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cgroup.ko 2024-09-12T12:46:09.578 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cluster.ko 2024-09-12T12:46:09.578 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_comment.ko 2024-09-12T12:46:09.578 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connbytes.ko 2024-09-12T12:46:09.578 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connlabel.ko 2024-09-12T12:46:09.578 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connlimit.ko 2024-09-12T12:46:09.578 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connmark.ko 2024-09-12T12:46:09.579 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_conntrack.ko 2024-09-12T12:46:09.579 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cpu.ko 2024-09-12T12:46:09.579 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_dccp.ko 2024-09-12T12:46:09.579 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_devgroup.ko 2024-09-12T12:46:09.579 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_dscp.ko 2024-09-12T12:46:09.579 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ecn.ko 2024-09-12T12:46:09.579 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_esp.ko 2024-09-12T12:46:09.579 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_hashlimit.ko 2024-09-12T12:46:09.579 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_helper.ko 2024-09-12T12:46:09.579 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_hl.ko 2024-09-12T12:46:09.579 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ipcomp.ko 2024-09-12T12:46:09.579 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_iprange.ko 2024-09-12T12:46:09.579 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ipvs.ko 2024-09-12T12:46:09.580 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_l2tp.ko 2024-09-12T12:46:09.580 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_length.ko 2024-09-12T12:46:09.580 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_limit.ko 2024-09-12T12:46:09.580 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_mac.ko 2024-09-12T12:46:09.580 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_mark.ko 2024-09-12T12:46:09.580 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_multiport.ko 2024-09-12T12:46:09.580 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_nat.ko 2024-09-12T12:46:09.580 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_nfacct.ko 2024-09-12T12:46:09.580 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_osf.ko 2024-09-12T12:46:09.580 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_owner.ko 2024-09-12T12:46:09.580 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_physdev.ko 2024-09-12T12:46:09.580 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_pkttype.ko 2024-09-12T12:46:09.581 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_policy.ko 2024-09-12T12:46:09.581 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_quota.ko 2024-09-12T12:46:09.581 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_rateest.ko 2024-09-12T12:46:09.581 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_realm.ko 2024-09-12T12:46:09.581 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_recent.ko 2024-09-12T12:46:09.581 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_sctp.ko 2024-09-12T12:46:09.581 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_set.ko 2024-09-12T12:46:09.581 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_socket.ko 2024-09-12T12:46:09.581 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_state.ko 2024-09-12T12:46:09.581 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_statistic.ko 2024-09-12T12:46:09.581 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_string.ko 2024-09-12T12:46:09.581 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_tcpmss.ko 2024-09-12T12:46:09.582 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_time.ko 2024-09-12T12:46:09.582 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_u32.ko 2024-09-12T12:46:09.582 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netlink 2024-09-12T12:46:09.582 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netlink/netlink_diag.ko 2024-09-12T12:46:09.582 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/nsh 2024-09-12T12:46:09.582 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/nsh/nsh.ko 2024-09-12T12:46:09.582 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch 2024-09-12T12:46:09.582 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/openvswitch.ko 2024-09-12T12:46:09.582 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-geneve.ko 2024-09-12T12:46:09.582 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-gre.ko 2024-09-12T12:46:09.582 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-vxlan.ko 2024-09-12T12:46:09.582 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/packet 2024-09-12T12:46:09.582 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/packet/af_packet_diag.ko 2024-09-12T12:46:09.583 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds 2024-09-12T12:46:09.583 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds.ko 2024-09-12T12:46:09.583 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds_rdma.ko 2024-09-12T12:46:09.583 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds_tcp.ko 2024-09-12T12:46:09.583 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched 2024-09-12T12:46:09.583 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_csum.ko 2024-09-12T12:46:09.583 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_gact.ko 2024-09-12T12:46:09.583 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_mirred.ko 2024-09-12T12:46:09.583 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_nat.ko 2024-09-12T12:46:09.583 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_pedit.ko 2024-09-12T12:46:09.583 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_police.ko 2024-09-12T12:46:09.583 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_simple.ko 2024-09-12T12:46:09.583 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_skbedit.ko 2024-09-12T12:46:09.584 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_vlan.ko 2024-09-12T12:46:09.584 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_basic.ko 2024-09-12T12:46:09.584 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_bpf.ko 2024-09-12T12:46:09.584 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_flow.ko 2024-09-12T12:46:09.584 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_fw.ko 2024-09-12T12:46:09.584 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_route.ko 2024-09-12T12:46:09.584 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_u32.ko 2024-09-12T12:46:09.584 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_cmp.ko 2024-09-12T12:46:09.584 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_ipset.ko 2024-09-12T12:46:09.584 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_meta.ko 2024-09-12T12:46:09.585 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_nbyte.ko 2024-09-12T12:46:09.585 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_text.ko 2024-09-12T12:46:09.585 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_u32.ko 2024-09-12T12:46:09.585 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_choke.ko 2024-09-12T12:46:09.585 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_codel.ko 2024-09-12T12:46:09.585 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_drr.ko 2024-09-12T12:46:09.585 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_fq.ko 2024-09-12T12:46:09.585 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_gred.ko 2024-09-12T12:46:09.585 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_hfsc.ko 2024-09-12T12:46:09.585 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_hhf.ko 2024-09-12T12:46:09.585 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_htb.ko 2024-09-12T12:46:09.585 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_ingress.ko 2024-09-12T12:46:09.585 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_mqprio.ko 2024-09-12T12:46:09.586 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_mqprio_lib.ko 2024-09-12T12:46:09.586 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_multiq.ko 2024-09-12T12:46:09.586 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_netem.ko 2024-09-12T12:46:09.586 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_pie.ko 2024-09-12T12:46:09.586 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_plug.ko 2024-09-12T12:46:09.586 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_prio.ko 2024-09-12T12:46:09.586 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_qfq.ko 2024-09-12T12:46:09.586 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_red.ko 2024-09-12T12:46:09.586 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_sfb.ko 2024-09-12T12:46:09.586 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_sfq.ko 2024-09-12T12:46:09.586 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_tbf.ko 2024-09-12T12:46:09.587 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_teql.ko 2024-09-12T12:46:09.587 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp 2024-09-12T12:46:09.587 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp/sctp.ko 2024-09-12T12:46:09.587 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp/sctp_diag.ko 2024-09-12T12:46:09.587 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc 2024-09-12T12:46:09.587 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss 2024-09-12T12:46:09.587 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-09-12T12:46:09.587 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-09-12T12:46:09.587 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/sunrpc.ko 2024-09-12T12:46:09.587 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/xprtrdma 2024-09-12T12:46:09.587 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-09-12T12:46:09.587 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc 2024-09-12T12:46:09.588 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc/tipc.ko 2024-09-12T12:46:09.588 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc/tipc_diag.ko 2024-09-12T12:46:09.588 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/unix 2024-09-12T12:46:09.588 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/unix/unix_diag.ko 2024-09-12T12:46:09.588 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock 2024-09-12T12:46:09.588 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-09-12T12:46:09.588 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-09-12T12:46:09.588 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock.ko 2024-09-12T12:46:09.588 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock_diag.ko 2024-09-12T12:46:09.588 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock_loopback.ko 2024-09-12T12:46:09.588 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm 2024-09-12T12:46:09.588 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm/xfrm_ipcomp.ko 2024-09-12T12:46:09.588 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security 2024-09-12T12:46:09.589 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys 2024-09-12T12:46:09.589 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys/encrypted-keys 2024-09-12T12:46:09.589 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-09-12T12:46:09.589 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys/trusted-keys 2024-09-12T12:46:09.589 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys/trusted-keys/trusted.ko 2024-09-12T12:46:09.589 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.alias 2024-09-12T12:46:09.589 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.alias.bin 2024-09-12T12:46:09.589 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin 2024-09-12T12:46:09.589 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.alias.bin 2024-09-12T12:46:09.589 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.bin 2024-09-12T12:46:09.589 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.modinfo 2024-09-12T12:46:09.590 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.dep 2024-09-12T12:46:09.590 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.dep.bin 2024-09-12T12:46:09.590 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.devname 2024-09-12T12:46:09.590 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.order 2024-09-12T12:46:09.590 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.softdep 2024-09-12T12:46:09.590 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.symbols 2024-09-12T12:46:09.590 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.symbols.bin 2024-09-12T12:46:09.590 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.weakdep 2024-09-12T12:46:09.590 INFO:teuthology.orchestra.run.smithi130.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/vmlinuz 2024-09-12T12:46:09.594 DEBUG:teuthology.task.kernel:get_image_version: 6.11.0-rc5-g2ca44a2f35ca 2024-09-12T12:46:09.594 DEBUG:teuthology.orchestra.run.smithi130:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-09-12T12:46:09.617 INFO:teuthology.orchestra.run.smithi130.stdout:postinstall scriptlet (using /bin/sh): 2024-09-12T12:46:09.617 INFO:teuthology.orchestra.run.smithi130.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-09-12T12:46:09.617 INFO:teuthology.orchestra.run.smithi130.stdout: /usr/bin/kernel-install add 6.11.0-rc5-g2ca44a2f35ca /lib/modules/6.11.0-rc5-g2ca44a2f35ca/vmlinuz 2024-09-12T12:46:09.617 INFO:teuthology.orchestra.run.smithi130.stdout:fi 2024-09-12T12:46:09.618 INFO:teuthology.orchestra.run.smithi130.stdout:for file in vmlinuz System.map config; do 2024-09-12T12:46:09.618 INFO:teuthology.orchestra.run.smithi130.stdout: if ! cmp --silent "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/${file}" "/boot/${file}-6.11.0-rc5-g2ca44a2f35ca"; then 2024-09-12T12:46:09.618 INFO:teuthology.orchestra.run.smithi130.stdout: cp "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/${file}" "/boot/${file}-6.11.0-rc5-g2ca44a2f35ca" 2024-09-12T12:46:09.618 INFO:teuthology.orchestra.run.smithi130.stdout: fi 2024-09-12T12:46:09.618 INFO:teuthology.orchestra.run.smithi130.stdout:done 2024-09-12T12:46:09.618 INFO:teuthology.orchestra.run.smithi130.stdout:if [ -d "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/dtb" ] && \ 2024-09-12T12:46:09.618 INFO:teuthology.orchestra.run.smithi130.stdout: ! diff -rq "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/dtb" "/boot/dtb-6.11.0-rc5-g2ca44a2f35ca" >/dev/null 2>&1; then 2024-09-12T12:46:09.618 INFO:teuthology.orchestra.run.smithi130.stdout: rm -rf "/boot/dtb-6.11.0-rc5-g2ca44a2f35ca" 2024-09-12T12:46:09.619 INFO:teuthology.orchestra.run.smithi130.stdout: cp -r "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/dtb" "/boot/dtb-6.11.0-rc5-g2ca44a2f35ca" 2024-09-12T12:46:09.619 INFO:teuthology.orchestra.run.smithi130.stdout:fi 2024-09-12T12:46:09.619 INFO:teuthology.orchestra.run.smithi130.stdout:if [ ! -e "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.dep" ]; then 2024-09-12T12:46:09.619 INFO:teuthology.orchestra.run.smithi130.stdout: /usr/sbin/depmod 6.11.0-rc5-g2ca44a2f35ca 2024-09-12T12:46:09.619 INFO:teuthology.orchestra.run.smithi130.stdout:fi 2024-09-12T12:46:09.619 INFO:teuthology.orchestra.run.smithi130.stdout:preuninstall scriptlet (using /bin/sh): 2024-09-12T12:46:09.619 INFO:teuthology.orchestra.run.smithi130.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-09-12T12:46:09.619 INFO:teuthology.orchestra.run.smithi130.stdout:kernel-install remove 6.11.0-rc5-g2ca44a2f35ca 2024-09-12T12:46:09.619 INFO:teuthology.orchestra.run.smithi130.stdout:fi 2024-09-12T12:46:09.620 DEBUG:teuthology.orchestra.run.smithi130:> sudo rpm -qi grub2-tools 2024-09-12T12:46:09.695 INFO:teuthology.orchestra.run.smithi130.stdout:Name : grub2-tools 2024-09-12T12:46:09.695 INFO:teuthology.orchestra.run.smithi130.stdout:Epoch : 1 2024-09-12T12:46:09.695 INFO:teuthology.orchestra.run.smithi130.stdout:Version : 2.06 2024-09-12T12:46:09.695 INFO:teuthology.orchestra.run.smithi130.stdout:Release : 82.el9 2024-09-12T12:46:09.695 INFO:teuthology.orchestra.run.smithi130.stdout:Architecture: x86_64 2024-09-12T12:46:09.695 INFO:teuthology.orchestra.run.smithi130.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-12T12:46:09.695 INFO:teuthology.orchestra.run.smithi130.stdout:Group : Unspecified 2024-09-12T12:46:09.695 INFO:teuthology.orchestra.run.smithi130.stdout:Size : 8274670 2024-09-12T12:46:09.696 INFO:teuthology.orchestra.run.smithi130.stdout:License : GPLv3+ 2024-09-12T12:46:09.696 INFO:teuthology.orchestra.run.smithi130.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-12T12:46:09.696 INFO:teuthology.orchestra.run.smithi130.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-12T12:46:09.696 INFO:teuthology.orchestra.run.smithi130.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-12T12:46:09.696 INFO:teuthology.orchestra.run.smithi130.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-12T12:46:09.696 INFO:teuthology.orchestra.run.smithi130.stdout:Packager : builder@centos.org 2024-09-12T12:46:09.696 INFO:teuthology.orchestra.run.smithi130.stdout:Vendor : CentOS 2024-09-12T12:46:09.696 INFO:teuthology.orchestra.run.smithi130.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-12T12:46:09.696 INFO:teuthology.orchestra.run.smithi130.stdout:Summary : Support tools for GRUB. 2024-09-12T12:46:09.696 INFO:teuthology.orchestra.run.smithi130.stdout:Description : 2024-09-12T12:46:09.696 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-09-12T12:46:09.696 INFO:teuthology.orchestra.run.smithi130.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-12T12:46:09.696 INFO:teuthology.orchestra.run.smithi130.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-12T12:46:09.697 INFO:teuthology.orchestra.run.smithi130.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-12T12:46:09.697 INFO:teuthology.orchestra.run.smithi130.stdout:hardware devices. 2024-09-12T12:46:09.697 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-09-12T12:46:09.697 INFO:teuthology.orchestra.run.smithi130.stdout:This subpackage provides tools for support of all platforms. 2024-09-12T12:46:09.698 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-12T12:46:09.698 INFO:teuthology.task.kernel:Updating grub on smithi130 to boot 6.11.0-rc5-g2ca44a2f35ca 2024-09-12T12:46:09.698 DEBUG:teuthology.orchestra.run.smithi130:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-12T12:46:10.260 INFO:teuthology.orchestra.run.smithi130.stderr:Generating grub configuration file ... 2024-09-12T12:46:11.601 INFO:teuthology.orchestra.run.smithi130.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-12T12:46:11.629 INFO:teuthology.orchestra.run.smithi130.stderr:done 2024-09-12T12:46:11.631 DEBUG:teuthology.orchestra.run.smithi130:> mktemp 2024-09-12T12:46:11.646 INFO:teuthology.orchestra.run.smithi130.stdout:/tmp/tmp.iKWaItJCWK 2024-09-12T12:46:11.647 DEBUG:teuthology.orchestra.run.smithi130:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.iKWaItJCWK 2024-09-12T12:46:11.721 DEBUG:teuthology.orchestra.run.smithi130:> sudo chmod 0666 /tmp/tmp.iKWaItJCWK 2024-09-12T12:46:11.868 DEBUG:teuthology.orchestra.remote:smithi130:/tmp/tmp.iKWaItJCWK is 6KB 2024-09-12T12:46:11.887 DEBUG:teuthology.orchestra.run.smithi130:> rm -fr /tmp/tmp.iKWaItJCWK 2024-09-12T12:46:11.902 DEBUG:teuthology.orchestra.run.smithi130:> sudo /bin/ls /boot/loader/entries || true 2024-09-12T12:46:11.965 INFO:teuthology.orchestra.run.smithi130.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-12T12:46:11.965 INFO:teuthology.orchestra.run.smithi130.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-12T12:46:11.965 INFO:teuthology.orchestra.run.smithi130.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-12T12:46:11.965 INFO:teuthology.orchestra.run.smithi130.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-6.11.0-rc5-g2ca44a2f35ca.conf 2024-09-12T12:46:11.966 DEBUG:teuthology.orchestra.run.smithi130:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-6.11.0-rc5-g2ca44a2f35ca 2024-09-12T12:46:12.124 DEBUG:teuthology.orchestra.run.smithi130:> sudo shutdown -r now 2024-09-12T12:46:31.449 INFO:teuthology.task.kernel:Checking client cluster1.mon.a for new kernel version... 2024-09-12T12:46:31.450 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi060.front.sepia.ceph.com' 2024-09-12T12:46:31.451 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi060.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-12T12:46:42.136 INFO:teuthology.task.kernel:Checking client cluster2.mon.a for new kernel version... 2024-09-12T12:46:42.137 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi130.front.sepia.ceph.com' 2024-09-12T12:46:42.137 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi130.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-12T12:46:49.917 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.60 2024-09-12T12:46:58.927 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi060.front.sepia.ceph.com' 2024-09-12T12:46:58.928 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi060.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-12T12:47:00.669 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.130 2024-09-12T12:47:09.673 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi130.front.sepia.ceph.com' 2024-09-12T12:47:09.674 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi130.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-12T12:47:58.977 DEBUG:teuthology.orchestra.remote:timed out 2024-09-12T12:48:09.685 DEBUG:teuthology.orchestra.remote:timed out 2024-09-12T12:48:10.980 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi060.front.sepia.ceph.com' 2024-09-12T12:48:10.981 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi060.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-12T12:48:11.391 DEBUG:teuthology.orchestra.run.smithi060:> true 2024-09-12T12:48:11.916 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi060.front.sepia.ceph.com' 2024-09-12T12:48:11.916 INFO:teuthology.task.kernel:Checking kernel version of cluster1.mon.a, want "6.11.0-rc5-g2ca44a2f35ca"... 2024-09-12T12:48:11.917 DEBUG:teuthology.orchestra.run.smithi060:> uname -r 2024-09-12T12:48:11.951 INFO:teuthology.orchestra.run.smithi060.stdout:6.11.0-rc5-g2ca44a2f35ca 2024-09-12T12:48:11.952 DEBUG:teuthology.task.kernel:current kernel version is 6.11.0-rc5-g2ca44a2f35ca vs 6.11.0-rc5-g2ca44a2f35ca 2024-09-12T12:48:11.952 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-12T12:48:11.952 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-12T12:48:12.953 INFO:teuthology.task.kernel:Enabling kdb on cluster1.mon.a... 2024-09-12T12:48:12.954 DEBUG:teuthology.orchestra.run.smithi060:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-12T12:48:13.055 INFO:teuthology.orchestra.run.smithi060.stdout:ttyS1 2024-09-12T12:48:13.096 DEBUG:teuthology.parallel:result is None 2024-09-12T12:48:21.686 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi130.front.sepia.ceph.com' 2024-09-12T12:48:21.687 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi130.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-12T12:48:22.052 DEBUG:teuthology.orchestra.run.smithi130:> true 2024-09-12T12:48:22.544 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi130.front.sepia.ceph.com' 2024-09-12T12:48:22.545 INFO:teuthology.task.kernel:Checking kernel version of cluster2.mon.a, want "6.11.0-rc5-g2ca44a2f35ca"... 2024-09-12T12:48:22.545 DEBUG:teuthology.orchestra.run.smithi130:> uname -r 2024-09-12T12:48:22.580 INFO:teuthology.orchestra.run.smithi130.stdout:6.11.0-rc5-g2ca44a2f35ca 2024-09-12T12:48:22.580 DEBUG:teuthology.task.kernel:current kernel version is 6.11.0-rc5-g2ca44a2f35ca vs 6.11.0-rc5-g2ca44a2f35ca 2024-09-12T12:48:22.580 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-12T12:48:22.580 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-12T12:48:23.581 INFO:teuthology.task.kernel:Enabling kdb on cluster2.mon.a... 2024-09-12T12:48:23.582 DEBUG:teuthology.orchestra.run.smithi130:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-12T12:48:23.677 INFO:teuthology.orchestra.run.smithi130.stdout:ttyS1 2024-09-12T12:48:23.717 DEBUG:teuthology.parallel:result is None 2024-09-12T12:48:23.717 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-12T12:48:23.728 INFO:teuthology.task.internal:Creating test directory... 2024-09-12T12:48:23.728 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-12T12:48:23.733 DEBUG:teuthology.orchestra.run.smithi130:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-12T12:48:23.770 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-12T12:48:23.781 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-12T12:48:23.790 INFO:teuthology.task.internal:Creating archive directory... 2024-09-12T12:48:23.791 DEBUG:teuthology.orchestra.run.smithi060:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-12T12:48:23.816 DEBUG:teuthology.orchestra.run.smithi130:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-12T12:48:23.870 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-12T12:48:23.880 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-12T12:48:23.880 DEBUG:teuthology.orchestra.run.smithi060:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-12T12:48:23.900 DEBUG:teuthology.orchestra.run.smithi130:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-12T12:48:23.955 INFO:teuthology.orchestra.run.smithi060.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-12T12:48:23.968 INFO:teuthology.orchestra.run.smithi130.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-12T12:48:23.978 INFO:teuthology.orchestra.run.smithi060.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-12T12:48:23.990 INFO:teuthology.orchestra.run.smithi130.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-12T12:48:23.994 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-12T12:48:24.004 INFO:teuthology.task.internal:Configuring sudo... 2024-09-12T12:48:24.004 DEBUG:teuthology.orchestra.run.smithi060:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-12T12:48:24.008 DEBUG:teuthology.orchestra.run.smithi130:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-12T12:48:24.064 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-12T12:48:24.073 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-12T12:48:24.073 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-12T12:48:24.077 DEBUG:teuthology.orchestra.run.smithi130:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-12T12:48:24.112 DEBUG:teuthology.orchestra.run.smithi060:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-12T12:48:24.212 DEBUG:teuthology.orchestra.run.smithi060:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-12T12:48:24.299 DEBUG:teuthology.orchestra.run.smithi060:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-12T12:48:24.363 DEBUG:teuthology.orchestra.run.smithi060:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-12T12:48:24.433 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-09-12T12:48:24.433 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-12T12:48:24.489 DEBUG:teuthology.orchestra.run.smithi130:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-12T12:48:24.550 DEBUG:teuthology.orchestra.run.smithi130:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-12T12:48:24.649 DEBUG:teuthology.orchestra.run.smithi130:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-12T12:48:24.710 DEBUG:teuthology.orchestra.run.smithi130:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-12T12:48:24.781 DEBUG:teuthology.orchestra.run.smithi130:> set -ex 2024-09-12T12:48:24.781 DEBUG:teuthology.orchestra.run.smithi130:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-12T12:48:24.835 DEBUG:teuthology.orchestra.run.smithi060:> sudo service rsyslog restart 2024-09-12T12:48:24.840 DEBUG:teuthology.orchestra.run.smithi130:> sudo service rsyslog restart 2024-09-12T12:48:24.932 INFO:teuthology.orchestra.run.smithi060.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-12T12:48:24.951 INFO:teuthology.orchestra.run.smithi130.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-12T12:48:25.355 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-12T12:48:25.362 INFO:teuthology.task.internal:Starting timer... 2024-09-12T12:48:25.362 INFO:teuthology.run_tasks:Running task pcp... 2024-09-12T12:48:25.372 INFO:teuthology.run_tasks:Running task selinux... 2024-09-12T12:48:25.380 DEBUG:teuthology.orchestra.run.smithi060:> sudo service auditd rotate 2024-09-12T12:48:25.504 INFO:teuthology.orchestra.run.smithi060.stdout:Rotating logs: 2024-09-12T12:48:25.508 DEBUG:teuthology.orchestra.run.smithi130:> sudo service auditd rotate 2024-09-12T12:48:25.606 INFO:teuthology.orchestra.run.smithi130.stdout:Rotating logs: 2024-09-12T12:48:25.610 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-12T12:48:25.610 DEBUG:teuthology.orchestra.run.smithi060:> /usr/sbin/getenforce 2024-09-12T12:48:25.654 INFO:teuthology.orchestra.run.smithi060.stdout:Permissive 2024-09-12T12:48:25.654 DEBUG:teuthology.orchestra.run.smithi130:> /usr/sbin/getenforce 2024-09-12T12:48:25.692 INFO:teuthology.orchestra.run.smithi130.stdout:Permissive 2024-09-12T12:48:25.693 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi060.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi130.front.sepia.ceph.com': 'permissive'} 2024-09-12T12:48:25.693 DEBUG:teuthology.orchestra.run.smithi060:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-12T12:48:25.751 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T12:48:25.751 DEBUG:teuthology.orchestra.run.smithi130:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-12T12:48:25.810 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T12:48:25.810 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-12T12:48:25.810 DEBUG:teuthology.orchestra.run.smithi060:> sudo /usr/sbin/setenforce permissive 2024-09-12T12:48:25.870 DEBUG:teuthology.orchestra.run.smithi130:> sudo /usr/sbin/setenforce permissive 2024-09-12T12:48:25.961 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-12T12:48:25.971 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-09-12T12:48:25.974 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-09-12T12:48:26.099 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-12T12:48:26.110 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-12T12:48:26.112 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi060.front.sepia.ceph.com,smithi130.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-12T12:53:16.467 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi060.front.sepia.ceph.com'), Remote(name='ubuntu@smithi130.front.sepia.ceph.com')] 2024-09-12T12:53:16.469 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi060.front.sepia.ceph.com' 2024-09-12T12:53:16.470 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi060.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-12T12:53:16.564 DEBUG:teuthology.orchestra.run.smithi060:> true 2024-09-12T12:53:16.679 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi060.front.sepia.ceph.com' 2024-09-12T12:53:16.680 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi130.front.sepia.ceph.com' 2024-09-12T12:53:16.680 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi130.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-12T12:53:16.764 DEBUG:teuthology.orchestra.run.smithi130:> true 2024-09-12T12:53:16.866 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi130.front.sepia.ceph.com' 2024-09-12T12:53:16.866 INFO:teuthology.run_tasks:Running task clock... 2024-09-12T12:53:16.877 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-12T12:53:16.878 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-12T12:53:16.878 DEBUG:teuthology.orchestra.run.smithi060:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-12T12:53:16.882 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-12T12:53:16.883 DEBUG:teuthology.orchestra.run.smithi130:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-12T12:53:16.942 INFO:teuthology.orchestra.run.smithi060.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-12T12:53:16.966 INFO:teuthology.orchestra.run.smithi130.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-12T12:53:16.973 INFO:teuthology.orchestra.run.smithi060.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-12T12:53:16.995 INFO:teuthology.orchestra.run.smithi130.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-12T12:53:17.024 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: ntpd: command not found 2024-09-12T12:53:17.045 INFO:teuthology.orchestra.run.smithi130.stderr:sudo: ntpd: command not found 2024-09-12T12:53:17.048 INFO:teuthology.orchestra.run.smithi060.stdout:506 Cannot talk to daemon 2024-09-12T12:53:17.069 INFO:teuthology.orchestra.run.smithi130.stdout:506 Cannot talk to daemon 2024-09-12T12:53:17.077 INFO:teuthology.orchestra.run.smithi060.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-12T12:53:17.098 INFO:teuthology.orchestra.run.smithi130.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-12T12:53:17.107 INFO:teuthology.orchestra.run.smithi060.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-12T12:53:17.128 INFO:teuthology.orchestra.run.smithi130.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-12T12:53:17.212 INFO:teuthology.orchestra.run.smithi060.stderr:bash: line 1: ntpq: command not found 2024-09-12T12:53:17.215 INFO:teuthology.orchestra.run.smithi130.stderr:bash: line 1: ntpq: command not found 2024-09-12T12:53:17.218 INFO:teuthology.orchestra.run.smithi060.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-12T12:53:17.218 INFO:teuthology.orchestra.run.smithi060.stdout:=============================================================================== 2024-09-12T12:53:17.219 INFO:teuthology.orchestra.run.smithi060.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T12:53:17.219 INFO:teuthology.orchestra.run.smithi060.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T12:53:17.219 INFO:teuthology.orchestra.run.smithi060.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T12:53:17.219 INFO:teuthology.orchestra.run.smithi060.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T12:53:17.222 INFO:teuthology.orchestra.run.smithi130.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-12T12:53:17.222 INFO:teuthology.orchestra.run.smithi130.stdout:=============================================================================== 2024-09-12T12:53:17.222 INFO:teuthology.orchestra.run.smithi130.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T12:53:17.222 INFO:teuthology.orchestra.run.smithi130.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T12:53:17.222 INFO:teuthology.orchestra.run.smithi130.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T12:53:17.222 INFO:teuthology.orchestra.run.smithi130.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T12:53:17.223 INFO:teuthology.run_tasks:Running task install... 2024-09-12T12:53:17.232 DEBUG:teuthology.task.install:project ceph 2024-09-12T12:53:17.232 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '086c737fa75c22d521a3932c59b31c5e1d1c9ef4'}, 'extra_system_packages': ['pv']} 2024-09-12T12:53:17.232 DEBUG:teuthology.task.install:config {'extra_packages': ['rbd-mirror'], 'flavor': 'default', 'sha1': '086c737fa75c22d521a3932c59b31c5e1d1c9ef4', 'extra_system_packages': ['pv']} 2024-09-12T12:53:17.232 INFO:teuthology.task.install:Using flavor: default 2024-09-12T12:53:17.238 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-12T12:53:17.238 INFO:teuthology.task.install:extra packages: ['rbd-mirror'] 2024-09-12T12:53:17.239 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': ['rbd-mirror'], 'extra_system_packages': ['pv'], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '086c737fa75c22d521a3932c59b31c5e1d1c9ef4', 'tag': None, 'wait_for_package': False} 2024-09-12T12:53:17.239 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=086c737fa75c22d521a3932c59b31c5e1d1c9ef4 2024-09-12T12:53:17.241 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': ['rbd-mirror'], 'extra_system_packages': ['pv'], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '086c737fa75c22d521a3932c59b31c5e1d1c9ef4', 'tag': None, 'wait_for_package': False} 2024-09-12T12:53:17.241 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=086c737fa75c22d521a3932c59b31c5e1d1c9ef4 2024-09-12T12:53:17.396 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/squid/086c737fa75c22d521a3932c59b31c5e1d1c9ef4/centos/9/flavors/default/ 2024-09-12T12:53:17.396 INFO:teuthology.task.install.rpm:Package version is 19.1.1-314.g086c737f 2024-09-12T12:53:17.399 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/squid/086c737fa75c22d521a3932c59b31c5e1d1c9ef4/centos/9/flavors/default/ 2024-09-12T12:53:17.400 INFO:teuthology.task.install.rpm:Package version is 19.1.1-314.g086c737f 2024-09-12T12:53:17.507 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/squid/086c737fa75c22d521a3932c59b31c5e1d1c9ef4/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/086c737fa75c22d521a3932c59b31c5e1d1c9ef4/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/086c737fa75c22d521a3932c59b31c5e1d1c9ef4/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-12T12:53:17.507 DEBUG:teuthology.orchestra.run.smithi130:> set -ex 2024-09-12T12:53:17.507 DEBUG:teuthology.orchestra.run.smithi130:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-12T12:53:17.515 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/squid/086c737fa75c22d521a3932c59b31c5e1d1c9ef4/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/086c737fa75c22d521a3932c59b31c5e1d1c9ef4/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/086c737fa75c22d521a3932c59b31c5e1d1c9ef4/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-12T12:53:17.515 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-09-12T12:53:17.515 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-12T12:53:17.563 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, rbd-mirror, pv, pv, pv on remote rpm x86_64 2024-09-12T12:53:17.563 DEBUG:teuthology.orchestra.run.smithi130:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/086c737fa75c22d521a3932c59b31c5e1d1c9ef4/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-12T12:53:17.572 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, rbd-mirror, pv, pv, pv, pv on remote rpm x86_64 2024-09-12T12:53:17.572 DEBUG:teuthology.orchestra.run.smithi060:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/086c737fa75c22d521a3932c59b31c5e1d1c9ef4/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-12T12:53:17.660 DEBUG:teuthology.orchestra.run.smithi130:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-12T12:53:17.669 DEBUG:teuthology.orchestra.run.smithi060:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-12T12:53:17.738 DEBUG:teuthology.orchestra.run.smithi130:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-12T12:53:17.748 DEBUG:teuthology.orchestra.run.smithi060:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-12T12:53:17.792 INFO:teuthology.orchestra.run.smithi130.stdout:check_obsoletes = 1 2024-09-12T12:53:17.795 DEBUG:teuthology.orchestra.run.smithi130:> sudo yum clean all 2024-09-12T12:53:17.803 INFO:teuthology.orchestra.run.smithi060.stdout:check_obsoletes = 1 2024-09-12T12:53:17.806 DEBUG:teuthology.orchestra.run.smithi060:> sudo yum clean all 2024-09-12T12:53:18.068 INFO:teuthology.orchestra.run.smithi130.stdout:45 files removed 2024-09-12T12:53:18.088 INFO:teuthology.orchestra.run.smithi060.stdout:45 files removed 2024-09-12T12:53:18.102 DEBUG:teuthology.orchestra.run.smithi130:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd rbd-mirror pv pv pv pv 2024-09-12T12:53:18.122 DEBUG:teuthology.orchestra.run.smithi060:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd rbd-mirror pv pv pv pv 2024-09-12T12:53:18.827 INFO:teuthology.orchestra.run.smithi060.stdout:ceph packages for x86_64 279 kB/s | 83 kB 00:00 2024-09-12T12:53:18.835 INFO:teuthology.orchestra.run.smithi130.stdout:ceph packages for x86_64 244 kB/s | 83 kB 00:00 2024-09-12T12:53:19.136 INFO:teuthology.orchestra.run.smithi060.stdout:ceph noarch packages 45 kB/s | 12 kB 00:00 2024-09-12T12:53:19.203 INFO:teuthology.orchestra.run.smithi130.stdout:ceph noarch packages 36 kB/s | 12 kB 00:00 2024-09-12T12:53:19.420 INFO:teuthology.orchestra.run.smithi060.stdout:ceph source packages 7.6 kB/s | 1.9 kB 00:00 2024-09-12T12:53:19.520 INFO:teuthology.orchestra.run.smithi130.stdout:ceph source packages 6.6 kB/s | 1.9 kB 00:00 2024-09-12T12:53:21.303 INFO:teuthology.orchestra.run.smithi130.stdout:CentOS Stream 9 - BaseOS 4.7 MB/s | 8.2 MB 00:01 2024-09-12T12:53:22.121 INFO:teuthology.orchestra.run.smithi060.stdout:CentOS Stream 9 - BaseOS 3.1 MB/s | 8.2 MB 00:02 2024-09-12T12:53:23.121 INFO:teuthology.orchestra.run.smithi130.stdout:CentOS Stream 9 - AppStream 30 MB/s | 20 MB 00:00 2024-09-12T12:53:24.371 INFO:teuthology.orchestra.run.smithi060.stdout:CentOS Stream 9 - AppStream 20 MB/s | 20 MB 00:01 2024-09-12T12:53:27.805 INFO:teuthology.orchestra.run.smithi130.stdout:CentOS Stream 9 - CRB 9.7 MB/s | 6.5 MB 00:00 2024-09-12T12:53:29.539 INFO:teuthology.orchestra.run.smithi130.stdout:CentOS Stream 9 - Extras packages 43 kB/s | 19 kB 00:00 2024-09-12T12:53:29.605 INFO:teuthology.orchestra.run.smithi060.stdout:CentOS Stream 9 - CRB 9.3 MB/s | 6.5 MB 00:00 2024-09-12T12:53:30.445 INFO:teuthology.orchestra.run.smithi130.stdout:Extra Packages for Enterprise Linux 29 MB/s | 23 MB 00:00 2024-09-12T12:53:31.439 INFO:teuthology.orchestra.run.smithi060.stdout:CentOS Stream 9 - Extras packages 41 kB/s | 19 kB 00:00 2024-09-12T12:53:32.235 INFO:teuthology.orchestra.run.smithi060.stdout:Extra Packages for Enterprise Linux 34 MB/s | 23 MB 00:00 2024-09-12T12:53:36.773 INFO:teuthology.orchestra.run.smithi130.stdout:lab-extras 27 kB/s | 1.7 kB 00:00 2024-09-12T12:53:38.916 INFO:teuthology.orchestra.run.smithi060.stdout:lab-extras 21 kB/s | 1.7 kB 00:00 2024-09-12T12:53:39.288 INFO:teuthology.orchestra.run.smithi130.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-12T12:53:39.288 INFO:teuthology.orchestra.run.smithi130.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-12T12:53:39.352 INFO:teuthology.orchestra.run.smithi130.stdout:Dependencies resolved. 2024-09-12T12:53:39.381 INFO:teuthology.orchestra.run.smithi130.stdout:====================================================================================== 2024-09-12T12:53:39.381 INFO:teuthology.orchestra.run.smithi130.stdout: Package Arch Version Repository Size 2024-09-12T12:53:39.381 INFO:teuthology.orchestra.run.smithi130.stdout:====================================================================================== 2024-09-12T12:53:39.381 INFO:teuthology.orchestra.run.smithi130.stdout:Installing: 2024-09-12T12:53:39.381 INFO:teuthology.orchestra.run.smithi130.stdout: ceph x86_64 2:19.1.1-314.g086c737f.el9 ceph 6.5 k 2024-09-12T12:53:39.381 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-base x86_64 2:19.1.1-314.g086c737f.el9 ceph 5.4 M 2024-09-12T12:53:39.382 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-fuse x86_64 2:19.1.1-314.g086c737f.el9 ceph 896 k 2024-09-12T12:53:39.382 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-314.g086c737f.el9 ceph 145 k 2024-09-12T12:53:39.382 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-mgr x86_64 2:19.1.1-314.g086c737f.el9 ceph 1.5 M 2024-09-12T12:53:39.382 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-mgr-cephadm noarch 2:19.1.1-314.g086c737f.el9 ceph-noarch 146 k 2024-09-12T12:53:39.382 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-mgr-dashboard noarch 2:19.1.1-314.g086c737f.el9 ceph-noarch 3.6 M 2024-09-12T12:53:39.382 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-314.g086c737f.el9 ceph-noarch 7.4 M 2024-09-12T12:53:39.382 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-mgr-rook noarch 2:19.1.1-314.g086c737f.el9 ceph-noarch 49 k 2024-09-12T12:53:39.382 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-radosgw x86_64 2:19.1.1-314.g086c737f.el9 ceph 11 M 2024-09-12T12:53:39.382 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-test x86_64 2:19.1.1-314.g086c737f.el9 ceph 49 M 2024-09-12T12:53:39.382 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-volume noarch 2:19.1.1-314.g086c737f.el9 ceph-noarch 285 k 2024-09-12T12:53:39.382 INFO:teuthology.orchestra.run.smithi130.stdout: cephadm noarch 2:19.1.1-314.g086c737f.el9 ceph-noarch 765 k 2024-09-12T12:53:39.382 INFO:teuthology.orchestra.run.smithi130.stdout: libcephfs-devel x86_64 2:19.1.1-314.g086c737f.el9 ceph 32 k 2024-09-12T12:53:39.383 INFO:teuthology.orchestra.run.smithi130.stdout: libcephfs2 x86_64 2:19.1.1-314.g086c737f.el9 ceph 753 k 2024-09-12T12:53:39.383 INFO:teuthology.orchestra.run.smithi130.stdout: librados-devel x86_64 2:19.1.1-314.g086c737f.el9 ceph 127 k 2024-09-12T12:53:39.383 INFO:teuthology.orchestra.run.smithi130.stdout: pv x86_64 1.6.20-1.el9 epel 65 k 2024-09-12T12:53:39.383 INFO:teuthology.orchestra.run.smithi130.stdout: python3-cephfs x86_64 2:19.1.1-314.g086c737f.el9 ceph 162 k 2024-09-12T12:53:39.383 INFO:teuthology.orchestra.run.smithi130.stdout: python3-rados x86_64 2:19.1.1-314.g086c737f.el9 ceph 320 k 2024-09-12T12:53:39.383 INFO:teuthology.orchestra.run.smithi130.stdout: python3-rbd x86_64 2:19.1.1-314.g086c737f.el9 ceph 301 k 2024-09-12T12:53:39.383 INFO:teuthology.orchestra.run.smithi130.stdout: python3-rgw x86_64 2:19.1.1-314.g086c737f.el9 ceph 99 k 2024-09-12T12:53:39.383 INFO:teuthology.orchestra.run.smithi130.stdout: rbd-fuse x86_64 2:19.1.1-314.g086c737f.el9 ceph 85 k 2024-09-12T12:53:39.383 INFO:teuthology.orchestra.run.smithi130.stdout: rbd-mirror x86_64 2:19.1.1-314.g086c737f.el9 ceph 3.1 M 2024-09-12T12:53:39.383 INFO:teuthology.orchestra.run.smithi130.stdout: rbd-nbd x86_64 2:19.1.1-314.g086c737f.el9 ceph 171 k 2024-09-12T12:53:39.383 INFO:teuthology.orchestra.run.smithi130.stdout:Upgrading: 2024-09-12T12:53:39.383 INFO:teuthology.orchestra.run.smithi130.stdout: librados2 x86_64 2:19.1.1-314.g086c737f.el9 ceph 3.4 M 2024-09-12T12:53:39.384 INFO:teuthology.orchestra.run.smithi130.stdout: librbd1 x86_64 2:19.1.1-314.g086c737f.el9 ceph 3.2 M 2024-09-12T12:53:39.384 INFO:teuthology.orchestra.run.smithi130.stdout:Installing dependencies: 2024-09-12T12:53:39.384 INFO:teuthology.orchestra.run.smithi130.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-12T12:53:39.384 INFO:teuthology.orchestra.run.smithi130.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-12T12:53:39.384 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-common x86_64 2:19.1.1-314.g086c737f.el9 ceph 21 M 2024-09-12T12:53:39.384 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-grafana-dashboards noarch 2:19.1.1-314.g086c737f.el9 ceph-noarch 32 k 2024-09-12T12:53:39.384 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-mds x86_64 2:19.1.1-314.g086c737f.el9 ceph 2.4 M 2024-09-12T12:53:39.384 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-mgr-modules-core noarch 2:19.1.1-314.g086c737f.el9 ceph-noarch 249 k 2024-09-12T12:53:39.384 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-mon x86_64 2:19.1.1-314.g086c737f.el9 ceph 4.7 M 2024-09-12T12:53:39.384 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-osd x86_64 2:19.1.1-314.g086c737f.el9 ceph 17 M 2024-09-12T12:53:39.384 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-prometheus-alerts noarch 2:19.1.1-314.g086c737f.el9 ceph-noarch 16 k 2024-09-12T12:53:39.385 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-selinux x86_64 2:19.1.1-314.g086c737f.el9 ceph 25 k 2024-09-12T12:53:39.385 INFO:teuthology.orchestra.run.smithi130.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-12T12:53:39.385 INFO:teuthology.orchestra.run.smithi130.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-12T12:53:39.385 INFO:teuthology.orchestra.run.smithi130.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-12T12:53:39.385 INFO:teuthology.orchestra.run.smithi130.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-12T12:53:39.385 INFO:teuthology.orchestra.run.smithi130.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-12T12:53:39.385 INFO:teuthology.orchestra.run.smithi130.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-12T12:53:39.385 INFO:teuthology.orchestra.run.smithi130.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-12T12:53:39.385 INFO:teuthology.orchestra.run.smithi130.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-12T12:53:39.385 INFO:teuthology.orchestra.run.smithi130.stdout: libcephsqlite x86_64 2:19.1.1-314.g086c737f.el9 ceph 164 k 2024-09-12T12:53:39.385 INFO:teuthology.orchestra.run.smithi130.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-12T12:53:39.386 INFO:teuthology.orchestra.run.smithi130.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-12T12:53:39.386 INFO:teuthology.orchestra.run.smithi130.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-12T12:53:39.386 INFO:teuthology.orchestra.run.smithi130.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-12T12:53:39.386 INFO:teuthology.orchestra.run.smithi130.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-12T12:53:39.386 INFO:teuthology.orchestra.run.smithi130.stdout: libradosstriper1 x86_64 2:19.1.1-314.g086c737f.el9 ceph 506 k 2024-09-12T12:53:39.386 INFO:teuthology.orchestra.run.smithi130.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-12T12:53:39.386 INFO:teuthology.orchestra.run.smithi130.stdout: librgw2 x86_64 2:19.1.1-314.g086c737f.el9 ceph 5.3 M 2024-09-12T12:53:39.386 INFO:teuthology.orchestra.run.smithi130.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-12T12:53:39.387 INFO:teuthology.orchestra.run.smithi130.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-12T12:53:39.387 INFO:teuthology.orchestra.run.smithi130.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-12T12:53:39.387 INFO:teuthology.orchestra.run.smithi130.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-12T12:53:39.387 INFO:teuthology.orchestra.run.smithi130.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-12T12:53:39.387 INFO:teuthology.orchestra.run.smithi130.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-12T12:53:39.387 INFO:teuthology.orchestra.run.smithi130.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-12T12:53:39.387 INFO:teuthology.orchestra.run.smithi130.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-12T12:53:39.387 INFO:teuthology.orchestra.run.smithi130.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-12T12:53:39.387 INFO:teuthology.orchestra.run.smithi130.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-12T12:53:39.387 INFO:teuthology.orchestra.run.smithi130.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-12T12:53:39.387 INFO:teuthology.orchestra.run.smithi130.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-12T12:53:39.388 INFO:teuthology.orchestra.run.smithi130.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-12T12:53:39.388 INFO:teuthology.orchestra.run.smithi130.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-12T12:53:39.388 INFO:teuthology.orchestra.run.smithi130.stdout: python3-ceph-argparse x86_64 2:19.1.1-314.g086c737f.el9 ceph 45 k 2024-09-12T12:53:39.388 INFO:teuthology.orchestra.run.smithi130.stdout: python3-ceph-common x86_64 2:19.1.1-314.g086c737f.el9 ceph 136 k 2024-09-12T12:53:39.388 INFO:teuthology.orchestra.run.smithi130.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-12T12:53:39.388 INFO:teuthology.orchestra.run.smithi130.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-12T12:53:39.388 INFO:teuthology.orchestra.run.smithi130.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-12T12:53:39.388 INFO:teuthology.orchestra.run.smithi130.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-12T12:53:39.389 INFO:teuthology.orchestra.run.smithi130.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-12T12:53:39.389 INFO:teuthology.orchestra.run.smithi130.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-12T12:53:39.389 INFO:teuthology.orchestra.run.smithi130.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-12T12:53:39.389 INFO:teuthology.orchestra.run.smithi130.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-12T12:53:39.389 INFO:teuthology.orchestra.run.smithi130.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-12T12:53:39.389 INFO:teuthology.orchestra.run.smithi130.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-12T12:53:39.389 INFO:teuthology.orchestra.run.smithi130.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-12T12:53:39.389 INFO:teuthology.orchestra.run.smithi130.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-12T12:53:39.389 INFO:teuthology.orchestra.run.smithi130.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-12T12:53:39.389 INFO:teuthology.orchestra.run.smithi130.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-12T12:53:39.390 INFO:teuthology.orchestra.run.smithi130.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-12T12:53:39.390 INFO:teuthology.orchestra.run.smithi130.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-12T12:53:39.390 INFO:teuthology.orchestra.run.smithi130.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-12T12:53:39.390 INFO:teuthology.orchestra.run.smithi130.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-12T12:53:39.390 INFO:teuthology.orchestra.run.smithi130.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-12T12:53:39.390 INFO:teuthology.orchestra.run.smithi130.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-12T12:53:39.390 INFO:teuthology.orchestra.run.smithi130.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-12T12:53:39.390 INFO:teuthology.orchestra.run.smithi130.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-12T12:53:39.390 INFO:teuthology.orchestra.run.smithi130.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-12T12:53:39.390 INFO:teuthology.orchestra.run.smithi130.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-12T12:53:39.390 INFO:teuthology.orchestra.run.smithi130.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-12T12:53:39.390 INFO:teuthology.orchestra.run.smithi130.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-12T12:53:39.391 INFO:teuthology.orchestra.run.smithi130.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-12T12:53:39.391 INFO:teuthology.orchestra.run.smithi130.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-12T12:53:39.391 INFO:teuthology.orchestra.run.smithi130.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-12T12:53:39.391 INFO:teuthology.orchestra.run.smithi130.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-12T12:53:39.391 INFO:teuthology.orchestra.run.smithi130.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-12T12:53:39.391 INFO:teuthology.orchestra.run.smithi130.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-12T12:53:39.391 INFO:teuthology.orchestra.run.smithi130.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-12T12:53:39.391 INFO:teuthology.orchestra.run.smithi130.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-12T12:53:39.392 INFO:teuthology.orchestra.run.smithi130.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-12T12:53:39.392 INFO:teuthology.orchestra.run.smithi130.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-12T12:53:39.392 INFO:teuthology.orchestra.run.smithi130.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-12T12:53:39.392 INFO:teuthology.orchestra.run.smithi130.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-12T12:53:39.392 INFO:teuthology.orchestra.run.smithi130.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-12T12:53:39.392 INFO:teuthology.orchestra.run.smithi130.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-12T12:53:39.392 INFO:teuthology.orchestra.run.smithi130.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-12T12:53:39.392 INFO:teuthology.orchestra.run.smithi130.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-12T12:53:39.392 INFO:teuthology.orchestra.run.smithi130.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-12T12:53:39.392 INFO:teuthology.orchestra.run.smithi130.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-12T12:53:39.392 INFO:teuthology.orchestra.run.smithi130.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-12T12:53:39.393 INFO:teuthology.orchestra.run.smithi130.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-12T12:53:39.393 INFO:teuthology.orchestra.run.smithi130.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-12T12:53:39.393 INFO:teuthology.orchestra.run.smithi130.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-12T12:53:39.393 INFO:teuthology.orchestra.run.smithi130.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-12T12:53:39.393 INFO:teuthology.orchestra.run.smithi130.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-12T12:53:39.393 INFO:teuthology.orchestra.run.smithi130.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-12T12:53:39.393 INFO:teuthology.orchestra.run.smithi130.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-12T12:53:39.393 INFO:teuthology.orchestra.run.smithi130.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-12T12:53:39.393 INFO:teuthology.orchestra.run.smithi130.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-12T12:53:39.393 INFO:teuthology.orchestra.run.smithi130.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-12T12:53:39.394 INFO:teuthology.orchestra.run.smithi130.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-12T12:53:39.394 INFO:teuthology.orchestra.run.smithi130.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-12T12:53:39.394 INFO:teuthology.orchestra.run.smithi130.stdout:Installing weak dependencies: 2024-09-12T12:53:39.394 INFO:teuthology.orchestra.run.smithi130.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-12T12:53:39.394 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-09-12T12:53:39.394 INFO:teuthology.orchestra.run.smithi130.stdout:Transaction Summary 2024-09-12T12:53:39.394 INFO:teuthology.orchestra.run.smithi130.stdout:====================================================================================== 2024-09-12T12:53:39.394 INFO:teuthology.orchestra.run.smithi130.stdout:Install 125 Packages 2024-09-12T12:53:39.394 INFO:teuthology.orchestra.run.smithi130.stdout:Upgrade 2 Packages 2024-09-12T12:53:39.394 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-09-12T12:53:39.395 INFO:teuthology.orchestra.run.smithi130.stdout:Total download size: 202 M 2024-09-12T12:53:39.395 INFO:teuthology.orchestra.run.smithi130.stdout:Downloading Packages: 2024-09-12T12:53:40.008 INFO:teuthology.orchestra.run.smithi130.stdout:(1/127): ceph-19.1.1-314.g086c737f.el9.x86_64.r 42 kB/s | 6.5 kB 00:00 2024-09-12T12:53:40.225 INFO:teuthology.orchestra.run.smithi130.stdout:(2/127): ceph-fuse-19.1.1-314.g086c737f.el9.x86 4.0 MB/s | 896 kB 00:00 2024-09-12T12:53:40.334 INFO:teuthology.orchestra.run.smithi130.stdout:(3/127): ceph-base-19.1.1-314.g086c737f.el9.x86 11 MB/s | 5.4 MB 00:00 2024-09-12T12:53:40.359 INFO:teuthology.orchestra.run.smithi130.stdout:(4/127): ceph-immutable-object-cache-19.1.1-314 1.1 MB/s | 145 kB 00:00 2024-09-12T12:53:40.493 INFO:teuthology.orchestra.run.smithi130.stdout:(5/127): ceph-mds-19.1.1-314.g086c737f.el9.x86_ 15 MB/s | 2.4 MB 00:00 2024-09-12T12:53:40.552 INFO:teuthology.orchestra.run.smithi130.stdout:(6/127): ceph-mgr-19.1.1-314.g086c737f.el9.x86_ 7.9 MB/s | 1.5 MB 00:00 2024-09-12T12:53:40.734 INFO:teuthology.orchestra.run.smithi130.stdout:(7/127): ceph-mon-19.1.1-314.g086c737f.el9.x86_ 19 MB/s | 4.7 MB 00:00 2024-09-12T12:53:41.227 INFO:teuthology.orchestra.run.smithi130.stdout:(8/127): ceph-common-19.1.1-314.g086c737f.el9.x 15 MB/s | 21 MB 00:01 2024-09-12T12:53:41.302 INFO:teuthology.orchestra.run.smithi130.stdout:(9/127): ceph-selinux-19.1.1-314.g086c737f.el9. 338 kB/s | 25 kB 00:00 2024-09-12T12:53:41.477 INFO:teuthology.orchestra.run.smithi060.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-12T12:53:41.477 INFO:teuthology.orchestra.run.smithi060.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-12T12:53:41.485 INFO:teuthology.orchestra.run.smithi130.stdout:(10/127): ceph-radosgw-19.1.1-314.g086c737f.el9 14 MB/s | 11 MB 00:00 2024-09-12T12:53:41.545 INFO:teuthology.orchestra.run.smithi060.stdout:Dependencies resolved. 2024-09-12T12:53:41.561 INFO:teuthology.orchestra.run.smithi130.stdout:(11/127): libcephfs-devel-19.1.1-314.g086c737f. 425 kB/s | 32 kB 00:00 2024-09-12T12:53:41.562 INFO:teuthology.orchestra.run.smithi060.stdout:====================================================================================== 2024-09-12T12:53:41.563 INFO:teuthology.orchestra.run.smithi060.stdout: Package Arch Version Repository Size 2024-09-12T12:53:41.563 INFO:teuthology.orchestra.run.smithi060.stdout:====================================================================================== 2024-09-12T12:53:41.563 INFO:teuthology.orchestra.run.smithi060.stdout:Installing: 2024-09-12T12:53:41.563 INFO:teuthology.orchestra.run.smithi060.stdout: ceph x86_64 2:19.1.1-314.g086c737f.el9 ceph 6.5 k 2024-09-12T12:53:41.563 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-base x86_64 2:19.1.1-314.g086c737f.el9 ceph 5.4 M 2024-09-12T12:53:41.563 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-fuse x86_64 2:19.1.1-314.g086c737f.el9 ceph 896 k 2024-09-12T12:53:41.563 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-314.g086c737f.el9 ceph 145 k 2024-09-12T12:53:41.563 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mgr x86_64 2:19.1.1-314.g086c737f.el9 ceph 1.5 M 2024-09-12T12:53:41.563 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mgr-cephadm noarch 2:19.1.1-314.g086c737f.el9 ceph-noarch 146 k 2024-09-12T12:53:41.563 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mgr-dashboard noarch 2:19.1.1-314.g086c737f.el9 ceph-noarch 3.6 M 2024-09-12T12:53:41.563 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-314.g086c737f.el9 ceph-noarch 7.4 M 2024-09-12T12:53:41.563 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mgr-rook noarch 2:19.1.1-314.g086c737f.el9 ceph-noarch 49 k 2024-09-12T12:53:41.563 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-radosgw x86_64 2:19.1.1-314.g086c737f.el9 ceph 11 M 2024-09-12T12:53:41.564 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-test x86_64 2:19.1.1-314.g086c737f.el9 ceph 49 M 2024-09-12T12:53:41.564 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-volume noarch 2:19.1.1-314.g086c737f.el9 ceph-noarch 285 k 2024-09-12T12:53:41.564 INFO:teuthology.orchestra.run.smithi060.stdout: cephadm noarch 2:19.1.1-314.g086c737f.el9 ceph-noarch 765 k 2024-09-12T12:53:41.564 INFO:teuthology.orchestra.run.smithi060.stdout: libcephfs-devel x86_64 2:19.1.1-314.g086c737f.el9 ceph 32 k 2024-09-12T12:53:41.564 INFO:teuthology.orchestra.run.smithi060.stdout: libcephfs2 x86_64 2:19.1.1-314.g086c737f.el9 ceph 753 k 2024-09-12T12:53:41.564 INFO:teuthology.orchestra.run.smithi060.stdout: librados-devel x86_64 2:19.1.1-314.g086c737f.el9 ceph 127 k 2024-09-12T12:53:41.564 INFO:teuthology.orchestra.run.smithi060.stdout: pv x86_64 1.6.20-1.el9 epel 65 k 2024-09-12T12:53:41.564 INFO:teuthology.orchestra.run.smithi060.stdout: python3-cephfs x86_64 2:19.1.1-314.g086c737f.el9 ceph 162 k 2024-09-12T12:53:41.564 INFO:teuthology.orchestra.run.smithi060.stdout: python3-rados x86_64 2:19.1.1-314.g086c737f.el9 ceph 320 k 2024-09-12T12:53:41.564 INFO:teuthology.orchestra.run.smithi060.stdout: python3-rbd x86_64 2:19.1.1-314.g086c737f.el9 ceph 301 k 2024-09-12T12:53:41.564 INFO:teuthology.orchestra.run.smithi060.stdout: python3-rgw x86_64 2:19.1.1-314.g086c737f.el9 ceph 99 k 2024-09-12T12:53:41.564 INFO:teuthology.orchestra.run.smithi060.stdout: rbd-fuse x86_64 2:19.1.1-314.g086c737f.el9 ceph 85 k 2024-09-12T12:53:41.564 INFO:teuthology.orchestra.run.smithi060.stdout: rbd-mirror x86_64 2:19.1.1-314.g086c737f.el9 ceph 3.1 M 2024-09-12T12:53:41.565 INFO:teuthology.orchestra.run.smithi060.stdout: rbd-nbd x86_64 2:19.1.1-314.g086c737f.el9 ceph 171 k 2024-09-12T12:53:41.565 INFO:teuthology.orchestra.run.smithi060.stdout:Upgrading: 2024-09-12T12:53:41.565 INFO:teuthology.orchestra.run.smithi060.stdout: librados2 x86_64 2:19.1.1-314.g086c737f.el9 ceph 3.4 M 2024-09-12T12:53:41.565 INFO:teuthology.orchestra.run.smithi060.stdout: librbd1 x86_64 2:19.1.1-314.g086c737f.el9 ceph 3.2 M 2024-09-12T12:53:41.565 INFO:teuthology.orchestra.run.smithi060.stdout:Installing dependencies: 2024-09-12T12:53:41.565 INFO:teuthology.orchestra.run.smithi060.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-12T12:53:41.565 INFO:teuthology.orchestra.run.smithi060.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-12T12:53:41.565 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-common x86_64 2:19.1.1-314.g086c737f.el9 ceph 21 M 2024-09-12T12:53:41.565 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-grafana-dashboards noarch 2:19.1.1-314.g086c737f.el9 ceph-noarch 32 k 2024-09-12T12:53:41.565 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mds x86_64 2:19.1.1-314.g086c737f.el9 ceph 2.4 M 2024-09-12T12:53:41.565 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mgr-modules-core noarch 2:19.1.1-314.g086c737f.el9 ceph-noarch 249 k 2024-09-12T12:53:41.565 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mon x86_64 2:19.1.1-314.g086c737f.el9 ceph 4.7 M 2024-09-12T12:53:41.565 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-osd x86_64 2:19.1.1-314.g086c737f.el9 ceph 17 M 2024-09-12T12:53:41.566 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-prometheus-alerts noarch 2:19.1.1-314.g086c737f.el9 ceph-noarch 16 k 2024-09-12T12:53:41.566 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-selinux x86_64 2:19.1.1-314.g086c737f.el9 ceph 25 k 2024-09-12T12:53:41.566 INFO:teuthology.orchestra.run.smithi060.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-12T12:53:41.566 INFO:teuthology.orchestra.run.smithi060.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-12T12:53:41.566 INFO:teuthology.orchestra.run.smithi060.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-12T12:53:41.566 INFO:teuthology.orchestra.run.smithi060.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-12T12:53:41.566 INFO:teuthology.orchestra.run.smithi060.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-12T12:53:41.566 INFO:teuthology.orchestra.run.smithi060.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-12T12:53:41.566 INFO:teuthology.orchestra.run.smithi060.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-12T12:53:41.566 INFO:teuthology.orchestra.run.smithi060.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-12T12:53:41.566 INFO:teuthology.orchestra.run.smithi060.stdout: libcephsqlite x86_64 2:19.1.1-314.g086c737f.el9 ceph 164 k 2024-09-12T12:53:41.566 INFO:teuthology.orchestra.run.smithi060.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-12T12:53:41.566 INFO:teuthology.orchestra.run.smithi060.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-12T12:53:41.567 INFO:teuthology.orchestra.run.smithi060.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-12T12:53:41.567 INFO:teuthology.orchestra.run.smithi060.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-12T12:53:41.567 INFO:teuthology.orchestra.run.smithi060.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-12T12:53:41.567 INFO:teuthology.orchestra.run.smithi060.stdout: libradosstriper1 x86_64 2:19.1.1-314.g086c737f.el9 ceph 506 k 2024-09-12T12:53:41.567 INFO:teuthology.orchestra.run.smithi060.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-12T12:53:41.567 INFO:teuthology.orchestra.run.smithi060.stdout: librgw2 x86_64 2:19.1.1-314.g086c737f.el9 ceph 5.3 M 2024-09-12T12:53:41.567 INFO:teuthology.orchestra.run.smithi060.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-12T12:53:41.567 INFO:teuthology.orchestra.run.smithi060.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-12T12:53:41.567 INFO:teuthology.orchestra.run.smithi060.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-12T12:53:41.567 INFO:teuthology.orchestra.run.smithi060.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-12T12:53:41.567 INFO:teuthology.orchestra.run.smithi060.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-12T12:53:41.567 INFO:teuthology.orchestra.run.smithi060.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-12T12:53:41.567 INFO:teuthology.orchestra.run.smithi060.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-12T12:53:41.567 INFO:teuthology.orchestra.run.smithi060.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-12T12:53:41.568 INFO:teuthology.orchestra.run.smithi060.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-12T12:53:41.568 INFO:teuthology.orchestra.run.smithi060.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-12T12:53:41.568 INFO:teuthology.orchestra.run.smithi060.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-12T12:53:41.568 INFO:teuthology.orchestra.run.smithi060.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-12T12:53:41.568 INFO:teuthology.orchestra.run.smithi060.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-12T12:53:41.568 INFO:teuthology.orchestra.run.smithi060.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-12T12:53:41.568 INFO:teuthology.orchestra.run.smithi060.stdout: python3-ceph-argparse x86_64 2:19.1.1-314.g086c737f.el9 ceph 45 k 2024-09-12T12:53:41.568 INFO:teuthology.orchestra.run.smithi060.stdout: python3-ceph-common x86_64 2:19.1.1-314.g086c737f.el9 ceph 136 k 2024-09-12T12:53:41.568 INFO:teuthology.orchestra.run.smithi060.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-12T12:53:41.568 INFO:teuthology.orchestra.run.smithi060.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-12T12:53:41.568 INFO:teuthology.orchestra.run.smithi060.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-12T12:53:41.568 INFO:teuthology.orchestra.run.smithi060.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-12T12:53:41.568 INFO:teuthology.orchestra.run.smithi060.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-12T12:53:41.569 INFO:teuthology.orchestra.run.smithi060.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-12T12:53:41.569 INFO:teuthology.orchestra.run.smithi060.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-12T12:53:41.569 INFO:teuthology.orchestra.run.smithi060.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-12T12:53:41.569 INFO:teuthology.orchestra.run.smithi060.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-12T12:53:41.569 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-12T12:53:41.569 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-12T12:53:41.569 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-12T12:53:41.569 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-12T12:53:41.569 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-12T12:53:41.569 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-12T12:53:41.569 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-12T12:53:41.569 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-12T12:53:41.569 INFO:teuthology.orchestra.run.smithi060.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-12T12:53:41.569 INFO:teuthology.orchestra.run.smithi060.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-12T12:53:41.570 INFO:teuthology.orchestra.run.smithi060.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-12T12:53:41.570 INFO:teuthology.orchestra.run.smithi060.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-12T12:53:41.570 INFO:teuthology.orchestra.run.smithi060.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-12T12:53:41.570 INFO:teuthology.orchestra.run.smithi060.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-12T12:53:41.570 INFO:teuthology.orchestra.run.smithi060.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-12T12:53:41.570 INFO:teuthology.orchestra.run.smithi060.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-12T12:53:41.570 INFO:teuthology.orchestra.run.smithi060.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-12T12:53:41.570 INFO:teuthology.orchestra.run.smithi060.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-12T12:53:41.570 INFO:teuthology.orchestra.run.smithi060.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-12T12:53:41.570 INFO:teuthology.orchestra.run.smithi060.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-12T12:53:41.570 INFO:teuthology.orchestra.run.smithi060.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-12T12:53:41.570 INFO:teuthology.orchestra.run.smithi060.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-12T12:53:41.571 INFO:teuthology.orchestra.run.smithi060.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-12T12:53:41.571 INFO:teuthology.orchestra.run.smithi060.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-12T12:53:41.571 INFO:teuthology.orchestra.run.smithi060.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-12T12:53:41.571 INFO:teuthology.orchestra.run.smithi060.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-12T12:53:41.571 INFO:teuthology.orchestra.run.smithi060.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-12T12:53:41.571 INFO:teuthology.orchestra.run.smithi060.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-12T12:53:41.571 INFO:teuthology.orchestra.run.smithi060.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-12T12:53:41.571 INFO:teuthology.orchestra.run.smithi060.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-12T12:53:41.571 INFO:teuthology.orchestra.run.smithi060.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-12T12:53:41.571 INFO:teuthology.orchestra.run.smithi060.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-12T12:53:41.571 INFO:teuthology.orchestra.run.smithi060.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-12T12:53:41.571 INFO:teuthology.orchestra.run.smithi060.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-12T12:53:41.571 INFO:teuthology.orchestra.run.smithi060.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-12T12:53:41.572 INFO:teuthology.orchestra.run.smithi060.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-12T12:53:41.572 INFO:teuthology.orchestra.run.smithi060.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-12T12:53:41.572 INFO:teuthology.orchestra.run.smithi060.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-12T12:53:41.572 INFO:teuthology.orchestra.run.smithi060.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-12T12:53:41.572 INFO:teuthology.orchestra.run.smithi060.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-12T12:53:41.572 INFO:teuthology.orchestra.run.smithi060.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-12T12:53:41.572 INFO:teuthology.orchestra.run.smithi060.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-12T12:53:41.572 INFO:teuthology.orchestra.run.smithi060.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-12T12:53:41.572 INFO:teuthology.orchestra.run.smithi060.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-12T12:53:41.572 INFO:teuthology.orchestra.run.smithi060.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-12T12:53:41.572 INFO:teuthology.orchestra.run.smithi060.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-12T12:53:41.572 INFO:teuthology.orchestra.run.smithi060.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-12T12:53:41.572 INFO:teuthology.orchestra.run.smithi060.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-12T12:53:41.573 INFO:teuthology.orchestra.run.smithi060.stdout:Installing weak dependencies: 2024-09-12T12:53:41.573 INFO:teuthology.orchestra.run.smithi060.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-12T12:53:41.573 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-12T12:53:41.573 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction Summary 2024-09-12T12:53:41.573 INFO:teuthology.orchestra.run.smithi060.stdout:====================================================================================== 2024-09-12T12:53:41.573 INFO:teuthology.orchestra.run.smithi060.stdout:Install 125 Packages 2024-09-12T12:53:41.573 INFO:teuthology.orchestra.run.smithi060.stdout:Upgrade 2 Packages 2024-09-12T12:53:41.573 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-12T12:53:41.574 INFO:teuthology.orchestra.run.smithi060.stdout:Total download size: 202 M 2024-09-12T12:53:41.574 INFO:teuthology.orchestra.run.smithi060.stdout:Downloading Packages: 2024-09-12T12:53:41.644 INFO:teuthology.orchestra.run.smithi130.stdout:(12/127): libcephfs2-19.1.1-314.g086c737f.el9.x 8.9 MB/s | 753 kB 00:00 2024-09-12T12:53:41.845 INFO:teuthology.orchestra.run.smithi130.stdout:(13/127): ceph-osd-19.1.1-314.g086c737f.el9.x86 13 MB/s | 17 MB 00:01 2024-09-12T12:53:41.870 INFO:teuthology.orchestra.run.smithi130.stdout:(14/127): libcephsqlite-19.1.1-314.g086c737f.el 727 kB/s | 164 kB 00:00 2024-09-12T12:53:41.895 INFO:teuthology.orchestra.run.smithi130.stdout:(15/127): librados-devel-19.1.1-314.g086c737f.e 2.5 MB/s | 127 kB 00:00 2024-09-12T12:53:41.928 INFO:teuthology.orchestra.run.smithi130.stdout:(16/127): libradosstriper1-19.1.1-314.g086c737f 8.5 MB/s | 506 kB 00:00 2024-09-12T12:53:42.004 INFO:teuthology.orchestra.run.smithi130.stdout:(17/127): python3-ceph-argparse-19.1.1-314.g086 603 kB/s | 45 kB 00:00 2024-09-12T12:53:42.079 INFO:teuthology.orchestra.run.smithi130.stdout:(18/127): python3-ceph-common-19.1.1-314.g086c7 1.8 MB/s | 136 kB 00:00 2024-09-12T12:53:42.154 INFO:teuthology.orchestra.run.smithi130.stdout:(19/127): python3-cephfs-19.1.1-314.g086c737f.e 2.1 MB/s | 162 kB 00:00 2024-09-12T12:53:42.167 INFO:teuthology.orchestra.run.smithi060.stdout:(1/127): ceph-19.1.1-314.g086c737f.el9.x86_64.r 33 kB/s | 6.5 kB 00:00 2024-09-12T12:53:42.288 INFO:teuthology.orchestra.run.smithi130.stdout:(20/127): librgw2-19.1.1-314.g086c737f.el9.x86_ 14 MB/s | 5.3 MB 00:00 2024-09-12T12:53:42.313 INFO:teuthology.orchestra.run.smithi130.stdout:(21/127): python3-rados-19.1.1-314.g086c737f.el 2.0 MB/s | 320 kB 00:00 2024-09-12T12:53:42.338 INFO:teuthology.orchestra.run.smithi130.stdout:(22/127): python3-rbd-19.1.1-314.g086c737f.el9. 5.9 MB/s | 301 kB 00:00 2024-09-12T12:53:42.363 INFO:teuthology.orchestra.run.smithi130.stdout:(23/127): python3-rgw-19.1.1-314.g086c737f.el9. 1.9 MB/s | 99 kB 00:00 2024-09-12T12:53:42.389 INFO:teuthology.orchestra.run.smithi130.stdout:(24/127): rbd-fuse-19.1.1-314.g086c737f.el9.x86 1.7 MB/s | 85 kB 00:00 2024-09-12T12:53:42.418 INFO:teuthology.orchestra.run.smithi060.stdout:(2/127): ceph-fuse-19.1.1-314.g086c737f.el9.x86 3.5 MB/s | 896 kB 00:00 2024-09-12T12:53:42.439 INFO:teuthology.orchestra.run.smithi130.stdout:(25/127): rbd-nbd-19.1.1-314.g086c737f.el9.x86_ 3.4 MB/s | 171 kB 00:00 2024-09-12T12:53:42.493 INFO:teuthology.orchestra.run.smithi060.stdout:(3/127): ceph-immutable-object-cache-19.1.1-314 1.9 MB/s | 145 kB 00:00 2024-09-12T12:53:42.539 INFO:teuthology.orchestra.run.smithi130.stdout:(26/127): ceph-grafana-dashboards-19.1.1-314.g0 318 kB/s | 32 kB 00:00 2024-09-12T12:53:42.598 INFO:teuthology.orchestra.run.smithi130.stdout:(27/127): rbd-mirror-19.1.1-314.g086c737f.el9.x 13 MB/s | 3.1 MB 00:00 2024-09-12T12:53:42.623 INFO:teuthology.orchestra.run.smithi130.stdout:(28/127): ceph-mgr-cephadm-19.1.1-314.g086c737f 1.7 MB/s | 146 kB 00:00 2024-09-12T12:53:42.932 INFO:teuthology.orchestra.run.smithi130.stdout:(29/127): ceph-mgr-dashboard-19.1.1-314.g086c73 11 MB/s | 3.6 MB 00:00 2024-09-12T12:53:43.024 INFO:teuthology.orchestra.run.smithi130.stdout:(30/127): ceph-mgr-modules-core-19.1.1-314.g086 2.7 MB/s | 249 kB 00:00 2024-09-12T12:53:43.124 INFO:teuthology.orchestra.run.smithi130.stdout:(31/127): ceph-mgr-rook-19.1.1-314.g086c737f.el 494 kB/s | 49 kB 00:00 2024-09-12T12:53:43.135 INFO:teuthology.orchestra.run.smithi060.stdout:(4/127): ceph-mds-19.1.1-314.g086c737f.el9.x86_ 3.7 MB/s | 2.4 MB 00:00 2024-09-12T12:53:43.207 INFO:teuthology.orchestra.run.smithi130.stdout:(32/127): ceph-prometheus-alerts-19.1.1-314.g08 198 kB/s | 16 kB 00:00 2024-09-12T12:53:43.300 INFO:teuthology.orchestra.run.smithi130.stdout:(33/127): ceph-mgr-diskprediction-local-19.1.1- 11 MB/s | 7.4 MB 00:00 2024-09-12T12:53:43.302 INFO:teuthology.orchestra.run.smithi060.stdout:(5/127): ceph-base-19.1.1-314.g086c737f.el9.x86 4.1 MB/s | 5.4 MB 00:01 2024-09-12T12:53:43.435 INFO:teuthology.orchestra.run.smithi060.stdout:(6/127): ceph-mgr-19.1.1-314.g086c737f.el9.x86_ 5.1 MB/s | 1.5 MB 00:00 2024-09-12T12:53:43.669 INFO:teuthology.orchestra.run.smithi060.stdout:(7/127): ceph-mon-19.1.1-314.g086c737f.el9.x86_ 13 MB/s | 4.7 MB 00:00 2024-09-12T12:53:43.758 INFO:teuthology.orchestra.run.smithi130.stdout:(34/127): ceph-test-19.1.1-314.g086c737f.el9.x8 20 MB/s | 49 MB 00:02 2024-09-12T12:53:43.783 INFO:teuthology.orchestra.run.smithi130.stdout:(35/127): ceph-volume-19.1.1-314.g086c737f.el9. 496 kB/s | 285 kB 00:00 2024-09-12T12:53:43.817 INFO:teuthology.orchestra.run.smithi130.stdout:(36/127): cephadm-19.1.1-314.g086c737f.el9.noar 1.4 MB/s | 765 kB 00:00 2024-09-12T12:53:43.926 INFO:teuthology.orchestra.run.smithi130.stdout:(37/127): fuse-2.9.9-16.el9.x86_64.rpm 479 kB/s | 80 kB 00:00 2024-09-12T12:53:43.984 INFO:teuthology.orchestra.run.smithi130.stdout:(38/127): libquadmath-11.5.0-2.el9.x86_64.rpm 1.1 MB/s | 191 kB 00:00 2024-09-12T12:53:44.035 INFO:teuthology.orchestra.run.smithi130.stdout:(39/127): libgfortran-11.5.0-2.el9.x86_64.rpm 3.1 MB/s | 801 kB 00:00 2024-09-12T12:53:44.060 INFO:teuthology.orchestra.run.smithi130.stdout:(40/127): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.9 MB/s | 253 kB 00:00 2024-09-12T12:53:44.102 INFO:teuthology.orchestra.run.smithi130.stdout:(41/127): python3-ply-3.11-14.el9.noarch.rpm 1.6 MB/s | 106 kB 00:00 2024-09-12T12:53:44.127 INFO:teuthology.orchestra.run.smithi130.stdout:(42/127): python3-pycparser-2.20-6.el9.noarch.r 2.0 MB/s | 135 kB 00:00 2024-09-12T12:53:44.152 INFO:teuthology.orchestra.run.smithi130.stdout:(43/127): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-09-12T12:53:44.194 INFO:teuthology.orchestra.run.smithi130.stdout:(44/127): python3-urllib3-1.26.5-6.el9.noarch.r 3.1 MB/s | 215 kB 00:00 2024-09-12T12:53:44.203 INFO:teuthology.orchestra.run.smithi060.stdout:(8/127): ceph-common-19.1.1-314.g086c737f.el9.x 9.5 MB/s | 21 MB 00:02 2024-09-12T12:53:44.236 INFO:teuthology.orchestra.run.smithi130.stdout:(45/127): python3-cryptography-36.0.1-4.el9.x86 4.9 MB/s | 1.2 MB 00:00 2024-09-12T12:53:44.253 INFO:teuthology.orchestra.run.smithi060.stdout:(9/127): ceph-selinux-19.1.1-314.g086c737f.el9. 507 kB/s | 25 kB 00:00 2024-09-12T12:53:44.286 INFO:teuthology.orchestra.run.smithi130.stdout:(46/127): boost-program-options-1.75.0-8.el9.x8 802 kB/s | 107 kB 00:00 2024-09-12T12:53:44.311 INFO:teuthology.orchestra.run.smithi130.stdout:(47/127): flexiblas-3.0.4-8.el9.x86_64.rpm 280 kB/s | 33 kB 00:00 2024-09-12T12:53:44.336 INFO:teuthology.orchestra.run.smithi130.stdout:(48/127): flexiblas-openblas-openmp-3.0.4-8.el9 365 kB/s | 18 kB 00:00 2024-09-12T12:53:44.378 INFO:teuthology.orchestra.run.smithi130.stdout:(49/127): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.1 MB/s | 45 kB 00:00 2024-09-12T12:53:44.404 INFO:teuthology.orchestra.run.smithi130.stdout:(50/127): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.7 MB/s | 160 kB 00:00 2024-09-12T12:53:44.420 INFO:teuthology.orchestra.run.smithi060.stdout:(10/127): ceph-radosgw-19.1.1-314.g086c737f.el9 14 MB/s | 11 MB 00:00 2024-09-12T12:53:44.470 INFO:teuthology.orchestra.run.smithi060.stdout:(11/127): libcephfs-devel-19.1.1-314.g086c737f. 641 kB/s | 32 kB 00:00 2024-09-12T12:53:44.471 INFO:teuthology.orchestra.run.smithi130.stdout:(51/127): flexiblas-netlib-3.0.4-8.el9.x86_64.r 13 MB/s | 3.0 MB 00:00 2024-09-12T12:53:44.521 INFO:teuthology.orchestra.run.smithi130.stdout:(52/127): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.4 MB/s | 292 kB 00:00 2024-09-12T12:53:44.554 INFO:teuthology.orchestra.run.smithi130.stdout:(53/127): librdkafka-1.6.1-102.el9.x86_64.rpm 3.7 MB/s | 662 kB 00:00 2024-09-12T12:53:44.580 INFO:teuthology.orchestra.run.smithi130.stdout:(54/127): openblas-0.3.26-2.el9.x86_64.rpm 661 kB/s | 38 kB 00:00 2024-09-12T12:53:44.605 INFO:teuthology.orchestra.run.smithi130.stdout:(55/127): lua-5.4.4-4.el9.x86_64.rpm 1.4 MB/s | 188 kB 00:00 2024-09-12T12:53:44.654 INFO:teuthology.orchestra.run.smithi060.stdout:(12/127): ceph-osd-19.1.1-314.g086c737f.el9.x86 14 MB/s | 17 MB 00:01 2024-09-12T12:53:44.655 INFO:teuthology.orchestra.run.smithi130.stdout:(56/127): protobuf-3.14.0-13.el9.x86_64.rpm 13 MB/s | 1.0 MB 00:00 2024-09-12T12:53:44.729 INFO:teuthology.orchestra.run.smithi060.stdout:(13/127): libcephfs2-19.1.1-314.g086c737f.el9.x 2.8 MB/s | 753 kB 00:00 2024-09-12T12:53:44.739 INFO:teuthology.orchestra.run.smithi130.stdout:(57/127): openblas-openmp-0.3.26-2.el9.x86_64.r 27 MB/s | 4.9 MB 00:00 2024-09-12T12:53:44.754 INFO:teuthology.orchestra.run.smithi060.stdout:(14/127): libcephsqlite-19.1.1-314.g086c737f.el 1.6 MB/s | 164 kB 00:00 2024-09-12T12:53:44.779 INFO:teuthology.orchestra.run.smithi060.stdout:(15/127): librados-devel-19.1.1-314.g086c737f.e 2.5 MB/s | 127 kB 00:00 2024-09-12T12:53:44.781 INFO:teuthology.orchestra.run.smithi130.stdout:(58/127): python3-devel-3.9.19-8.el9.x86_64.rpm 1.9 MB/s | 245 kB 00:00 2024-09-12T12:53:44.813 INFO:teuthology.orchestra.run.smithi060.stdout:(16/127): libradosstriper1-19.1.1-314.g086c737f 8.5 MB/s | 506 kB 00:00 2024-09-12T12:53:44.814 INFO:teuthology.orchestra.run.smithi130.stdout:(59/127): python3-jinja2-2.11.3-6.el9.noarch.rp 3.2 MB/s | 248 kB 00:00 2024-09-12T12:53:44.848 INFO:teuthology.orchestra.run.smithi130.stdout:(60/127): python3-jsonpatch-1.21-16.el9.noarch. 382 kB/s | 26 kB 00:00 2024-09-12T12:53:44.863 INFO:teuthology.orchestra.run.smithi060.stdout:(17/127): python3-ceph-argparse-19.1.1-314.g086 906 kB/s | 45 kB 00:00 2024-09-12T12:53:44.873 INFO:teuthology.orchestra.run.smithi130.stdout:(61/127): python3-jsonpointer-2.0-4.el9.noarch. 327 kB/s | 19 kB 00:00 2024-09-12T12:53:44.890 INFO:teuthology.orchestra.run.smithi130.stdout:(62/127): python3-mako-1.1.4-6.el9.noarch.rpm 4.1 MB/s | 172 kB 00:00 2024-09-12T12:53:44.916 INFO:teuthology.orchestra.run.smithi130.stdout:(63/127): python3-markupsafe-1.1.1-12.el9.x86_6 839 kB/s | 35 kB 00:00 2024-09-12T12:53:44.930 INFO:teuthology.orchestra.run.smithi060.stdout:(18/127): python3-ceph-common-19.1.1-314.g086c7 2.0 MB/s | 136 kB 00:00 2024-09-12T12:53:44.974 INFO:teuthology.orchestra.run.smithi130.stdout:(64/127): python3-numpy-f2py-1.20.1-5.el9.x86_6 4.2 MB/s | 250 kB 00:00 2024-09-12T12:53:45.005 INFO:teuthology.orchestra.run.smithi060.stdout:(19/127): python3-cephfs-19.1.1-314.g086c737f.e 2.1 MB/s | 162 kB 00:00 2024-09-12T12:53:45.066 INFO:teuthology.orchestra.run.smithi130.stdout:(65/127): python3-babel-2.9.1-2.el9.noarch.rpm 13 MB/s | 6.0 MB 00:00 2024-09-12T12:53:45.091 INFO:teuthology.orchestra.run.smithi130.stdout:(66/127): python3-oauthlib-3.1.1-5.el9.noarch.r 1.9 MB/s | 222 kB 00:00 2024-09-12T12:53:45.105 INFO:teuthology.orchestra.run.smithi060.stdout:(20/127): librgw2-19.1.1-314.g086c737f.el9.x86_ 16 MB/s | 5.3 MB 00:00 2024-09-12T12:53:45.116 INFO:teuthology.orchestra.run.smithi130.stdout:(67/127): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-09-12T12:53:45.131 INFO:teuthology.orchestra.run.smithi060.stdout:(21/127): python3-rados-19.1.1-314.g086c737f.el 2.5 MB/s | 320 kB 00:00 2024-09-12T12:53:45.142 INFO:teuthology.orchestra.run.smithi130.stdout:(68/127): python3-prettytable-0.7.2-27.el9.noar 848 kB/s | 42 kB 00:00 2024-09-12T12:53:45.164 INFO:teuthology.orchestra.run.smithi060.stdout:(22/127): python3-rbd-19.1.1-314.g086c737f.el9. 5.0 MB/s | 301 kB 00:00 2024-09-12T12:53:45.167 INFO:teuthology.orchestra.run.smithi130.stdout:(69/127): python3-protobuf-3.14.0-13.el9.noarch 5.3 MB/s | 269 kB 00:00 2024-09-12T12:53:45.190 INFO:teuthology.orchestra.run.smithi060.stdout:(23/127): python3-rgw-19.1.1-314.g086c737f.el9. 1.7 MB/s | 99 kB 00:00 2024-09-12T12:53:45.192 INFO:teuthology.orchestra.run.smithi130.stdout:(70/127): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 159 kB 00:00 2024-09-12T12:53:45.215 INFO:teuthology.orchestra.run.smithi060.stdout:(24/127): rbd-fuse-19.1.1-314.g086c737f.el9.x86 1.7 MB/s | 85 kB 00:00 2024-09-12T12:53:45.217 INFO:teuthology.orchestra.run.smithi130.stdout:(71/127): python3-pyasn1-modules-0.4.8-6.el9.no 5.4 MB/s | 279 kB 00:00 2024-09-12T12:53:45.234 INFO:teuthology.orchestra.run.smithi130.stdout:(72/127): python3-pytz-2021.1-5.el9.noarch.rpm 1.2 MB/s | 51 kB 00:00 2024-09-12T12:53:45.251 INFO:teuthology.orchestra.run.smithi130.stdout:(73/127): python3-requests-oauthlib-1.3.0-12.el 1.6 MB/s | 54 kB 00:00 2024-09-12T12:53:45.265 INFO:teuthology.orchestra.run.smithi060.stdout:(25/127): rbd-nbd-19.1.1-314.g086c737f.el9.x86_ 3.4 MB/s | 171 kB 00:00 2024-09-12T12:53:45.343 INFO:teuthology.orchestra.run.smithi130.stdout:(74/127): python3-numpy-1.20.1-5.el9.x86_64.rpm 11 MB/s | 5.1 MB 00:00 2024-09-12T12:53:45.374 INFO:teuthology.orchestra.run.smithi060.stdout:(26/127): rbd-mirror-19.1.1-314.g086c737f.el9.x 17 MB/s | 3.1 MB 00:00 2024-09-12T12:53:45.393 INFO:teuthology.orchestra.run.smithi130.stdout:(75/127): python3-toml-0.10.2-6.el9.noarch.rpm 294 kB/s | 42 kB 00:00 2024-09-12T12:53:45.399 INFO:teuthology.orchestra.run.smithi060.stdout:(27/127): ceph-grafana-dashboards-19.1.1-314.g0 238 kB/s | 32 kB 00:00 2024-09-12T12:53:45.419 INFO:teuthology.orchestra.run.smithi130.stdout:(76/127): qatlib-24.02.0-1.el9.x86_64.rpm 2.9 MB/s | 221 kB 00:00 2024-09-12T12:53:45.424 INFO:teuthology.orchestra.run.smithi060.stdout:(28/127): ceph-mgr-cephadm-19.1.1-314.g086c737f 2.9 MB/s | 146 kB 00:00 2024-09-12T12:53:45.436 INFO:teuthology.orchestra.run.smithi130.stdout:(77/127): qatlib-service-24.02.0-1.el9.x86_64.r 860 kB/s | 36 kB 00:00 2024-09-12T12:53:45.453 INFO:teuthology.orchestra.run.smithi130.stdout:(78/127): qatzip-libs-1.2.0-1.el9.x86_64.rpm 1.4 MB/s | 47 kB 00:00 2024-09-12T12:53:45.486 INFO:teuthology.orchestra.run.smithi130.stdout:(79/127): socat-1.7.4.1-6.el9.x86_64.rpm 5.9 MB/s | 304 kB 00:00 2024-09-12T12:53:45.503 INFO:teuthology.orchestra.run.smithi130.stdout:(80/127): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-09-12T12:53:45.650 INFO:teuthology.orchestra.run.smithi060.stdout:(29/127): ceph-mgr-dashboard-19.1.1-314.g086c73 14 MB/s | 3.6 MB 00:00 2024-09-12T12:53:45.733 INFO:teuthology.orchestra.run.smithi060.stdout:(30/127): ceph-mgr-modules-core-19.1.1-314.g086 2.9 MB/s | 249 kB 00:00 2024-09-12T12:53:45.770 INFO:teuthology.orchestra.run.smithi130.stdout:(81/127): python3-scipy-1.6.2-8.el9.x86_64.rpm 30 MB/s | 16 MB 00:00 2024-09-12T12:53:45.812 INFO:teuthology.orchestra.run.smithi130.stdout:(82/127): lua-devel-5.4.4-4.el9.x86_64.rpm 68 kB/s | 22 kB 00:00 2024-09-12T12:53:45.871 INFO:teuthology.orchestra.run.smithi130.stdout:(83/127): abseil-cpp-20211102.0-3.el9.x86_64.rp 5.4 MB/s | 548 kB 00:00 2024-09-12T12:53:45.875 INFO:teuthology.orchestra.run.smithi060.stdout:(31/127): ceph-mgr-diskprediction-local-19.1.1- 16 MB/s | 7.4 MB 00:00 2024-09-12T12:53:45.887 INFO:teuthology.orchestra.run.smithi130.stdout:(84/127): grpc-data-1.46.7-10.el9.noarch.rpm 1.2 MB/s | 19 kB 00:00 2024-09-12T12:53:45.901 INFO:teuthology.orchestra.run.smithi060.stdout:(32/127): ceph-mgr-rook-19.1.1-314.g086c737f.el 296 kB/s | 49 kB 00:00 2024-09-12T12:53:45.913 INFO:teuthology.orchestra.run.smithi130.stdout:(85/127): gperftools-libs-2.9.1-3.el9.x86_64.rp 3.0 MB/s | 308 kB 00:00 2024-09-12T12:53:45.926 INFO:teuthology.orchestra.run.smithi060.stdout:(33/127): ceph-prometheus-alerts-19.1.1-314.g08 329 kB/s | 16 kB 00:00 2024-09-12T12:53:45.946 INFO:teuthology.orchestra.run.smithi130.stdout:(86/127): protobuf-compiler-3.14.0-13.el9.x86_6 1.9 MB/s | 863 kB 00:00 2024-09-12T12:53:45.963 INFO:teuthology.orchestra.run.smithi130.stdout:(87/127): libarrow-doc-9.0.0-11.el9.noarch.rpm 521 kB/s | 26 kB 00:00 2024-09-12T12:53:45.976 INFO:teuthology.orchestra.run.smithi060.stdout:(34/127): ceph-volume-19.1.1-314.g086c737f.el9. 3.7 MB/s | 285 kB 00:00 2024-09-12T12:53:45.997 INFO:teuthology.orchestra.run.smithi130.stdout:(88/127): libunwind-1.6.2-1.el9.x86_64.rpm 2.8 MB/s | 67 kB 00:00 2024-09-12T12:53:46.010 INFO:teuthology.orchestra.run.smithi060.stdout:(35/127): cephadm-19.1.1-314.g086c737f.el9.noar 9.0 MB/s | 765 kB 00:00 2024-09-12T12:53:46.030 INFO:teuthology.orchestra.run.smithi130.stdout:(89/127): luarocks-3.9.2-5.el9.noarch.rpm 4.7 MB/s | 151 kB 00:00 2024-09-12T12:53:46.064 INFO:teuthology.orchestra.run.smithi130.stdout:(90/127): liboath-2.6.7-2.el9.x86_64.rpm 414 kB/s | 49 kB 00:00 2024-09-12T12:53:46.098 INFO:teuthology.orchestra.run.smithi130.stdout:(91/127): pv-1.6.20-1.el9.x86_64.rpm 1.9 MB/s | 65 kB 00:00 2024-09-12T12:53:46.102 INFO:teuthology.orchestra.run.smithi060.stdout:(36/127): fuse-2.9.9-16.el9.x86_64.rpm 638 kB/s | 80 kB 00:00 2024-09-12T12:53:46.148 INFO:teuthology.orchestra.run.smithi130.stdout:(92/127): parquet-libs-9.0.0-11.el9.x86_64.rpm 7.0 MB/s | 839 kB 00:00 2024-09-12T12:53:46.165 INFO:teuthology.orchestra.run.smithi130.stdout:(93/127): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 2.5 MB/s | 43 kB 00:00 2024-09-12T12:53:46.190 INFO:teuthology.orchestra.run.smithi130.stdout:(94/127): python3-asyncssh-2.13.2-5.el9.noarch. 5.8 MB/s | 548 kB 00:00 2024-09-12T12:53:46.249 INFO:teuthology.orchestra.run.smithi130.stdout:(95/127): python3-cachetools-4.2.4-1.el9.noarch 386 kB/s | 32 kB 00:00 2024-09-12T12:53:46.274 INFO:teuthology.orchestra.run.smithi130.stdout:(96/127): python3-certifi-2023.05.07-4.el9.noar 169 kB/s | 14 kB 00:00 2024-09-12T12:53:46.299 INFO:teuthology.orchestra.run.smithi130.stdout:(97/127): python3-cheroot-8.6.0-4.el9.noarch.rp 3.3 MB/s | 172 kB 00:00 2024-09-12T12:53:46.333 INFO:teuthology.orchestra.run.smithi130.stdout:(98/127): python3-cherrypy-18.6.1-2.el9.noarch. 6.0 MB/s | 358 kB 00:00 2024-09-12T12:53:46.358 INFO:teuthology.orchestra.run.smithi130.stdout:(99/127): python3-google-auth-2.34.0-1.el9.noar 3.7 MB/s | 220 kB 00:00 2024-09-12T12:53:46.384 INFO:teuthology.orchestra.run.smithi130.stdout:(100/127): python3-grpcio-tools-1.46.7-10.el9.x 5.6 MB/s | 144 kB 00:00 2024-09-12T12:53:46.409 INFO:teuthology.orchestra.run.smithi130.stdout:(101/127): python3-jaraco-8.2.1-3.el9.noarch.rp 434 kB/s | 11 kB 00:00 2024-09-12T12:53:46.434 INFO:teuthology.orchestra.run.smithi130.stdout:(102/127): python3-jaraco-classes-3.2.1-5.el9.n 715 kB/s | 18 kB 00:00 2024-09-12T12:53:46.459 INFO:teuthology.orchestra.run.smithi130.stdout:(103/127): python3-jaraco-collections-3.0.0-8.e 926 kB/s | 23 kB 00:00 2024-09-12T12:53:46.485 INFO:teuthology.orchestra.run.smithi130.stdout:(104/127): python3-jaraco-functools-3.5.0-2.el9 784 kB/s | 19 kB 00:00 2024-09-12T12:53:46.510 INFO:teuthology.orchestra.run.smithi130.stdout:(105/127): python3-jaraco-text-3.2.0-6.el9.noar 792 kB/s | 20 kB 00:00 2024-09-12T12:53:46.569 INFO:teuthology.orchestra.run.smithi130.stdout:(106/127): python3-kubernetes-26.1.0-2.el9.noar 18 MB/s | 1.0 MB 00:00 2024-09-12T12:53:46.594 INFO:teuthology.orchestra.run.smithi130.stdout:(107/127): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-12T12:53:46.611 INFO:teuthology.orchestra.run.smithi060.stdout:(37/127): ceph-test-19.1.1-314.g086c737f.el9.x8 21 MB/s | 49 MB 00:02 2024-09-12T12:53:46.653 INFO:teuthology.orchestra.run.smithi060.stdout:(38/127): libgfortran-11.5.0-2.el9.x86_64.rpm 1.2 MB/s | 801 kB 00:00 2024-09-12T12:53:46.678 INFO:teuthology.orchestra.run.smithi130.stdout:(108/127): python3-grpcio-1.46.7-10.el9.x86_64. 5.9 MB/s | 2.0 MB 00:00 2024-09-12T12:53:46.678 INFO:teuthology.orchestra.run.smithi060.stdout:(39/127): libquadmath-11.5.0-2.el9.x86_64.rpm 332 kB/s | 191 kB 00:00 2024-09-12T12:53:46.703 INFO:teuthology.orchestra.run.smithi130.stdout:(109/127): python3-more-itertools-8.12.0-2.el9. 724 kB/s | 79 kB 00:00 2024-09-12T12:53:46.728 INFO:teuthology.orchestra.run.smithi130.stdout:(110/127): python3-natsort-7.1.1-5.el9.noarch.r 1.1 MB/s | 58 kB 00:00 2024-09-12T12:53:46.753 INFO:teuthology.orchestra.run.smithi060.stdout:(40/127): python3-ply-3.11-14.el9.noarch.rpm 1.4 MB/s | 106 kB 00:00 2024-09-12T12:53:46.753 INFO:teuthology.orchestra.run.smithi130.stdout:(111/127): python3-portend-3.1.0-2.el9.noarch.r 660 kB/s | 16 kB 00:00 2024-09-12T12:53:46.778 INFO:teuthology.orchestra.run.smithi060.stdout:(41/127): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.5 MB/s | 253 kB 00:00 2024-09-12T12:53:46.779 INFO:teuthology.orchestra.run.smithi130.stdout:(112/127): python3-pyOpenSSL-21.0.0-1.el9.noarc 3.5 MB/s | 90 kB 00:00 2024-09-12T12:53:46.804 INFO:teuthology.orchestra.run.smithi130.stdout:(113/127): python3-repoze-lru-0.7-16.el9.noarch 1.2 MB/s | 31 kB 00:00 2024-09-12T12:53:46.820 INFO:teuthology.orchestra.run.smithi060.stdout:(42/127): python3-cryptography-36.0.1-4.el9.x86 7.3 MB/s | 1.2 MB 00:00 2024-09-12T12:53:46.829 INFO:teuthology.orchestra.run.smithi130.stdout:(114/127): python3-pecan-1.4.2-3.el9.noarch.rpm 2.1 MB/s | 272 kB 00:00 2024-09-12T12:53:46.846 INFO:teuthology.orchestra.run.smithi060.stdout:(43/127): python3-pycparser-2.20-6.el9.noarch.r 1.4 MB/s | 135 kB 00:00 2024-09-12T12:53:46.871 INFO:teuthology.orchestra.run.smithi060.stdout:(44/127): python3-requests-2.25.1-8.el9.noarch. 1.3 MB/s | 125 kB 00:00 2024-09-12T12:53:46.894 INFO:teuthology.orchestra.run.smithi130.stdout:(115/127): libarrow-9.0.0-11.el9.x86_64.rpm 4.4 MB/s | 4.4 MB 00:01 2024-09-12T12:53:46.919 INFO:teuthology.orchestra.run.smithi130.stdout:(116/127): python3-rsa-4.9-2.el9.noarch.rpm 659 kB/s | 59 kB 00:00 2024-09-12T12:53:46.938 INFO:teuthology.orchestra.run.smithi060.stdout:(45/127): python3-urllib3-1.26.5-6.el9.noarch.r 1.8 MB/s | 215 kB 00:00 2024-09-12T12:53:46.945 INFO:teuthology.orchestra.run.smithi130.stdout:(117/127): python3-tempora-5.0.0-2.el9.noarch.r 714 kB/s | 36 kB 00:00 2024-09-12T12:53:46.963 INFO:teuthology.orchestra.run.smithi060.stdout:(46/127): flexiblas-3.0.4-8.el9.x86_64.rpm 356 kB/s | 33 kB 00:00 2024-09-12T12:53:46.970 INFO:teuthology.orchestra.run.smithi130.stdout:(118/127): python3-typing-extensions-4.12.2-2.e 1.4 MB/s | 74 kB 00:00 2024-09-12T12:53:46.988 INFO:teuthology.orchestra.run.smithi060.stdout:(47/127): boost-program-options-1.75.0-8.el9.x8 752 kB/s | 107 kB 00:00 2024-09-12T12:53:46.995 INFO:teuthology.orchestra.run.smithi130.stdout:(119/127): python3-routes-2.5.1-5.el9.noarch.rp 988 kB/s | 188 kB 00:00 2024-09-12T12:53:47.017 INFO:teuthology.orchestra.run.smithi060.stdout:(48/127): flexiblas-openblas-openmp-3.0.4-8.el9 365 kB/s | 18 kB 00:00 2024-09-12T12:53:47.020 INFO:teuthology.orchestra.run.smithi130.stdout:(120/127): python3-websocket-client-1.2.3-2.el9 1.7 MB/s | 90 kB 00:00 2024-09-12T12:53:47.047 INFO:teuthology.orchestra.run.smithi060.stdout:(49/127): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.7 MB/s | 160 kB 00:00 2024-09-12T12:53:47.068 INFO:teuthology.orchestra.run.smithi130.stdout:(121/127): python3-webob-1.8.8-2.el9.noarch.rpm 1.8 MB/s | 230 kB 00:00 2024-09-12T12:53:47.072 INFO:teuthology.orchestra.run.smithi060.stdout:(50/127): librabbitmq-0.11.0-7.el9.x86_64.rpm 773 kB/s | 45 kB 00:00 2024-09-12T12:53:47.096 INFO:teuthology.orchestra.run.smithi130.stdout:(122/127): python3-zc-lockfile-2.0-10.el9.noarc 265 kB/s | 20 kB 00:00 2024-09-12T12:53:47.121 INFO:teuthology.orchestra.run.smithi130.stdout:(123/127): re2-20211101-20.el9.x86_64.rpm 3.5 MB/s | 191 kB 00:00 2024-09-12T12:53:47.146 INFO:teuthology.orchestra.run.smithi130.stdout:(124/127): python3-werkzeug-2.0.3-3.el9.1.noarc 2.8 MB/s | 427 kB 00:00 2024-09-12T12:53:47.147 INFO:teuthology.orchestra.run.smithi060.stdout:(51/127): librdkafka-1.6.1-102.el9.x86_64.rpm 6.5 MB/s | 662 kB 00:00 2024-09-12T12:53:47.173 INFO:teuthology.orchestra.run.smithi060.stdout:(52/127): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.9 MB/s | 292 kB 00:00 2024-09-12T12:53:47.198 INFO:teuthology.orchestra.run.smithi060.stdout:(53/127): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 188 kB 00:00 2024-09-12T12:53:47.205 INFO:teuthology.orchestra.run.smithi130.stdout:(125/127): thrift-0.15.0-2.el9.x86_64.rpm 15 MB/s | 1.6 MB 00:00 2024-09-12T12:53:47.223 INFO:teuthology.orchestra.run.smithi060.stdout:(54/127): openblas-0.3.26-2.el9.x86_64.rpm 776 kB/s | 38 kB 00:00 2024-09-12T12:53:47.380 INFO:teuthology.orchestra.run.smithi130.stdout:(126/127): librados2-19.1.1-314.g086c737f.el9.x 13 MB/s | 3.4 MB 00:00 2024-09-12T12:53:47.449 INFO:teuthology.orchestra.run.smithi060.stdout:(55/127): protobuf-3.14.0-13.el9.x86_64.rpm 4.5 MB/s | 1.0 MB 00:00 2024-09-12T12:53:47.547 INFO:teuthology.orchestra.run.smithi130.stdout:(127/127): librbd1-19.1.1-314.g086c737f.el9.x86 7.9 MB/s | 3.2 MB 00:00 2024-09-12T12:53:47.552 INFO:teuthology.orchestra.run.smithi130.stdout:-------------------------------------------------------------------------------- 2024-09-12T12:53:47.553 INFO:teuthology.orchestra.run.smithi130.stdout:Total 25 MB/s | 202 MB 00:08 2024-09-12T12:53:47.657 INFO:teuthology.orchestra.run.smithi060.stdout:(56/127): flexiblas-netlib-3.0.4-8.el9.x86_64.r 4.2 MB/s | 3.0 MB 00:00 2024-09-12T12:53:47.741 INFO:teuthology.orchestra.run.smithi060.stdout:(57/127): python3-devel-3.9.19-8.el9.x86_64.rpm 2.9 MB/s | 245 kB 00:00 2024-09-12T12:53:47.833 INFO:teuthology.orchestra.run.smithi060.stdout:(58/127): python3-jinja2-2.11.3-6.el9.noarch.rp 2.7 MB/s | 248 kB 00:00 2024-09-12T12:53:47.933 INFO:teuthology.orchestra.run.smithi060.stdout:(59/127): openblas-openmp-0.3.26-2.el9.x86_64.r 6.7 MB/s | 4.9 MB 00:00 2024-09-12T12:53:47.958 INFO:teuthology.orchestra.run.smithi060.stdout:(60/127): python3-jsonpatch-1.21-16.el9.noarch. 205 kB/s | 26 kB 00:00 2024-09-12T12:53:47.984 INFO:teuthology.orchestra.run.smithi060.stdout:(61/127): python3-jsonpointer-2.0-4.el9.noarch. 381 kB/s | 19 kB 00:00 2024-09-12T12:53:48.009 INFO:teuthology.orchestra.run.smithi060.stdout:(62/127): python3-mako-1.1.4-6.el9.noarch.rpm 3.3 MB/s | 172 kB 00:00 2024-09-12T12:53:48.034 INFO:teuthology.orchestra.run.smithi060.stdout:(63/127): python3-markupsafe-1.1.1-12.el9.x86_6 696 kB/s | 35 kB 00:00 2024-09-12T12:53:48.135 INFO:teuthology.orchestra.run.smithi060.stdout:(64/127): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.5 MB/s | 250 kB 00:00 2024-09-12T12:53:48.226 INFO:teuthology.orchestra.run.smithi060.stdout:(65/127): python3-oauthlib-3.1.1-5.el9.noarch.r 2.4 MB/s | 222 kB 00:00 2024-09-12T12:53:48.285 INFO:teuthology.orchestra.run.smithi060.stdout:(66/127): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 77 kB 00:00 2024-09-12T12:53:48.327 INFO:teuthology.orchestra.run.smithi060.stdout:(67/127): python3-prettytable-0.7.2-27.el9.noar 1.0 MB/s | 42 kB 00:00 2024-09-12T12:53:48.427 INFO:teuthology.orchestra.run.smithi060.stdout:(68/127): python3-protobuf-3.14.0-13.el9.noarch 2.6 MB/s | 269 kB 00:00 2024-09-12T12:53:48.503 INFO:teuthology.orchestra.run.smithi060.stdout:(69/127): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.1 MB/s | 159 kB 00:00 2024-09-12T12:53:48.628 INFO:teuthology.orchestra.run.smithi060.stdout:(70/127): python3-pyasn1-modules-0.4.8-6.el9.no 2.2 MB/s | 279 kB 00:00 2024-09-12T12:53:48.670 INFO:teuthology.orchestra.run.smithi060.stdout:(71/127): python3-pytz-2021.1-5.el9.noarch.rpm 1.2 MB/s | 51 kB 00:00 2024-09-12T12:53:48.737 INFO:teuthology.orchestra.run.smithi060.stdout:(72/127): python3-numpy-1.20.1-5.el9.x86_64.rpm 7.0 MB/s | 5.1 MB 00:00 2024-09-12T12:53:48.762 INFO:teuthology.orchestra.run.smithi060.stdout:(73/127): python3-requests-oauthlib-1.3.0-12.el 585 kB/s | 54 kB 00:00 2024-09-12T12:53:48.804 INFO:teuthology.orchestra.run.smithi060.stdout:(74/127): python3-toml-0.10.2-6.el9.noarch.rpm 1.0 MB/s | 42 kB 00:00 2024-09-12T12:53:48.896 INFO:teuthology.orchestra.run.smithi060.stdout:(75/127): qatlib-24.02.0-1.el9.x86_64.rpm 2.4 MB/s | 221 kB 00:00 2024-09-12T12:53:48.901 INFO:teuthology.orchestra.run.smithi130.stdout:Running transaction check 2024-09-12T12:53:48.929 INFO:teuthology.orchestra.run.smithi060.stdout:(76/127): qatlib-service-24.02.0-1.el9.x86_64.r 1.1 MB/s | 36 kB 00:00 2024-09-12T12:53:48.963 INFO:teuthology.orchestra.run.smithi060.stdout:(77/127): qatzip-libs-1.2.0-1.el9.x86_64.rpm 1.4 MB/s | 47 kB 00:00 2024-09-12T12:53:49.024 INFO:teuthology.orchestra.run.smithi130.stdout:Transaction check succeeded. 2024-09-12T12:53:49.025 INFO:teuthology.orchestra.run.smithi130.stdout:Running transaction test 2024-09-12T12:53:49.088 INFO:teuthology.orchestra.run.smithi060.stdout:(78/127): socat-1.7.4.1-6.el9.x86_64.rpm 2.4 MB/s | 304 kB 00:00 2024-09-12T12:53:49.139 INFO:teuthology.orchestra.run.smithi060.stdout:(79/127): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.3 MB/s | 64 kB 00:00 2024-09-12T12:53:49.172 INFO:teuthology.orchestra.run.smithi060.stdout:(80/127): lua-devel-5.4.4-4.el9.x86_64.rpm 674 kB/s | 22 kB 00:00 2024-09-12T12:53:49.289 INFO:teuthology.orchestra.run.smithi060.stdout:(81/127): python3-babel-2.9.1-2.el9.noarch.rpm 3.2 MB/s | 6.0 MB 00:01 2024-09-12T12:53:49.348 INFO:teuthology.orchestra.run.smithi060.stdout:(82/127): abseil-cpp-20211102.0-3.el9.x86_64.rp 9.2 MB/s | 548 kB 00:00 2024-09-12T12:53:49.381 INFO:teuthology.orchestra.run.smithi060.stdout:(83/127): gperftools-libs-2.9.1-3.el9.x86_64.rp 9.0 MB/s | 308 kB 00:00 2024-09-12T12:53:49.406 INFO:teuthology.orchestra.run.smithi060.stdout:(84/127): grpc-data-1.46.7-10.el9.noarch.rpm 786 kB/s | 19 kB 00:00 2024-09-12T12:53:49.532 INFO:teuthology.orchestra.run.smithi060.stdout:(85/127): libarrow-9.0.0-11.el9.x86_64.rpm 35 MB/s | 4.4 MB 00:00 2024-09-12T12:53:49.565 INFO:teuthology.orchestra.run.smithi060.stdout:(86/127): libarrow-doc-9.0.0-11.el9.noarch.rpm 786 kB/s | 26 kB 00:00 2024-09-12T12:53:49.591 INFO:teuthology.orchestra.run.smithi060.stdout:(87/127): liboath-2.6.7-2.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2024-09-12T12:53:49.616 INFO:teuthology.orchestra.run.smithi060.stdout:(88/127): libunwind-1.6.2-1.el9.x86_64.rpm 2.6 MB/s | 67 kB 00:00 2024-09-12T12:53:49.658 INFO:teuthology.orchestra.run.smithi060.stdout:(89/127): protobuf-compiler-3.14.0-13.el9.x86_6 1.7 MB/s | 863 kB 00:00 2024-09-12T12:53:49.700 INFO:teuthology.orchestra.run.smithi060.stdout:(90/127): luarocks-3.9.2-5.el9.noarch.rpm 1.8 MB/s | 151 kB 00:00 2024-09-12T12:53:49.734 INFO:teuthology.orchestra.run.smithi060.stdout:(91/127): pv-1.6.20-1.el9.x86_64.rpm 2.0 MB/s | 65 kB 00:00 2024-09-12T12:53:49.768 INFO:teuthology.orchestra.run.smithi060.stdout:(92/127): python3-asyncssh-2.13.2-5.el9.noarch. 16 MB/s | 548 kB 00:00 2024-09-12T12:53:49.793 INFO:teuthology.orchestra.run.smithi060.stdout:(93/127): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 1.7 MB/s | 43 kB 00:00 2024-09-12T12:53:49.818 INFO:teuthology.orchestra.run.smithi060.stdout:(94/127): python3-cachetools-4.2.4-1.el9.noarch 1.3 MB/s | 32 kB 00:00 2024-09-12T12:53:49.843 INFO:teuthology.orchestra.run.smithi060.stdout:(95/127): python3-certifi-2023.05.07-4.el9.noar 571 kB/s | 14 kB 00:00 2024-09-12T12:53:49.902 INFO:teuthology.orchestra.run.smithi060.stdout:(96/127): parquet-libs-9.0.0-11.el9.x86_64.rpm 3.4 MB/s | 839 kB 00:00 2024-09-12T12:53:49.927 INFO:teuthology.orchestra.run.smithi060.stdout:(97/127): python3-cheroot-8.6.0-4.el9.noarch.rp 2.0 MB/s | 172 kB 00:00 2024-09-12T12:53:49.961 INFO:teuthology.orchestra.run.smithi060.stdout:(98/127): python3-google-auth-2.34.0-1.el9.noar 6.5 MB/s | 220 kB 00:00 2024-09-12T12:53:49.986 INFO:teuthology.orchestra.run.smithi060.stdout:(99/127): python3-cherrypy-18.6.1-2.el9.noarch. 4.2 MB/s | 358 kB 00:00 2024-09-12T12:53:50.128 INFO:teuthology.orchestra.run.smithi060.stdout:(100/127): python3-grpcio-tools-1.46.7-10.el9.x 1.0 MB/s | 144 kB 00:00 2024-09-12T12:53:50.153 INFO:teuthology.orchestra.run.smithi060.stdout:(101/127): python3-jaraco-8.2.1-3.el9.noarch.rp 429 kB/s | 11 kB 00:00 2024-09-12T12:53:50.178 INFO:teuthology.orchestra.run.smithi060.stdout:(102/127): python3-jaraco-classes-3.2.1-5.el9.n 714 kB/s | 18 kB 00:00 2024-09-12T12:53:50.204 INFO:teuthology.orchestra.run.smithi060.stdout:(103/127): python3-jaraco-collections-3.0.0-8.e 929 kB/s | 23 kB 00:00 2024-09-12T12:53:50.229 INFO:teuthology.orchestra.run.smithi060.stdout:(104/127): python3-jaraco-functools-3.5.0-2.el9 784 kB/s | 19 kB 00:00 2024-09-12T12:53:50.254 INFO:teuthology.orchestra.run.smithi060.stdout:(105/127): python3-jaraco-text-3.2.0-6.el9.noar 784 kB/s | 20 kB 00:00 2024-09-12T12:53:50.313 INFO:teuthology.orchestra.run.smithi060.stdout:(106/127): python3-kubernetes-26.1.0-2.el9.noar 18 MB/s | 1.0 MB 00:00 2024-09-12T12:53:50.338 INFO:teuthology.orchestra.run.smithi060.stdout:(107/127): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-12T12:53:50.363 INFO:teuthology.orchestra.run.smithi060.stdout:(108/127): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-09-12T12:53:50.422 INFO:teuthology.orchestra.run.smithi060.stdout:(109/127): python3-grpcio-1.46.7-10.el9.x86_64. 4.4 MB/s | 2.0 MB 00:00 2024-09-12T12:53:50.448 INFO:teuthology.orchestra.run.smithi060.stdout:(110/127): python3-natsort-7.1.1-5.el9.noarch.r 690 kB/s | 58 kB 00:00 2024-09-12T12:53:50.473 INFO:teuthology.orchestra.run.smithi060.stdout:(111/127): python3-portend-3.1.0-2.el9.noarch.r 658 kB/s | 16 kB 00:00 2024-09-12T12:53:50.526 INFO:teuthology.orchestra.run.smithi060.stdout:(112/127): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.7 MB/s | 90 kB 00:00 2024-09-12T12:53:50.572 INFO:teuthology.orchestra.run.smithi060.stdout:(113/127): python3-pecan-1.4.2-3.el9.noarch.rpm 2.0 MB/s | 272 kB 00:00 2024-09-12T12:53:50.583 INFO:teuthology.orchestra.run.smithi060.stdout:(114/127): python3-repoze-lru-0.7-16.el9.noarch 555 kB/s | 31 kB 00:00 2024-09-12T12:53:50.611 INFO:teuthology.orchestra.run.smithi060.stdout:(115/127): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-09-12T12:53:50.634 INFO:teuthology.orchestra.run.smithi060.stdout:(116/127): python3-tempora-5.0.0-2.el9.noarch.r 1.4 MB/s | 36 kB 00:00 2024-09-12T12:53:50.661 INFO:teuthology.orchestra.run.smithi060.stdout:(117/127): python3-routes-2.5.1-5.el9.noarch.rp 1.8 MB/s | 188 kB 00:00 2024-09-12T12:53:50.701 INFO:teuthology.orchestra.run.smithi060.stdout:(118/127): python3-typing-extensions-4.12.2-2.e 1.2 MB/s | 74 kB 00:00 2024-09-12T12:53:50.721 INFO:teuthology.orchestra.run.smithi060.stdout:(119/127): python3-websocket-client-1.2.3-2.el9 3.5 MB/s | 90 kB 00:00 2024-09-12T12:53:50.751 INFO:teuthology.orchestra.run.smithi060.stdout:(120/127): python3-webob-1.8.8-2.el9.noarch.rpm 2.4 MB/s | 230 kB 00:00 2024-09-12T12:53:50.776 INFO:teuthology.orchestra.run.smithi060.stdout:(121/127): python3-zc-lockfile-2.0-10.el9.noarc 797 kB/s | 20 kB 00:00 2024-09-12T12:53:50.801 INFO:teuthology.orchestra.run.smithi060.stdout:(122/127): re2-20211101-20.el9.x86_64.rpm 7.6 MB/s | 191 kB 00:00 2024-09-12T12:53:50.831 INFO:teuthology.orchestra.run.smithi060.stdout:(123/127): python3-werkzeug-2.0.3-3.el9.1.noarc 3.8 MB/s | 427 kB 00:00 2024-09-12T12:53:50.885 INFO:teuthology.orchestra.run.smithi060.stdout:(124/127): thrift-0.15.0-2.el9.x86_64.rpm 19 MB/s | 1.6 MB 00:00 2024-09-12T12:53:50.983 INFO:teuthology.orchestra.run.smithi130.stdout:Transaction test succeeded. 2024-09-12T12:53:50.983 INFO:teuthology.orchestra.run.smithi130.stdout:Running transaction 2024-09-12T12:53:51.127 INFO:teuthology.orchestra.run.smithi060.stdout:(125/127): librados2-19.1.1-314.g086c737f.el9.x 11 MB/s | 3.4 MB 00:00 2024-09-12T12:53:51.194 INFO:teuthology.orchestra.run.smithi060.stdout:(126/127): librbd1-19.1.1-314.g086c737f.el9.x86 10 MB/s | 3.2 MB 00:00 2024-09-12T12:53:51.469 INFO:teuthology.orchestra.run.smithi060.stdout:(127/127): python3-scipy-1.6.2-8.el9.x86_64.rpm 5.9 MB/s | 16 MB 00:02 2024-09-12T12:53:51.473 INFO:teuthology.orchestra.run.smithi060.stdout:-------------------------------------------------------------------------------- 2024-09-12T12:53:51.473 INFO:teuthology.orchestra.run.smithi060.stdout:Total 20 MB/s | 202 MB 00:09 2024-09-12T12:53:52.843 INFO:teuthology.orchestra.run.smithi060.stdout:Running transaction check 2024-09-12T12:53:52.977 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction check succeeded. 2024-09-12T12:53:52.977 INFO:teuthology.orchestra.run.smithi060.stdout:Running transaction test 2024-09-12T12:53:55.121 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction test succeeded. 2024-09-12T12:53:55.121 INFO:teuthology.orchestra.run.smithi060.stdout:Running transaction 2024-09-12T12:53:56.674 INFO:teuthology.orchestra.run.smithi130.stdout: Preparing : 1/1 2024-09-12T12:53:56.785 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : thrift-0.15.0-2.el9.x86_64 1/129 2024-09-12T12:53:56.817 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 2/129 2024-09-12T12:53:56.883 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 3/129 2024-09-12T12:53:57.276 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 4/129 2024-09-12T12:53:57.421 INFO:teuthology.orchestra.run.smithi130.stdout: Upgrading : librados2-2:19.1.1-314.g086c737f.el9.x86_64 5/129 2024-09-12T12:53:57.576 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: librados2-2:19.1.1-314.g086c737f.el9.x86_64 5/129 2024-09-12T12:53:57.610 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : libcephfs2-2:19.1.1-314.g086c737f.el9.x86_64 6/129 2024-09-12T12:53:57.727 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: libcephfs2-2:19.1.1-314.g086c737f.el9.x86_64 6/129 2024-09-12T12:53:57.781 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-rados-2:19.1.1-314.g086c737f.el9.x86_64 7/129 2024-09-12T12:53:57.845 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 8/129 2024-09-12T12:53:57.885 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 9/129 2024-09-12T12:53:57.920 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : libcephsqlite-2:19.1.1-314.g086c737f.el9.x86_64 10/129 2024-09-12T12:53:58.039 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: libcephsqlite-2:19.1.1-314.g086c737f.el9.x86_64 10/129 2024-09-12T12:53:58.080 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : libradosstriper1-2:19.1.1-314.g086c737f.el9.x86_ 11/129 2024-09-12T12:53:58.146 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: libradosstriper1-2:19.1.1-314.g086c737f.el9.x86_ 11/129 2024-09-12T12:53:58.172 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 12/129 2024-09-12T12:53:58.215 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 12/129 2024-09-12T12:53:58.457 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 13/129 2024-09-12T12:53:58.491 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 13/129 2024-09-12T12:53:58.622 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : re2-1:20211101-20.el9.x86_64 14/129 2024-09-12T12:53:58.733 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 15/129 2024-09-12T12:53:58.781 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 16/129 2024-09-12T12:53:58.825 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : liboath-2.6.7-2.el9.x86_64 17/129 2024-09-12T12:53:58.909 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 18/129 2024-09-12T12:53:58.969 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 19/129 2024-09-12T12:53:59.008 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-packaging-20.9-5.el9.noarch 20/129 2024-09-12T12:53:59.062 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 21/129 2024-09-12T12:53:59.123 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 22/129 2024-09-12T12:53:59.160 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : lua-5.4.4-4.el9.x86_64 23/129 2024-09-12T12:53:59.248 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 24/129 2024-09-12T12:53:59.312 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 25/129 2024-09-12T12:53:59.353 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 26/129 2024-09-12T12:53:59.393 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 27/129 2024-09-12T12:53:59.474 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 28/129 2024-09-12T12:53:59.509 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-ceph-common-2:19.1.1-314.g086c737f.el9.x 29/129 2024-09-12T12:53:59.578 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-ceph-argparse-2:19.1.1-314.g086c737f.el9 30/129 2024-09-12T12:53:59.606 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-cephfs-2:19.1.1-314.g086c737f.el9.x86_64 31/129 2024-09-12T12:53:59.697 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 32/129 2024-09-12T12:53:59.757 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : luarocks-3.9.2-5.el9.noarch 33/129 2024-09-12T12:53:59.849 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 34/129 2024-09-12T12:54:00.018 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 35/129 2024-09-12T12:54:00.085 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 36/129 2024-09-12T12:54:00.534 INFO:teuthology.orchestra.run.smithi060.stdout: Preparing : 1/1 2024-09-12T12:54:00.609 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : thrift-0.15.0-2.el9.x86_64 1/129 2024-09-12T12:54:00.713 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 2/129 2024-09-12T12:54:00.725 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-rsa-4.9-2.el9.noarch 37/129 2024-09-12T12:54:00.820 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 38/129 2024-09-12T12:54:00.855 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 39/129 2024-09-12T12:54:00.888 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 3/129 2024-09-12T12:54:00.899 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 40/129 2024-09-12T12:54:00.938 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : librados-devel-2:19.1.1-314.g086c737f.el9.x86_64 41/129 2024-09-12T12:54:00.978 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 42/129 2024-09-12T12:54:01.039 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 43/129 2024-09-12T12:54:01.128 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 44/129 2024-09-12T12:54:01.163 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 45/129 2024-09-12T12:54:01.199 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 46/129 2024-09-12T12:54:01.283 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 47/129 2024-09-12T12:54:01.286 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 4/129 2024-09-12T12:54:01.344 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 48/129 2024-09-12T12:54:01.390 INFO:teuthology.orchestra.run.smithi060.stdout: Upgrading : librados2-2:19.1.1-314.g086c737f.el9.x86_64 5/129 2024-09-12T12:54:01.393 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 49/129 2024-09-12T12:54:01.540 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: librados2-2:19.1.1-314.g086c737f.el9.x86_64 5/129 2024-09-12T12:54:01.566 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : libcephfs2-2:19.1.1-314.g086c737f.el9.x86_64 6/129 2024-09-12T12:54:01.569 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 50/129 2024-09-12T12:54:01.618 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 51/129 2024-09-12T12:54:01.663 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: libcephfs2-2:19.1.1-314.g086c737f.el9.x86_64 6/129 2024-09-12T12:54:01.694 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 52/129 2024-09-12T12:54:01.719 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-rados-2:19.1.1-314.g086c737f.el9.x86_64 7/129 2024-09-12T12:54:01.780 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 8/129 2024-09-12T12:54:01.812 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 53/129 2024-09-12T12:54:01.825 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 9/129 2024-09-12T12:54:01.851 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 54/129 2024-09-12T12:54:01.852 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : libcephsqlite-2:19.1.1-314.g086c737f.el9.x86_64 10/129 2024-09-12T12:54:01.895 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 55/129 2024-09-12T12:54:01.932 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 56/129 2024-09-12T12:54:01.956 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 57/129 2024-09-12T12:54:01.980 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: libcephsqlite-2:19.1.1-314.g086c737f.el9.x86_64 10/129 2024-09-12T12:54:02.011 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : libradosstriper1-2:19.1.1-314.g086c737f.el9.x86_ 11/129 2024-09-12T12:54:02.076 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 58/129 2024-09-12T12:54:02.085 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: libradosstriper1-2:19.1.1-314.g086c737f.el9.x86_ 11/129 2024-09-12T12:54:02.113 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 12/129 2024-09-12T12:54:02.151 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 12/129 2024-09-12T12:54:02.243 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 59/129 2024-09-12T12:54:02.389 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 13/129 2024-09-12T12:54:02.456 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 13/129 2024-09-12T12:54:02.549 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 60/129 2024-09-12T12:54:02.603 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : re2-1:20211101-20.el9.x86_64 14/129 2024-09-12T12:54:02.609 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 61/129 2024-09-12T12:54:02.656 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 62/129 2024-09-12T12:54:02.693 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 63/129 2024-09-12T12:54:02.728 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 64/129 2024-09-12T12:54:02.730 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 15/129 2024-09-12T12:54:02.761 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 65/129 2024-09-12T12:54:02.771 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 16/129 2024-09-12T12:54:02.804 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 66/129 2024-09-12T12:54:02.816 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : liboath-2.6.7-2.el9.x86_64 17/129 2024-09-12T12:54:02.836 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 67/129 2024-09-12T12:54:02.902 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 18/129 2024-09-12T12:54:02.935 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 68/129 2024-09-12T12:54:02.962 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 19/129 2024-09-12T12:54:02.995 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 69/129 2024-09-12T12:54:03.008 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-packaging-20.9-5.el9.noarch 20/129 2024-09-12T12:54:03.031 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 70/129 2024-09-12T12:54:03.089 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 21/129 2024-09-12T12:54:03.134 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 22/129 2024-09-12T12:54:03.153 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 71/129 2024-09-12T12:54:03.169 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : lua-5.4.4-4.el9.x86_64 23/129 2024-09-12T12:54:03.222 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 72/129 2024-09-12T12:54:03.259 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 24/129 2024-09-12T12:54:03.321 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 25/129 2024-09-12T12:54:03.360 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 26/129 2024-09-12T12:54:03.409 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 27/129 2024-09-12T12:54:03.502 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 28/129 2024-09-12T12:54:03.533 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-ceph-common-2:19.1.1-314.g086c737f.el9.x 29/129 2024-09-12T12:54:03.578 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-ceph-argparse-2:19.1.1-314.g086c737f.el9 30/129 2024-09-12T12:54:03.605 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-cephfs-2:19.1.1-314.g086c737f.el9.x86_64 31/129 2024-09-12T12:54:03.703 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 32/129 2024-09-12T12:54:03.758 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : luarocks-3.9.2-5.el9.noarch 33/129 2024-09-12T12:54:03.854 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 34/129 2024-09-12T12:54:04.027 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 35/129 2024-09-12T12:54:04.103 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 36/129 2024-09-12T12:54:04.122 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 73/129 2024-09-12T12:54:04.174 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 74/129 2024-09-12T12:54:04.236 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 75/129 2024-09-12T12:54:04.355 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 76/129 2024-09-12T12:54:04.399 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 77/129 2024-09-12T12:54:04.635 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : openblas-0.3.26-2.el9.x86_64 78/129 2024-09-12T12:54:04.671 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 79/129 2024-09-12T12:54:04.765 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 80/129 2024-09-12T12:54:04.786 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-rsa-4.9-2.el9.noarch 37/129 2024-09-12T12:54:04.882 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 38/129 2024-09-12T12:54:04.913 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 39/129 2024-09-12T12:54:04.958 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 40/129 2024-09-12T12:54:05.022 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : librados-devel-2:19.1.1-314.g086c737f.el9.x86_64 41/129 2024-09-12T12:54:05.063 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 42/129 2024-09-12T12:54:05.125 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 43/129 2024-09-12T12:54:05.209 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 44/129 2024-09-12T12:54:05.255 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 45/129 2024-09-12T12:54:05.292 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 46/129 2024-09-12T12:54:05.352 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 47/129 2024-09-12T12:54:05.404 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 48/129 2024-09-12T12:54:05.453 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 49/129 2024-09-12T12:54:05.568 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 81/129 2024-09-12T12:54:05.637 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 50/129 2024-09-12T12:54:05.685 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 51/129 2024-09-12T12:54:05.709 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 82/129 2024-09-12T12:54:05.729 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 52/129 2024-09-12T12:54:05.864 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 53/129 2024-09-12T12:54:05.910 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 54/129 2024-09-12T12:54:05.963 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 55/129 2024-09-12T12:54:05.999 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 56/129 2024-09-12T12:54:06.023 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 57/129 2024-09-12T12:54:06.142 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 58/129 2024-09-12T12:54:06.311 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 59/129 2024-09-12T12:54:06.372 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 60/129 2024-09-12T12:54:06.423 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 61/129 2024-09-12T12:54:06.466 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 62/129 2024-09-12T12:54:06.502 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 63/129 2024-09-12T12:54:06.537 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 64/129 2024-09-12T12:54:06.570 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 65/129 2024-09-12T12:54:06.614 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 66/129 2024-09-12T12:54:06.648 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 67/129 2024-09-12T12:54:06.752 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 68/129 2024-09-12T12:54:06.806 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 69/129 2024-09-12T12:54:06.852 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 70/129 2024-09-12T12:54:06.979 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 71/129 2024-09-12T12:54:07.047 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 72/129 2024-09-12T12:54:07.299 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/129 2024-09-12T12:54:07.526 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 84/129 2024-09-12T12:54:08.005 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 73/129 2024-09-12T12:54:08.010 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 85/129 2024-09-12T12:54:08.052 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 74/129 2024-09-12T12:54:08.068 INFO:teuthology.orchestra.run.smithi130.stdout: Upgrading : librbd1-2:19.1.1-314.g086c737f.el9.x86_64 86/129 2024-09-12T12:54:08.077 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 75/129 2024-09-12T12:54:08.189 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 76/129 2024-09-12T12:54:08.197 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: librbd1-2:19.1.1-314.g086c737f.el9.x86_64 86/129 2024-09-12T12:54:08.233 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-rbd-2:19.1.1-314.g086c737f.el9.x86_64 87/129 2024-09-12T12:54:08.235 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 77/129 2024-09-12T12:54:08.293 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 88/129 2024-09-12T12:54:08.483 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : openblas-0.3.26-2.el9.x86_64 78/129 2024-09-12T12:54:08.523 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 79/129 2024-09-12T12:54:08.621 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 80/129 2024-09-12T12:54:08.811 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 89/129 2024-09-12T12:54:08.846 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : librgw2-2:19.1.1-314.g086c737f.el9.x86_64 90/129 2024-09-12T12:54:09.270 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: librgw2-2:19.1.1-314.g086c737f.el9.x86_64 90/129 2024-09-12T12:54:09.298 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-rgw-2:19.1.1-314.g086c737f.el9.x86_64 91/129 2024-09-12T12:54:09.483 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 81/129 2024-09-12T12:54:09.691 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 82/129 2024-09-12T12:54:11.385 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/129 2024-09-12T12:54:11.490 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 84/129 2024-09-12T12:54:11.809 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: ceph-common-2:19.1.1-314.g086c737f.el9.x86_64 92/129 2024-09-12T12:54:11.845 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 85/129 2024-09-12T12:54:11.860 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : ceph-common-2:19.1.1-314.g086c737f.el9.x86_64 92/129 2024-09-12T12:54:11.891 INFO:teuthology.orchestra.run.smithi060.stdout: Upgrading : librbd1-2:19.1.1-314.g086c737f.el9.x86_64 86/129 2024-09-12T12:54:12.005 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: librbd1-2:19.1.1-314.g086c737f.el9.x86_64 86/129 2024-09-12T12:54:12.047 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-rbd-2:19.1.1-314.g086c737f.el9.x86_64 87/129 2024-09-12T12:54:12.099 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 88/129 2024-09-12T12:54:12.490 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: ceph-common-2:19.1.1-314.g086c737f.el9.x86_64 92/129 2024-09-12T12:54:12.538 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : ceph-base-2:19.1.1-314.g086c737f.el9.x86_64 93/129 2024-09-12T12:54:12.649 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: ceph-base-2:19.1.1-314.g086c737f.el9.x86_64 93/129 2024-09-12T12:54:12.649 INFO:teuthology.orchestra.run.smithi130.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-12T12:54:12.649 INFO:teuthology.orchestra.run.smithi130.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-12T12:54:12.649 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-09-12T12:54:12.652 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 89/129 2024-09-12T12:54:12.698 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : ceph-selinux-2:19.1.1-314.g086c737f.el9.x86_64 94/129 2024-09-12T12:54:12.710 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : librgw2-2:19.1.1-314.g086c737f.el9.x86_64 90/129 2024-09-12T12:54:13.119 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: librgw2-2:19.1.1-314.g086c737f.el9.x86_64 90/129 2024-09-12T12:54:13.146 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-rgw-2:19.1.1-314.g086c737f.el9.x86_64 91/129 2024-09-12T12:54:15.821 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-common-2:19.1.1-314.g086c737f.el9.x86_64 92/129 2024-09-12T12:54:15.859 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-common-2:19.1.1-314.g086c737f.el9.x86_64 92/129 2024-09-12T12:54:16.545 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-common-2:19.1.1-314.g086c737f.el9.x86_64 92/129 2024-09-12T12:54:16.589 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-base-2:19.1.1-314.g086c737f.el9.x86_64 93/129 2024-09-12T12:54:16.706 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-base-2:19.1.1-314.g086c737f.el9.x86_64 93/129 2024-09-12T12:54:16.707 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-12T12:54:16.707 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-12T12:54:16.707 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-12T12:54:16.749 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-selinux-2:19.1.1-314.g086c737f.el9.x86_64 94/129 2024-09-12T12:54:25.888 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: ceph-selinux-2:19.1.1-314.g086c737f.el9.x86_64 94/129 2024-09-12T12:54:25.889 INFO:teuthology.orchestra.run.smithi130.stdout:skipping the directory /sys 2024-09-12T12:54:25.889 INFO:teuthology.orchestra.run.smithi130.stdout:skipping the directory /proc 2024-09-12T12:54:25.889 INFO:teuthology.orchestra.run.smithi130.stdout:skipping the directory /mnt 2024-09-12T12:54:25.889 INFO:teuthology.orchestra.run.smithi130.stdout:skipping the directory /var/tmp 2024-09-12T12:54:25.889 INFO:teuthology.orchestra.run.smithi130.stdout:skipping the directory /home 2024-09-12T12:54:25.889 INFO:teuthology.orchestra.run.smithi130.stdout:skipping the directory /root 2024-09-12T12:54:25.889 INFO:teuthology.orchestra.run.smithi130.stdout:skipping the directory /tmp 2024-09-12T12:54:25.889 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-09-12T12:54:26.117 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : ceph-volume-2:19.1.1-314.g086c737f.el9.noarch 95/129 2024-09-12T12:54:26.152 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: ceph-volume-2:19.1.1-314.g086c737f.el9.noarch 95/129 2024-09-12T12:54:26.153 INFO:teuthology.orchestra.run.smithi130.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T12:54:26.153 INFO:teuthology.orchestra.run.smithi130.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-12T12:54:26.153 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-09-12T12:54:27.718 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : ceph-osd-2:19.1.1-314.g086c737f.el9.x86_64 96/129 2024-09-12T12:54:27.775 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: ceph-osd-2:19.1.1-314.g086c737f.el9.x86_64 96/129 2024-09-12T12:54:27.776 INFO:teuthology.orchestra.run.smithi130.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T12:54:27.776 INFO:teuthology.orchestra.run.smithi130.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-12T12:54:27.776 INFO:teuthology.orchestra.run.smithi130.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-12T12:54:27.776 INFO:teuthology.orchestra.run.smithi130.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-12T12:54:27.776 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-09-12T12:54:28.018 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : ceph-mds-2:19.1.1-314.g086c737f.el9.x86_64 97/129 2024-09-12T12:54:28.069 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: ceph-mds-2:19.1.1-314.g086c737f.el9.x86_64 97/129 2024-09-12T12:54:28.069 INFO:teuthology.orchestra.run.smithi130.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T12:54:28.069 INFO:teuthology.orchestra.run.smithi130.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-12T12:54:28.069 INFO:teuthology.orchestra.run.smithi130.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-12T12:54:28.069 INFO:teuthology.orchestra.run.smithi130.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-12T12:54:28.070 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-09-12T12:54:28.544 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : ceph-mon-2:19.1.1-314.g086c737f.el9.x86_64 98/129 2024-09-12T12:54:28.594 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: ceph-mon-2:19.1.1-314.g086c737f.el9.x86_64 98/129 2024-09-12T12:54:28.595 INFO:teuthology.orchestra.run.smithi130.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T12:54:28.595 INFO:teuthology.orchestra.run.smithi130.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-12T12:54:28.595 INFO:teuthology.orchestra.run.smithi130.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-12T12:54:28.595 INFO:teuthology.orchestra.run.smithi130.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-12T12:54:28.595 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-09-12T12:54:28.702 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-ply-3.11-14.el9.noarch 99/129 2024-09-12T12:54:28.766 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 100/129 2024-09-12T12:54:28.991 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 101/129 2024-09-12T12:54:29.054 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 102/129 2024-09-12T12:54:29.165 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 103/129 2024-09-12T12:54:29.274 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 104/129 2024-09-12T12:54:29.428 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 105/129 2024-09-12T12:54:29.488 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 106/129 2024-09-12T12:54:29.552 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 107/129 2024-09-12T12:54:29.578 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : fuse-2.9.9-16.el9.x86_64 108/129 2024-09-12T12:54:29.923 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-selinux-2:19.1.1-314.g086c737f.el9.x86_64 94/129 2024-09-12T12:54:29.923 INFO:teuthology.orchestra.run.smithi060.stdout:skipping the directory /sys 2024-09-12T12:54:29.923 INFO:teuthology.orchestra.run.smithi060.stdout:skipping the directory /proc 2024-09-12T12:54:29.923 INFO:teuthology.orchestra.run.smithi060.stdout:skipping the directory /mnt 2024-09-12T12:54:29.923 INFO:teuthology.orchestra.run.smithi060.stdout:skipping the directory /var/tmp 2024-09-12T12:54:29.923 INFO:teuthology.orchestra.run.smithi060.stdout:skipping the directory /home 2024-09-12T12:54:29.923 INFO:teuthology.orchestra.run.smithi060.stdout:skipping the directory /root 2024-09-12T12:54:29.923 INFO:teuthology.orchestra.run.smithi060.stdout:skipping the directory /tmp 2024-09-12T12:54:29.923 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-12T12:54:30.017 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: cephadm-2:19.1.1-314.g086c737f.el9.noarch 109/129 2024-09-12T12:54:30.052 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : cephadm-2:19.1.1-314.g086c737f.el9.noarch 109/129 2024-09-12T12:54:30.093 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-314.g086c737f.el 110/129 2024-09-12T12:54:30.143 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-volume-2:19.1.1-314.g086c737f.el9.noarch 95/129 2024-09-12T12:54:30.161 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-314.g086c737f.e 111/129 2024-09-12T12:54:30.180 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-volume-2:19.1.1-314.g086c737f.el9.noarch 95/129 2024-09-12T12:54:30.180 INFO:teuthology.orchestra.run.smithi060.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T12:54:30.180 INFO:teuthology.orchestra.run.smithi060.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-12T12:54:30.181 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-12T12:54:30.191 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-314.g086c737f.el9.noar 112/129 2024-09-12T12:54:31.402 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-314.g086c737f.el9.noar 112/129 2024-09-12T12:54:31.443 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-314.g086c737f.el9.no 113/129 2024-09-12T12:54:31.838 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-osd-2:19.1.1-314.g086c737f.el9.x86_64 96/129 2024-09-12T12:54:31.896 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-osd-2:19.1.1-314.g086c737f.el9.x86_64 96/129 2024-09-12T12:54:31.897 INFO:teuthology.orchestra.run.smithi060.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T12:54:31.897 INFO:teuthology.orchestra.run.smithi060.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-12T12:54:31.897 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-12T12:54:31.897 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-12T12:54:31.897 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-12T12:54:32.155 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-mds-2:19.1.1-314.g086c737f.el9.x86_64 97/129 2024-09-12T12:54:32.206 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-mds-2:19.1.1-314.g086c737f.el9.x86_64 97/129 2024-09-12T12:54:32.206 INFO:teuthology.orchestra.run.smithi060.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T12:54:32.206 INFO:teuthology.orchestra.run.smithi060.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-12T12:54:32.206 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-12T12:54:32.206 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-12T12:54:32.206 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-12T12:54:32.523 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-314.g086c737f.el9.no 113/129 2024-09-12T12:54:32.551 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-314.g086c 114/129 2024-09-12T12:54:32.686 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-314.g086c 114/129 2024-09-12T12:54:32.706 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-mon-2:19.1.1-314.g086c737f.el9.x86_64 98/129 2024-09-12T12:54:32.758 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-mon-2:19.1.1-314.g086c737f.el9.x86_64 98/129 2024-09-12T12:54:32.758 INFO:teuthology.orchestra.run.smithi060.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T12:54:32.758 INFO:teuthology.orchestra.run.smithi060.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-12T12:54:32.758 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-12T12:54:32.758 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-12T12:54:32.758 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-12T12:54:32.842 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-ply-3.11-14.el9.noarch 99/129 2024-09-12T12:54:32.856 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-314.g086c737f.el9 115/129 2024-09-12T12:54:32.915 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 100/129 2024-09-12T12:54:32.930 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : ceph-mgr-2:19.1.1-314.g086c737f.el9.x86_64 116/129 2024-09-12T12:54:32.980 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: ceph-mgr-2:19.1.1-314.g086c737f.el9.x86_64 116/129 2024-09-12T12:54:32.981 INFO:teuthology.orchestra.run.smithi130.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T12:54:32.981 INFO:teuthology.orchestra.run.smithi130.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-12T12:54:32.981 INFO:teuthology.orchestra.run.smithi130.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-12T12:54:32.981 INFO:teuthology.orchestra.run.smithi130.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-12T12:54:32.981 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-09-12T12:54:33.037 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : ceph-mgr-rook-2:19.1.1-314.g086c737f.el9.noarch 117/129 2024-09-12T12:54:33.063 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-314.g086c737f.el9.noarch 117/129 2024-09-12T12:54:33.169 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : ceph-2:19.1.1-314.g086c737f.el9.x86_64 118/129 2024-09-12T12:54:33.176 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 101/129 2024-09-12T12:54:33.251 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 102/129 2024-09-12T12:54:33.340 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 103/129 2024-09-12T12:54:33.449 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 104/129 2024-09-12T12:54:33.621 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 105/129 2024-09-12T12:54:33.670 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 106/129 2024-09-12T12:54:33.706 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 107/129 2024-09-12T12:54:33.732 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : fuse-2.9.9-16.el9.x86_64 108/129 2024-09-12T12:54:34.159 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : ceph-fuse-2:19.1.1-314.g086c737f.el9.x86_64 119/129 2024-09-12T12:54:34.222 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : ceph-radosgw-2:19.1.1-314.g086c737f.el9.x86_64 120/129 2024-09-12T12:54:34.273 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-314.g086c737f.el9.x86_64 120/129 2024-09-12T12:54:34.274 INFO:teuthology.orchestra.run.smithi130.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T12:54:34.274 INFO:teuthology.orchestra.run.smithi130.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-12T12:54:34.274 INFO:teuthology.orchestra.run.smithi130.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-12T12:54:34.274 INFO:teuthology.orchestra.run.smithi130.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-12T12:54:34.274 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-09-12T12:54:34.322 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-314.g086c73 121/129 2024-09-12T12:54:34.373 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-314.g086c73 121/129 2024-09-12T12:54:34.373 INFO:teuthology.orchestra.run.smithi130.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T12:54:34.373 INFO:teuthology.orchestra.run.smithi130.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-12T12:54:34.373 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-09-12T12:54:34.376 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: cephadm-2:19.1.1-314.g086c737f.el9.noarch 109/129 2024-09-12T12:54:34.406 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : cephadm-2:19.1.1-314.g086c737f.el9.noarch 109/129 2024-09-12T12:54:34.440 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-314.g086c737f.el 110/129 2024-09-12T12:54:34.510 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-314.g086c737f.e 111/129 2024-09-12T12:54:34.537 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-314.g086c737f.el9.noar 112/129 2024-09-12T12:54:34.698 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : rbd-mirror-2:19.1.1-314.g086c737f.el9.x86_64 122/129 2024-09-12T12:54:34.749 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: rbd-mirror-2:19.1.1-314.g086c737f.el9.x86_64 122/129 2024-09-12T12:54:34.749 INFO:teuthology.orchestra.run.smithi130.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T12:54:34.749 INFO:teuthology.orchestra.run.smithi130.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-12T12:54:34.749 INFO:teuthology.orchestra.run.smithi130.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-12T12:54:34.749 INFO:teuthology.orchestra.run.smithi130.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-12T12:54:34.749 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-09-12T12:54:35.827 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-314.g086c737f.el9.noar 112/129 2024-09-12T12:54:35.864 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-314.g086c737f.el9.no 113/129 2024-09-12T12:54:37.015 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-314.g086c737f.el9.no 113/129 2024-09-12T12:54:37.047 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-314.g086c 114/129 2024-09-12T12:54:37.191 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-314.g086c 114/129 2024-09-12T12:54:37.369 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-314.g086c737f.el9 115/129 2024-09-12T12:54:37.408 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-mgr-2:19.1.1-314.g086c737f.el9.x86_64 116/129 2024-09-12T12:54:37.460 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-mgr-2:19.1.1-314.g086c737f.el9.x86_64 116/129 2024-09-12T12:54:37.461 INFO:teuthology.orchestra.run.smithi060.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T12:54:37.461 INFO:teuthology.orchestra.run.smithi060.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-12T12:54:37.461 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-12T12:54:37.461 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-12T12:54:37.461 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-12T12:54:37.516 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-mgr-rook-2:19.1.1-314.g086c737f.el9.noarch 117/129 2024-09-12T12:54:37.543 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-314.g086c737f.el9.noarch 117/129 2024-09-12T12:54:37.670 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-2:19.1.1-314.g086c737f.el9.x86_64 118/129 2024-09-12T12:54:38.737 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-fuse-2:19.1.1-314.g086c737f.el9.x86_64 119/129 2024-09-12T12:54:38.776 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-radosgw-2:19.1.1-314.g086c737f.el9.x86_64 120/129 2024-09-12T12:54:38.828 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-314.g086c737f.el9.x86_64 120/129 2024-09-12T12:54:38.828 INFO:teuthology.orchestra.run.smithi060.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T12:54:38.828 INFO:teuthology.orchestra.run.smithi060.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-12T12:54:38.828 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-12T12:54:38.829 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-12T12:54:38.829 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-12T12:54:38.877 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-314.g086c73 121/129 2024-09-12T12:54:38.927 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-314.g086c73 121/129 2024-09-12T12:54:38.927 INFO:teuthology.orchestra.run.smithi060.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T12:54:38.927 INFO:teuthology.orchestra.run.smithi060.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-12T12:54:38.927 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-12T12:54:39.278 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : rbd-mirror-2:19.1.1-314.g086c737f.el9.x86_64 122/129 2024-09-12T12:54:39.329 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: rbd-mirror-2:19.1.1-314.g086c737f.el9.x86_64 122/129 2024-09-12T12:54:39.329 INFO:teuthology.orchestra.run.smithi060.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T12:54:39.330 INFO:teuthology.orchestra.run.smithi060.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-12T12:54:39.330 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-12T12:54:39.330 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-12T12:54:39.330 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-12T12:54:39.685 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : ceph-test-2:19.1.1-314.g086c737f.el9.x86_64 123/129 2024-09-12T12:54:39.760 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : rbd-fuse-2:19.1.1-314.g086c737f.el9.x86_64 124/129 2024-09-12T12:54:39.791 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : rbd-nbd-2:19.1.1-314.g086c737f.el9.x86_64 125/129 2024-09-12T12:54:39.830 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : libcephfs-devel-2:19.1.1-314.g086c737f.el9.x86_6 126/129 2024-09-12T12:54:39.866 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : pv-1.6.20-1.el9.x86_64 127/129 2024-09-12T12:54:39.866 INFO:teuthology.orchestra.run.smithi130.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 128/129 2024-09-12T12:54:40.006 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 128/129 2024-09-12T12:54:40.006 INFO:teuthology.orchestra.run.smithi130.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 129/129 2024-09-12T12:54:40.437 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 129/129 2024-09-12T12:54:40.437 INFO:teuthology.orchestra.run.smithi130.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-09-12T12:54:40.437 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-09-12T12:54:44.575 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-test-2:19.1.1-314.g086c737f.el9.x86_64 123/129 2024-09-12T12:54:44.642 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : rbd-fuse-2:19.1.1-314.g086c737f.el9.x86_64 124/129 2024-09-12T12:54:44.673 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : rbd-nbd-2:19.1.1-314.g086c737f.el9.x86_64 125/129 2024-09-12T12:54:44.712 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : libcephfs-devel-2:19.1.1-314.g086c737f.el9.x86_6 126/129 2024-09-12T12:54:44.747 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : pv-1.6.20-1.el9.x86_64 127/129 2024-09-12T12:54:44.748 INFO:teuthology.orchestra.run.smithi060.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 128/129 2024-09-12T12:54:44.896 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 128/129 2024-09-12T12:54:44.896 INFO:teuthology.orchestra.run.smithi060.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 129/129 2024-09-12T12:54:45.311 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 129/129 2024-09-12T12:54:45.311 INFO:teuthology.orchestra.run.smithi060.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-09-12T12:54:45.311 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-12T12:54:45.324 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : ceph-2:19.1.1-314.g086c737f.el9.x86_64 1/129 2024-09-12T12:54:45.324 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : ceph-base-2:19.1.1-314.g086c737f.el9.x86_64 2/129 2024-09-12T12:54:45.324 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : ceph-common-2:19.1.1-314.g086c737f.el9.x86_64 3/129 2024-09-12T12:54:45.324 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : ceph-fuse-2:19.1.1-314.g086c737f.el9.x86_64 4/129 2024-09-12T12:54:45.325 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-314.g086c73 5/129 2024-09-12T12:54:45.325 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : ceph-mds-2:19.1.1-314.g086c737f.el9.x86_64 6/129 2024-09-12T12:54:45.325 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : ceph-mgr-2:19.1.1-314.g086c737f.el9.x86_64 7/129 2024-09-12T12:54:45.325 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : ceph-mon-2:19.1.1-314.g086c737f.el9.x86_64 8/129 2024-09-12T12:54:45.325 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : ceph-osd-2:19.1.1-314.g086c737f.el9.x86_64 9/129 2024-09-12T12:54:45.325 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : ceph-radosgw-2:19.1.1-314.g086c737f.el9.x86_64 10/129 2024-09-12T12:54:45.325 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : ceph-selinux-2:19.1.1-314.g086c737f.el9.x86_64 11/129 2024-09-12T12:54:45.325 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : ceph-test-2:19.1.1-314.g086c737f.el9.x86_64 12/129 2024-09-12T12:54:45.325 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : libcephfs-devel-2:19.1.1-314.g086c737f.el9.x86_6 13/129 2024-09-12T12:54:45.325 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : libcephfs2-2:19.1.1-314.g086c737f.el9.x86_64 14/129 2024-09-12T12:54:45.325 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : libcephsqlite-2:19.1.1-314.g086c737f.el9.x86_64 15/129 2024-09-12T12:54:45.325 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : librados-devel-2:19.1.1-314.g086c737f.el9.x86_64 16/129 2024-09-12T12:54:45.325 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : libradosstriper1-2:19.1.1-314.g086c737f.el9.x86_ 17/129 2024-09-12T12:54:45.326 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : librgw2-2:19.1.1-314.g086c737f.el9.x86_64 18/129 2024-09-12T12:54:45.326 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-ceph-argparse-2:19.1.1-314.g086c737f.el9 19/129 2024-09-12T12:54:45.327 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-ceph-common-2:19.1.1-314.g086c737f.el9.x 20/129 2024-09-12T12:54:45.327 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-cephfs-2:19.1.1-314.g086c737f.el9.x86_64 21/129 2024-09-12T12:54:45.327 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-rados-2:19.1.1-314.g086c737f.el9.x86_64 22/129 2024-09-12T12:54:45.327 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-rbd-2:19.1.1-314.g086c737f.el9.x86_64 23/129 2024-09-12T12:54:45.327 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-rgw-2:19.1.1-314.g086c737f.el9.x86_64 24/129 2024-09-12T12:54:45.327 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : rbd-fuse-2:19.1.1-314.g086c737f.el9.x86_64 25/129 2024-09-12T12:54:45.327 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : rbd-mirror-2:19.1.1-314.g086c737f.el9.x86_64 26/129 2024-09-12T12:54:45.327 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : rbd-nbd-2:19.1.1-314.g086c737f.el9.x86_64 27/129 2024-09-12T12:54:45.327 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-314.g086c737f.e 28/129 2024-09-12T12:54:45.327 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-314.g086c737f.el9.noar 29/129 2024-09-12T12:54:45.327 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-314.g086c737f.el9.no 30/129 2024-09-12T12:54:45.328 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-314.g086c 31/129 2024-09-12T12:54:45.328 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-314.g086c737f.el9 32/129 2024-09-12T12:54:45.328 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : ceph-mgr-rook-2:19.1.1-314.g086c737f.el9.noarch 33/129 2024-09-12T12:54:45.328 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-314.g086c737f.el 34/129 2024-09-12T12:54:45.328 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : ceph-volume-2:19.1.1-314.g086c737f.el9.noarch 35/129 2024-09-12T12:54:45.328 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : cephadm-2:19.1.1-314.g086c737f.el9.noarch 36/129 2024-09-12T12:54:45.328 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 37/129 2024-09-12T12:54:45.328 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 38/129 2024-09-12T12:54:45.328 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 39/129 2024-09-12T12:54:45.328 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 40/129 2024-09-12T12:54:45.328 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 41/129 2024-09-12T12:54:45.328 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-ply-3.11-14.el9.noarch 42/129 2024-09-12T12:54:45.328 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 43/129 2024-09-12T12:54:45.329 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 44/129 2024-09-12T12:54:45.329 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 45/129 2024-09-12T12:54:45.329 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 46/129 2024-09-12T12:54:45.329 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 47/129 2024-09-12T12:54:45.329 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 48/129 2024-09-12T12:54:45.329 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 49/129 2024-09-12T12:54:45.329 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 50/129 2024-09-12T12:54:45.329 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 51/129 2024-09-12T12:54:45.329 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 52/129 2024-09-12T12:54:45.329 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 53/129 2024-09-12T12:54:45.330 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : lua-5.4.4-4.el9.x86_64 54/129 2024-09-12T12:54:45.330 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 55/129 2024-09-12T12:54:45.330 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 56/129 2024-09-12T12:54:45.330 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 57/129 2024-09-12T12:54:45.330 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 58/129 2024-09-12T12:54:45.330 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 59/129 2024-09-12T12:54:45.330 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 60/129 2024-09-12T12:54:45.330 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 61/129 2024-09-12T12:54:45.330 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 62/129 2024-09-12T12:54:45.330 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 63/129 2024-09-12T12:54:45.330 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 64/129 2024-09-12T12:54:45.330 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 65/129 2024-09-12T12:54:45.331 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 66/129 2024-09-12T12:54:45.331 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 67/129 2024-09-12T12:54:45.331 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 68/129 2024-09-12T12:54:45.331 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 69/129 2024-09-12T12:54:45.331 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 70/129 2024-09-12T12:54:45.331 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 71/129 2024-09-12T12:54:45.331 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 72/129 2024-09-12T12:54:45.331 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 73/129 2024-09-12T12:54:45.331 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 74/129 2024-09-12T12:54:45.331 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 75/129 2024-09-12T12:54:45.331 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 76/129 2024-09-12T12:54:45.331 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 77/129 2024-09-12T12:54:45.331 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 78/129 2024-09-12T12:54:45.332 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 79/129 2024-09-12T12:54:45.332 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 80/129 2024-09-12T12:54:45.332 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 81/129 2024-09-12T12:54:45.332 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 82/129 2024-09-12T12:54:45.332 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 83/129 2024-09-12T12:54:45.332 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 84/129 2024-09-12T12:54:45.332 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 85/129 2024-09-12T12:54:45.332 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 86/129 2024-09-12T12:54:45.332 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 87/129 2024-09-12T12:54:45.332 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 88/129 2024-09-12T12:54:45.332 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 89/129 2024-09-12T12:54:45.332 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 90/129 2024-09-12T12:54:45.332 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 91/129 2024-09-12T12:54:45.333 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 92/129 2024-09-12T12:54:45.333 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : pv-1.6.20-1.el9.x86_64 93/129 2024-09-12T12:54:45.333 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 94/129 2024-09-12T12:54:45.333 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 95/129 2024-09-12T12:54:45.333 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 96/129 2024-09-12T12:54:45.333 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 97/129 2024-09-12T12:54:45.333 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 98/129 2024-09-12T12:54:45.333 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 99/129 2024-09-12T12:54:45.333 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 100/129 2024-09-12T12:54:45.333 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 101/129 2024-09-12T12:54:45.333 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 102/129 2024-09-12T12:54:45.333 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 103/129 2024-09-12T12:54:45.333 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 104/129 2024-09-12T12:54:45.334 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 105/129 2024-09-12T12:54:45.334 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 106/129 2024-09-12T12:54:45.334 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 107/129 2024-09-12T12:54:45.334 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 108/129 2024-09-12T12:54:45.334 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 109/129 2024-09-12T12:54:45.334 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 110/129 2024-09-12T12:54:45.334 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 111/129 2024-09-12T12:54:45.334 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 112/129 2024-09-12T12:54:45.334 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 113/129 2024-09-12T12:54:45.334 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 114/129 2024-09-12T12:54:45.334 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 115/129 2024-09-12T12:54:45.334 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 116/129 2024-09-12T12:54:45.334 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 117/129 2024-09-12T12:54:45.334 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 118/129 2024-09-12T12:54:45.335 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 119/129 2024-09-12T12:54:45.335 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 120/129 2024-09-12T12:54:45.335 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 121/129 2024-09-12T12:54:45.335 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 122/129 2024-09-12T12:54:45.335 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 123/129 2024-09-12T12:54:45.335 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : re2-1:20211101-20.el9.x86_64 124/129 2024-09-12T12:54:45.335 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 125/129 2024-09-12T12:54:45.335 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : librados2-2:19.1.1-314.g086c737f.el9.x86_64 126/129 2024-09-12T12:54:45.335 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 127/129 2024-09-12T12:54:45.335 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : librbd1-2:19.1.1-314.g086c737f.el9.x86_64 128/129 2024-09-12T12:54:47.924 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 129/129 2024-09-12T12:54:47.924 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-09-12T12:54:47.924 INFO:teuthology.orchestra.run.smithi130.stdout:Upgraded: 2024-09-12T12:54:47.924 INFO:teuthology.orchestra.run.smithi130.stdout: librados2-2:19.1.1-314.g086c737f.el9.x86_64 2024-09-12T12:54:47.924 INFO:teuthology.orchestra.run.smithi130.stdout: librbd1-2:19.1.1-314.g086c737f.el9.x86_64 2024-09-12T12:54:47.925 INFO:teuthology.orchestra.run.smithi130.stdout:Installed: 2024-09-12T12:54:47.925 INFO:teuthology.orchestra.run.smithi130.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-12T12:54:47.925 INFO:teuthology.orchestra.run.smithi130.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-12T12:54:47.925 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-2:19.1.1-314.g086c737f.el9.x86_64 2024-09-12T12:54:47.925 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-base-2:19.1.1-314.g086c737f.el9.x86_64 2024-09-12T12:54:47.925 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-common-2:19.1.1-314.g086c737f.el9.x86_64 2024-09-12T12:54:47.925 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-fuse-2:19.1.1-314.g086c737f.el9.x86_64 2024-09-12T12:54:47.925 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-grafana-dashboards-2:19.1.1-314.g086c737f.el9.noarch 2024-09-12T12:54:47.925 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-immutable-object-cache-2:19.1.1-314.g086c737f.el9.x86_64 2024-09-12T12:54:47.925 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-mds-2:19.1.1-314.g086c737f.el9.x86_64 2024-09-12T12:54:47.925 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-mgr-2:19.1.1-314.g086c737f.el9.x86_64 2024-09-12T12:54:47.925 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-mgr-cephadm-2:19.1.1-314.g086c737f.el9.noarch 2024-09-12T12:54:47.925 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-mgr-dashboard-2:19.1.1-314.g086c737f.el9.noarch 2024-09-12T12:54:47.926 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-mgr-diskprediction-local-2:19.1.1-314.g086c737f.el9.noarch 2024-09-12T12:54:47.926 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-mgr-modules-core-2:19.1.1-314.g086c737f.el9.noarch 2024-09-12T12:54:47.926 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-mgr-rook-2:19.1.1-314.g086c737f.el9.noarch 2024-09-12T12:54:47.926 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-mon-2:19.1.1-314.g086c737f.el9.x86_64 2024-09-12T12:54:47.926 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-osd-2:19.1.1-314.g086c737f.el9.x86_64 2024-09-12T12:54:47.926 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-prometheus-alerts-2:19.1.1-314.g086c737f.el9.noarch 2024-09-12T12:54:47.926 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-radosgw-2:19.1.1-314.g086c737f.el9.x86_64 2024-09-12T12:54:47.926 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-selinux-2:19.1.1-314.g086c737f.el9.x86_64 2024-09-12T12:54:47.926 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-test-2:19.1.1-314.g086c737f.el9.x86_64 2024-09-12T12:54:47.926 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-volume-2:19.1.1-314.g086c737f.el9.noarch 2024-09-12T12:54:47.926 INFO:teuthology.orchestra.run.smithi130.stdout: cephadm-2:19.1.1-314.g086c737f.el9.noarch 2024-09-12T12:54:47.926 INFO:teuthology.orchestra.run.smithi130.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-12T12:54:47.926 INFO:teuthology.orchestra.run.smithi130.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-12T12:54:47.927 INFO:teuthology.orchestra.run.smithi130.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-12T12:54:47.927 INFO:teuthology.orchestra.run.smithi130.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-12T12:54:47.927 INFO:teuthology.orchestra.run.smithi130.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-12T12:54:47.927 INFO:teuthology.orchestra.run.smithi130.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-12T12:54:47.927 INFO:teuthology.orchestra.run.smithi130.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-12T12:54:47.927 INFO:teuthology.orchestra.run.smithi130.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-12T12:54:47.927 INFO:teuthology.orchestra.run.smithi130.stdout: libcephfs-devel-2:19.1.1-314.g086c737f.el9.x86_64 2024-09-12T12:54:47.927 INFO:teuthology.orchestra.run.smithi130.stdout: libcephfs2-2:19.1.1-314.g086c737f.el9.x86_64 2024-09-12T12:54:47.927 INFO:teuthology.orchestra.run.smithi130.stdout: libcephsqlite-2:19.1.1-314.g086c737f.el9.x86_64 2024-09-12T12:54:47.927 INFO:teuthology.orchestra.run.smithi130.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-12T12:54:47.927 INFO:teuthology.orchestra.run.smithi130.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-12T12:54:47.927 INFO:teuthology.orchestra.run.smithi130.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-12T12:54:47.927 INFO:teuthology.orchestra.run.smithi130.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-12T12:54:47.928 INFO:teuthology.orchestra.run.smithi130.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-12T12:54:47.928 INFO:teuthology.orchestra.run.smithi130.stdout: librados-devel-2:19.1.1-314.g086c737f.el9.x86_64 2024-09-12T12:54:47.928 INFO:teuthology.orchestra.run.smithi130.stdout: libradosstriper1-2:19.1.1-314.g086c737f.el9.x86_64 2024-09-12T12:54:47.928 INFO:teuthology.orchestra.run.smithi130.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-12T12:54:47.928 INFO:teuthology.orchestra.run.smithi130.stdout: librgw2-2:19.1.1-314.g086c737f.el9.x86_64 2024-09-12T12:54:47.928 INFO:teuthology.orchestra.run.smithi130.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-12T12:54:47.928 INFO:teuthology.orchestra.run.smithi130.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-12T12:54:47.928 INFO:teuthology.orchestra.run.smithi130.stdout: lua-5.4.4-4.el9.x86_64 2024-09-12T12:54:47.928 INFO:teuthology.orchestra.run.smithi130.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-12T12:54:47.928 INFO:teuthology.orchestra.run.smithi130.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-12T12:54:47.928 INFO:teuthology.orchestra.run.smithi130.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-12T12:54:47.928 INFO:teuthology.orchestra.run.smithi130.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-12T12:54:47.928 INFO:teuthology.orchestra.run.smithi130.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-12T12:54:47.929 INFO:teuthology.orchestra.run.smithi130.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-12T12:54:47.929 INFO:teuthology.orchestra.run.smithi130.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-12T12:54:47.929 INFO:teuthology.orchestra.run.smithi130.stdout: pv-1.6.20-1.el9.x86_64 2024-09-12T12:54:47.929 INFO:teuthology.orchestra.run.smithi130.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-12T12:54:47.929 INFO:teuthology.orchestra.run.smithi130.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-12T12:54:47.929 INFO:teuthology.orchestra.run.smithi130.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-12T12:54:47.929 INFO:teuthology.orchestra.run.smithi130.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-12T12:54:47.929 INFO:teuthology.orchestra.run.smithi130.stdout: python3-ceph-argparse-2:19.1.1-314.g086c737f.el9.x86_64 2024-09-12T12:54:47.929 INFO:teuthology.orchestra.run.smithi130.stdout: python3-ceph-common-2:19.1.1-314.g086c737f.el9.x86_64 2024-09-12T12:54:47.929 INFO:teuthology.orchestra.run.smithi130.stdout: python3-cephfs-2:19.1.1-314.g086c737f.el9.x86_64 2024-09-12T12:54:47.929 INFO:teuthology.orchestra.run.smithi130.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-12T12:54:47.929 INFO:teuthology.orchestra.run.smithi130.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-12T12:54:47.929 INFO:teuthology.orchestra.run.smithi130.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-12T12:54:47.930 INFO:teuthology.orchestra.run.smithi130.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-12T12:54:47.930 INFO:teuthology.orchestra.run.smithi130.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-12T12:54:47.930 INFO:teuthology.orchestra.run.smithi130.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-12T12:54:47.930 INFO:teuthology.orchestra.run.smithi130.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-12T12:54:47.930 INFO:teuthology.orchestra.run.smithi130.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-12T12:54:47.930 INFO:teuthology.orchestra.run.smithi130.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-12T12:54:47.930 INFO:teuthology.orchestra.run.smithi130.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-12T12:54:47.930 INFO:teuthology.orchestra.run.smithi130.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-12T12:54:47.930 INFO:teuthology.orchestra.run.smithi130.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-12T12:54:47.930 INFO:teuthology.orchestra.run.smithi130.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-12T12:54:47.930 INFO:teuthology.orchestra.run.smithi130.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-12T12:54:47.931 INFO:teuthology.orchestra.run.smithi130.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-12T12:54:47.931 INFO:teuthology.orchestra.run.smithi130.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-12T12:54:47.931 INFO:teuthology.orchestra.run.smithi130.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-12T12:54:47.931 INFO:teuthology.orchestra.run.smithi130.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-12T12:54:47.931 INFO:teuthology.orchestra.run.smithi130.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-12T12:54:47.931 INFO:teuthology.orchestra.run.smithi130.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-12T12:54:47.931 INFO:teuthology.orchestra.run.smithi130.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-12T12:54:47.931 INFO:teuthology.orchestra.run.smithi130.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-12T12:54:47.931 INFO:teuthology.orchestra.run.smithi130.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-12T12:54:47.931 INFO:teuthology.orchestra.run.smithi130.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-12T12:54:47.931 INFO:teuthology.orchestra.run.smithi130.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-12T12:54:47.931 INFO:teuthology.orchestra.run.smithi130.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-12T12:54:47.931 INFO:teuthology.orchestra.run.smithi130.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-12T12:54:47.931 INFO:teuthology.orchestra.run.smithi130.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-12T12:54:47.932 INFO:teuthology.orchestra.run.smithi130.stdout: python3-ply-3.11-14.el9.noarch 2024-09-12T12:54:47.932 INFO:teuthology.orchestra.run.smithi130.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-12T12:54:47.932 INFO:teuthology.orchestra.run.smithi130.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-12T12:54:47.932 INFO:teuthology.orchestra.run.smithi130.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-12T12:54:47.932 INFO:teuthology.orchestra.run.smithi130.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-12T12:54:47.932 INFO:teuthology.orchestra.run.smithi130.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-12T12:54:47.932 INFO:teuthology.orchestra.run.smithi130.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-12T12:54:47.932 INFO:teuthology.orchestra.run.smithi130.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-12T12:54:47.932 INFO:teuthology.orchestra.run.smithi130.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-12T12:54:47.932 INFO:teuthology.orchestra.run.smithi130.stdout: python3-rados-2:19.1.1-314.g086c737f.el9.x86_64 2024-09-12T12:54:47.932 INFO:teuthology.orchestra.run.smithi130.stdout: python3-rbd-2:19.1.1-314.g086c737f.el9.x86_64 2024-09-12T12:54:47.932 INFO:teuthology.orchestra.run.smithi130.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-12T12:54:47.933 INFO:teuthology.orchestra.run.smithi130.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-12T12:54:47.933 INFO:teuthology.orchestra.run.smithi130.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-12T12:54:47.933 INFO:teuthology.orchestra.run.smithi130.stdout: python3-rgw-2:19.1.1-314.g086c737f.el9.x86_64 2024-09-12T12:54:47.933 INFO:teuthology.orchestra.run.smithi130.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-12T12:54:47.933 INFO:teuthology.orchestra.run.smithi130.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-12T12:54:47.933 INFO:teuthology.orchestra.run.smithi130.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-12T12:54:47.933 INFO:teuthology.orchestra.run.smithi130.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-12T12:54:47.933 INFO:teuthology.orchestra.run.smithi130.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-12T12:54:47.933 INFO:teuthology.orchestra.run.smithi130.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-12T12:54:47.933 INFO:teuthology.orchestra.run.smithi130.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-12T12:54:47.933 INFO:teuthology.orchestra.run.smithi130.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-12T12:54:47.933 INFO:teuthology.orchestra.run.smithi130.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-12T12:54:47.933 INFO:teuthology.orchestra.run.smithi130.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-12T12:54:47.933 INFO:teuthology.orchestra.run.smithi130.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-12T12:54:47.934 INFO:teuthology.orchestra.run.smithi130.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-12T12:54:47.934 INFO:teuthology.orchestra.run.smithi130.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-12T12:54:47.934 INFO:teuthology.orchestra.run.smithi130.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-12T12:54:47.934 INFO:teuthology.orchestra.run.smithi130.stdout: rbd-fuse-2:19.1.1-314.g086c737f.el9.x86_64 2024-09-12T12:54:47.934 INFO:teuthology.orchestra.run.smithi130.stdout: rbd-mirror-2:19.1.1-314.g086c737f.el9.x86_64 2024-09-12T12:54:47.934 INFO:teuthology.orchestra.run.smithi130.stdout: rbd-nbd-2:19.1.1-314.g086c737f.el9.x86_64 2024-09-12T12:54:47.934 INFO:teuthology.orchestra.run.smithi130.stdout: re2-1:20211101-20.el9.x86_64 2024-09-12T12:54:47.934 INFO:teuthology.orchestra.run.smithi130.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-12T12:54:47.934 INFO:teuthology.orchestra.run.smithi130.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-12T12:54:47.935 INFO:teuthology.orchestra.run.smithi130.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-12T12:54:47.935 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-09-12T12:54:47.935 INFO:teuthology.orchestra.run.smithi130.stdout:Complete! 2024-09-12T12:54:48.445 DEBUG:teuthology.parallel:result is None 2024-09-12T12:54:50.410 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-2:19.1.1-314.g086c737f.el9.x86_64 1/129 2024-09-12T12:54:50.410 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-base-2:19.1.1-314.g086c737f.el9.x86_64 2/129 2024-09-12T12:54:50.410 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-common-2:19.1.1-314.g086c737f.el9.x86_64 3/129 2024-09-12T12:54:50.411 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-fuse-2:19.1.1-314.g086c737f.el9.x86_64 4/129 2024-09-12T12:54:50.411 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-314.g086c73 5/129 2024-09-12T12:54:50.411 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-mds-2:19.1.1-314.g086c737f.el9.x86_64 6/129 2024-09-12T12:54:50.411 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-mgr-2:19.1.1-314.g086c737f.el9.x86_64 7/129 2024-09-12T12:54:50.411 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-mon-2:19.1.1-314.g086c737f.el9.x86_64 8/129 2024-09-12T12:54:50.411 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-osd-2:19.1.1-314.g086c737f.el9.x86_64 9/129 2024-09-12T12:54:50.411 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-radosgw-2:19.1.1-314.g086c737f.el9.x86_64 10/129 2024-09-12T12:54:50.411 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-selinux-2:19.1.1-314.g086c737f.el9.x86_64 11/129 2024-09-12T12:54:50.411 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-test-2:19.1.1-314.g086c737f.el9.x86_64 12/129 2024-09-12T12:54:50.411 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libcephfs-devel-2:19.1.1-314.g086c737f.el9.x86_6 13/129 2024-09-12T12:54:50.411 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libcephfs2-2:19.1.1-314.g086c737f.el9.x86_64 14/129 2024-09-12T12:54:50.411 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libcephsqlite-2:19.1.1-314.g086c737f.el9.x86_64 15/129 2024-09-12T12:54:50.411 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : librados-devel-2:19.1.1-314.g086c737f.el9.x86_64 16/129 2024-09-12T12:54:50.412 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libradosstriper1-2:19.1.1-314.g086c737f.el9.x86_ 17/129 2024-09-12T12:54:50.412 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : librgw2-2:19.1.1-314.g086c737f.el9.x86_64 18/129 2024-09-12T12:54:50.412 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-ceph-argparse-2:19.1.1-314.g086c737f.el9 19/129 2024-09-12T12:54:50.412 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-ceph-common-2:19.1.1-314.g086c737f.el9.x 20/129 2024-09-12T12:54:50.412 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-cephfs-2:19.1.1-314.g086c737f.el9.x86_64 21/129 2024-09-12T12:54:50.412 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-rados-2:19.1.1-314.g086c737f.el9.x86_64 22/129 2024-09-12T12:54:50.412 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-rbd-2:19.1.1-314.g086c737f.el9.x86_64 23/129 2024-09-12T12:54:50.412 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-rgw-2:19.1.1-314.g086c737f.el9.x86_64 24/129 2024-09-12T12:54:50.412 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : rbd-fuse-2:19.1.1-314.g086c737f.el9.x86_64 25/129 2024-09-12T12:54:50.412 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : rbd-mirror-2:19.1.1-314.g086c737f.el9.x86_64 26/129 2024-09-12T12:54:50.412 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : rbd-nbd-2:19.1.1-314.g086c737f.el9.x86_64 27/129 2024-09-12T12:54:50.412 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-314.g086c737f.e 28/129 2024-09-12T12:54:50.422 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-314.g086c737f.el9.noar 29/129 2024-09-12T12:54:50.422 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-314.g086c737f.el9.no 30/129 2024-09-12T12:54:50.422 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-314.g086c 31/129 2024-09-12T12:54:50.422 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-314.g086c737f.el9 32/129 2024-09-12T12:54:50.422 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-mgr-rook-2:19.1.1-314.g086c737f.el9.noarch 33/129 2024-09-12T12:54:50.422 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-314.g086c737f.el 34/129 2024-09-12T12:54:50.422 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-volume-2:19.1.1-314.g086c737f.el9.noarch 35/129 2024-09-12T12:54:50.422 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : cephadm-2:19.1.1-314.g086c737f.el9.noarch 36/129 2024-09-12T12:54:50.422 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 37/129 2024-09-12T12:54:50.423 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 38/129 2024-09-12T12:54:50.423 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 39/129 2024-09-12T12:54:50.423 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 40/129 2024-09-12T12:54:50.423 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 41/129 2024-09-12T12:54:50.423 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-ply-3.11-14.el9.noarch 42/129 2024-09-12T12:54:50.423 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 43/129 2024-09-12T12:54:50.423 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 44/129 2024-09-12T12:54:50.423 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 45/129 2024-09-12T12:54:50.423 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 46/129 2024-09-12T12:54:50.423 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 47/129 2024-09-12T12:54:50.423 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 48/129 2024-09-12T12:54:50.423 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 49/129 2024-09-12T12:54:50.423 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 50/129 2024-09-12T12:54:50.424 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 51/129 2024-09-12T12:54:50.424 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 52/129 2024-09-12T12:54:50.424 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 53/129 2024-09-12T12:54:50.424 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : lua-5.4.4-4.el9.x86_64 54/129 2024-09-12T12:54:50.424 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 55/129 2024-09-12T12:54:50.424 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 56/129 2024-09-12T12:54:50.424 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 57/129 2024-09-12T12:54:50.424 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 58/129 2024-09-12T12:54:50.424 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 59/129 2024-09-12T12:54:50.424 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 60/129 2024-09-12T12:54:50.424 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 61/129 2024-09-12T12:54:50.424 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 62/129 2024-09-12T12:54:50.424 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 63/129 2024-09-12T12:54:50.425 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 64/129 2024-09-12T12:54:50.425 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 65/129 2024-09-12T12:54:50.425 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 66/129 2024-09-12T12:54:50.425 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 67/129 2024-09-12T12:54:50.425 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 68/129 2024-09-12T12:54:50.425 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 69/129 2024-09-12T12:54:50.425 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 70/129 2024-09-12T12:54:50.425 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 71/129 2024-09-12T12:54:50.425 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 72/129 2024-09-12T12:54:50.425 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 73/129 2024-09-12T12:54:50.425 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 74/129 2024-09-12T12:54:50.425 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 75/129 2024-09-12T12:54:50.425 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 76/129 2024-09-12T12:54:50.426 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 77/129 2024-09-12T12:54:50.426 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 78/129 2024-09-12T12:54:50.426 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 79/129 2024-09-12T12:54:50.426 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 80/129 2024-09-12T12:54:50.426 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 81/129 2024-09-12T12:54:50.426 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 82/129 2024-09-12T12:54:50.426 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 83/129 2024-09-12T12:54:50.426 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 84/129 2024-09-12T12:54:50.426 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 85/129 2024-09-12T12:54:50.426 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 86/129 2024-09-12T12:54:50.426 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 87/129 2024-09-12T12:54:50.426 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 88/129 2024-09-12T12:54:50.426 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 89/129 2024-09-12T12:54:50.427 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 90/129 2024-09-12T12:54:50.427 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 91/129 2024-09-12T12:54:50.427 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 92/129 2024-09-12T12:54:50.427 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : pv-1.6.20-1.el9.x86_64 93/129 2024-09-12T12:54:50.427 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 94/129 2024-09-12T12:54:50.427 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 95/129 2024-09-12T12:54:50.427 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 96/129 2024-09-12T12:54:50.427 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 97/129 2024-09-12T12:54:50.427 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 98/129 2024-09-12T12:54:50.427 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 99/129 2024-09-12T12:54:50.427 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 100/129 2024-09-12T12:54:50.428 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 101/129 2024-09-12T12:54:50.428 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 102/129 2024-09-12T12:54:50.428 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 103/129 2024-09-12T12:54:50.428 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 104/129 2024-09-12T12:54:50.428 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 105/129 2024-09-12T12:54:50.428 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 106/129 2024-09-12T12:54:50.428 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 107/129 2024-09-12T12:54:50.428 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 108/129 2024-09-12T12:54:50.428 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 109/129 2024-09-12T12:54:50.428 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 110/129 2024-09-12T12:54:50.429 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 111/129 2024-09-12T12:54:50.429 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 112/129 2024-09-12T12:54:50.429 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 113/129 2024-09-12T12:54:50.429 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 114/129 2024-09-12T12:54:50.429 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 115/129 2024-09-12T12:54:50.429 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 116/129 2024-09-12T12:54:50.429 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 117/129 2024-09-12T12:54:50.429 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 118/129 2024-09-12T12:54:50.429 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 119/129 2024-09-12T12:54:50.429 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 120/129 2024-09-12T12:54:50.429 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 121/129 2024-09-12T12:54:50.429 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 122/129 2024-09-12T12:54:50.429 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 123/129 2024-09-12T12:54:50.429 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : re2-1:20211101-20.el9.x86_64 124/129 2024-09-12T12:54:50.430 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 125/129 2024-09-12T12:54:50.430 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : librados2-2:19.1.1-314.g086c737f.el9.x86_64 126/129 2024-09-12T12:54:50.430 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 127/129 2024-09-12T12:54:50.430 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : librbd1-2:19.1.1-314.g086c737f.el9.x86_64 128/129 2024-09-12T12:54:52.962 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 129/129 2024-09-12T12:54:52.962 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-12T12:54:52.962 INFO:teuthology.orchestra.run.smithi060.stdout:Upgraded: 2024-09-12T12:54:52.962 INFO:teuthology.orchestra.run.smithi060.stdout: librados2-2:19.1.1-314.g086c737f.el9.x86_64 2024-09-12T12:54:52.962 INFO:teuthology.orchestra.run.smithi060.stdout: librbd1-2:19.1.1-314.g086c737f.el9.x86_64 2024-09-12T12:54:52.962 INFO:teuthology.orchestra.run.smithi060.stdout:Installed: 2024-09-12T12:54:52.962 INFO:teuthology.orchestra.run.smithi060.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-12T12:54:52.963 INFO:teuthology.orchestra.run.smithi060.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-12T12:54:52.963 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-2:19.1.1-314.g086c737f.el9.x86_64 2024-09-12T12:54:52.963 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-base-2:19.1.1-314.g086c737f.el9.x86_64 2024-09-12T12:54:52.963 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-common-2:19.1.1-314.g086c737f.el9.x86_64 2024-09-12T12:54:52.963 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-fuse-2:19.1.1-314.g086c737f.el9.x86_64 2024-09-12T12:54:52.963 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-grafana-dashboards-2:19.1.1-314.g086c737f.el9.noarch 2024-09-12T12:54:52.963 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-immutable-object-cache-2:19.1.1-314.g086c737f.el9.x86_64 2024-09-12T12:54:52.963 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mds-2:19.1.1-314.g086c737f.el9.x86_64 2024-09-12T12:54:52.963 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mgr-2:19.1.1-314.g086c737f.el9.x86_64 2024-09-12T12:54:52.963 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mgr-cephadm-2:19.1.1-314.g086c737f.el9.noarch 2024-09-12T12:54:52.963 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mgr-dashboard-2:19.1.1-314.g086c737f.el9.noarch 2024-09-12T12:54:52.963 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mgr-diskprediction-local-2:19.1.1-314.g086c737f.el9.noarch 2024-09-12T12:54:52.963 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mgr-modules-core-2:19.1.1-314.g086c737f.el9.noarch 2024-09-12T12:54:52.964 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mgr-rook-2:19.1.1-314.g086c737f.el9.noarch 2024-09-12T12:54:52.964 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mon-2:19.1.1-314.g086c737f.el9.x86_64 2024-09-12T12:54:52.964 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-osd-2:19.1.1-314.g086c737f.el9.x86_64 2024-09-12T12:54:52.964 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-prometheus-alerts-2:19.1.1-314.g086c737f.el9.noarch 2024-09-12T12:54:52.964 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-radosgw-2:19.1.1-314.g086c737f.el9.x86_64 2024-09-12T12:54:52.964 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-selinux-2:19.1.1-314.g086c737f.el9.x86_64 2024-09-12T12:54:52.964 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-test-2:19.1.1-314.g086c737f.el9.x86_64 2024-09-12T12:54:52.964 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-volume-2:19.1.1-314.g086c737f.el9.noarch 2024-09-12T12:54:52.964 INFO:teuthology.orchestra.run.smithi060.stdout: cephadm-2:19.1.1-314.g086c737f.el9.noarch 2024-09-12T12:54:52.964 INFO:teuthology.orchestra.run.smithi060.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-12T12:54:52.964 INFO:teuthology.orchestra.run.smithi060.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-12T12:54:52.964 INFO:teuthology.orchestra.run.smithi060.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-12T12:54:52.965 INFO:teuthology.orchestra.run.smithi060.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-12T12:54:52.965 INFO:teuthology.orchestra.run.smithi060.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-12T12:54:52.965 INFO:teuthology.orchestra.run.smithi060.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-12T12:54:52.965 INFO:teuthology.orchestra.run.smithi060.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-12T12:54:52.965 INFO:teuthology.orchestra.run.smithi060.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-12T12:54:52.965 INFO:teuthology.orchestra.run.smithi060.stdout: libcephfs-devel-2:19.1.1-314.g086c737f.el9.x86_64 2024-09-12T12:54:52.965 INFO:teuthology.orchestra.run.smithi060.stdout: libcephfs2-2:19.1.1-314.g086c737f.el9.x86_64 2024-09-12T12:54:52.965 INFO:teuthology.orchestra.run.smithi060.stdout: libcephsqlite-2:19.1.1-314.g086c737f.el9.x86_64 2024-09-12T12:54:52.965 INFO:teuthology.orchestra.run.smithi060.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-12T12:54:52.965 INFO:teuthology.orchestra.run.smithi060.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-12T12:54:52.965 INFO:teuthology.orchestra.run.smithi060.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-12T12:54:52.965 INFO:teuthology.orchestra.run.smithi060.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-12T12:54:52.966 INFO:teuthology.orchestra.run.smithi060.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-12T12:54:52.966 INFO:teuthology.orchestra.run.smithi060.stdout: librados-devel-2:19.1.1-314.g086c737f.el9.x86_64 2024-09-12T12:54:52.966 INFO:teuthology.orchestra.run.smithi060.stdout: libradosstriper1-2:19.1.1-314.g086c737f.el9.x86_64 2024-09-12T12:54:52.966 INFO:teuthology.orchestra.run.smithi060.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-12T12:54:52.966 INFO:teuthology.orchestra.run.smithi060.stdout: librgw2-2:19.1.1-314.g086c737f.el9.x86_64 2024-09-12T12:54:52.966 INFO:teuthology.orchestra.run.smithi060.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-12T12:54:52.966 INFO:teuthology.orchestra.run.smithi060.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-12T12:54:52.966 INFO:teuthology.orchestra.run.smithi060.stdout: lua-5.4.4-4.el9.x86_64 2024-09-12T12:54:52.966 INFO:teuthology.orchestra.run.smithi060.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-12T12:54:52.966 INFO:teuthology.orchestra.run.smithi060.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-12T12:54:52.966 INFO:teuthology.orchestra.run.smithi060.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-12T12:54:52.966 INFO:teuthology.orchestra.run.smithi060.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-12T12:54:52.966 INFO:teuthology.orchestra.run.smithi060.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-12T12:54:52.967 INFO:teuthology.orchestra.run.smithi060.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-12T12:54:52.967 INFO:teuthology.orchestra.run.smithi060.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-12T12:54:52.967 INFO:teuthology.orchestra.run.smithi060.stdout: pv-1.6.20-1.el9.x86_64 2024-09-12T12:54:52.967 INFO:teuthology.orchestra.run.smithi060.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-12T12:54:52.967 INFO:teuthology.orchestra.run.smithi060.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-12T12:54:52.967 INFO:teuthology.orchestra.run.smithi060.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-12T12:54:52.967 INFO:teuthology.orchestra.run.smithi060.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-12T12:54:52.967 INFO:teuthology.orchestra.run.smithi060.stdout: python3-ceph-argparse-2:19.1.1-314.g086c737f.el9.x86_64 2024-09-12T12:54:52.967 INFO:teuthology.orchestra.run.smithi060.stdout: python3-ceph-common-2:19.1.1-314.g086c737f.el9.x86_64 2024-09-12T12:54:52.967 INFO:teuthology.orchestra.run.smithi060.stdout: python3-cephfs-2:19.1.1-314.g086c737f.el9.x86_64 2024-09-12T12:54:52.967 INFO:teuthology.orchestra.run.smithi060.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-12T12:54:52.967 INFO:teuthology.orchestra.run.smithi060.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-12T12:54:52.967 INFO:teuthology.orchestra.run.smithi060.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-12T12:54:52.968 INFO:teuthology.orchestra.run.smithi060.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-12T12:54:52.968 INFO:teuthology.orchestra.run.smithi060.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-12T12:54:52.968 INFO:teuthology.orchestra.run.smithi060.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-12T12:54:52.968 INFO:teuthology.orchestra.run.smithi060.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-12T12:54:52.968 INFO:teuthology.orchestra.run.smithi060.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-12T12:54:52.968 INFO:teuthology.orchestra.run.smithi060.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-12T12:54:52.968 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-12T12:54:52.968 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-12T12:54:52.968 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-12T12:54:52.968 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-12T12:54:52.968 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-12T12:54:52.968 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-12T12:54:52.969 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-12T12:54:52.969 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-12T12:54:52.969 INFO:teuthology.orchestra.run.smithi060.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-12T12:54:52.969 INFO:teuthology.orchestra.run.smithi060.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-12T12:54:52.969 INFO:teuthology.orchestra.run.smithi060.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-12T12:54:52.969 INFO:teuthology.orchestra.run.smithi060.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-12T12:54:52.969 INFO:teuthology.orchestra.run.smithi060.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-12T12:54:52.969 INFO:teuthology.orchestra.run.smithi060.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-12T12:54:52.969 INFO:teuthology.orchestra.run.smithi060.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-12T12:54:52.969 INFO:teuthology.orchestra.run.smithi060.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-12T12:54:52.969 INFO:teuthology.orchestra.run.smithi060.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-12T12:54:52.969 INFO:teuthology.orchestra.run.smithi060.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-12T12:54:52.970 INFO:teuthology.orchestra.run.smithi060.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-12T12:54:52.970 INFO:teuthology.orchestra.run.smithi060.stdout: python3-ply-3.11-14.el9.noarch 2024-09-12T12:54:52.970 INFO:teuthology.orchestra.run.smithi060.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-12T12:54:52.970 INFO:teuthology.orchestra.run.smithi060.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-12T12:54:52.970 INFO:teuthology.orchestra.run.smithi060.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-12T12:54:52.970 INFO:teuthology.orchestra.run.smithi060.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-12T12:54:52.970 INFO:teuthology.orchestra.run.smithi060.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-12T12:54:52.970 INFO:teuthology.orchestra.run.smithi060.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-12T12:54:52.970 INFO:teuthology.orchestra.run.smithi060.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-12T12:54:52.970 INFO:teuthology.orchestra.run.smithi060.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-12T12:54:52.970 INFO:teuthology.orchestra.run.smithi060.stdout: python3-rados-2:19.1.1-314.g086c737f.el9.x86_64 2024-09-12T12:54:52.970 INFO:teuthology.orchestra.run.smithi060.stdout: python3-rbd-2:19.1.1-314.g086c737f.el9.x86_64 2024-09-12T12:54:52.970 INFO:teuthology.orchestra.run.smithi060.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-12T12:54:52.971 INFO:teuthology.orchestra.run.smithi060.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-12T12:54:52.971 INFO:teuthology.orchestra.run.smithi060.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-12T12:54:52.971 INFO:teuthology.orchestra.run.smithi060.stdout: python3-rgw-2:19.1.1-314.g086c737f.el9.x86_64 2024-09-12T12:54:52.971 INFO:teuthology.orchestra.run.smithi060.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-12T12:54:52.971 INFO:teuthology.orchestra.run.smithi060.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-12T12:54:52.971 INFO:teuthology.orchestra.run.smithi060.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-12T12:54:52.971 INFO:teuthology.orchestra.run.smithi060.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-12T12:54:52.971 INFO:teuthology.orchestra.run.smithi060.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-12T12:54:52.971 INFO:teuthology.orchestra.run.smithi060.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-12T12:54:52.971 INFO:teuthology.orchestra.run.smithi060.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-12T12:54:52.971 INFO:teuthology.orchestra.run.smithi060.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-12T12:54:52.971 INFO:teuthology.orchestra.run.smithi060.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-12T12:54:52.972 INFO:teuthology.orchestra.run.smithi060.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-12T12:54:52.972 INFO:teuthology.orchestra.run.smithi060.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-12T12:54:52.972 INFO:teuthology.orchestra.run.smithi060.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-12T12:54:52.972 INFO:teuthology.orchestra.run.smithi060.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-12T12:54:52.972 INFO:teuthology.orchestra.run.smithi060.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-12T12:54:52.972 INFO:teuthology.orchestra.run.smithi060.stdout: rbd-fuse-2:19.1.1-314.g086c737f.el9.x86_64 2024-09-12T12:54:52.972 INFO:teuthology.orchestra.run.smithi060.stdout: rbd-mirror-2:19.1.1-314.g086c737f.el9.x86_64 2024-09-12T12:54:52.972 INFO:teuthology.orchestra.run.smithi060.stdout: rbd-nbd-2:19.1.1-314.g086c737f.el9.x86_64 2024-09-12T12:54:52.972 INFO:teuthology.orchestra.run.smithi060.stdout: re2-1:20211101-20.el9.x86_64 2024-09-12T12:54:52.972 INFO:teuthology.orchestra.run.smithi060.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-12T12:54:52.972 INFO:teuthology.orchestra.run.smithi060.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-12T12:54:52.973 INFO:teuthology.orchestra.run.smithi060.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-12T12:54:52.973 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-12T12:54:52.973 INFO:teuthology.orchestra.run.smithi060.stdout:Complete! 2024-09-12T12:54:53.408 DEBUG:teuthology.parallel:result is None 2024-09-12T12:54:53.409 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=086c737fa75c22d521a3932c59b31c5e1d1c9ef4 2024-09-12T12:54:53.554 DEBUG:teuthology.orchestra.run.smithi060:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-12T12:54:53.599 INFO:teuthology.orchestra.run.smithi060.stdout:19.1.1-314.g086c737f.el9 2024-09-12T12:54:53.600 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-314.g086c737f.el9 2024-09-12T12:54:53.600 INFO:teuthology.task.install:The correct ceph version 19.1.1-314.g086c737f is installed. 2024-09-12T12:54:53.602 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=086c737fa75c22d521a3932c59b31c5e1d1c9ef4 2024-09-12T12:54:53.739 DEBUG:teuthology.orchestra.run.smithi130:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-12T12:54:53.788 INFO:teuthology.orchestra.run.smithi130.stdout:19.1.1-314.g086c737f.el9 2024-09-12T12:54:53.789 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-314.g086c737f.el9 2024-09-12T12:54:53.789 INFO:teuthology.task.install:The correct ceph version 19.1.1-314.g086c737f is installed. 2024-09-12T12:54:53.791 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-12T12:54:53.791 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-09-12T12:54:53.791 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-12T12:54:53.849 DEBUG:teuthology.orchestra.run.smithi130:> set -ex 2024-09-12T12:54:53.849 DEBUG:teuthology.orchestra.run.smithi130:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-12T12:54:53.912 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-12T12:54:53.912 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-09-12T12:54:53.913 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd of=/usr/bin/daemon-helper 2024-09-12T12:54:53.968 DEBUG:teuthology.orchestra.run.smithi060:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-12T12:54:54.063 DEBUG:teuthology.orchestra.run.smithi130:> set -ex 2024-09-12T12:54:54.063 DEBUG:teuthology.orchestra.run.smithi130:> sudo dd of=/usr/bin/daemon-helper 2024-09-12T12:54:54.122 DEBUG:teuthology.orchestra.run.smithi130:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-12T12:54:54.217 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-12T12:54:54.218 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-09-12T12:54:54.218 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-12T12:54:54.273 DEBUG:teuthology.orchestra.run.smithi060:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-12T12:54:54.369 DEBUG:teuthology.orchestra.run.smithi130:> set -ex 2024-09-12T12:54:54.370 DEBUG:teuthology.orchestra.run.smithi130:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-12T12:54:54.425 DEBUG:teuthology.orchestra.run.smithi130:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-12T12:54:54.520 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-12T12:54:54.521 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-09-12T12:54:54.521 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd of=/usr/bin/stdin-killer 2024-09-12T12:54:54.576 DEBUG:teuthology.orchestra.run.smithi060:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-12T12:54:54.671 DEBUG:teuthology.orchestra.run.smithi130:> set -ex 2024-09-12T12:54:54.671 DEBUG:teuthology.orchestra.run.smithi130:> sudo dd of=/usr/bin/stdin-killer 2024-09-12T12:54:54.727 DEBUG:teuthology.orchestra.run.smithi130:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-12T12:54:54.821 INFO:teuthology.run_tasks:Running task ceph... 2024-09-12T12:54:54.905 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-09-12T12:54:54.905 DEBUG:teuthology.orchestra.run.smithi060:> sudo chmod 777 /var/log/ceph 2024-09-12T12:54:54.909 DEBUG:teuthology.orchestra.run.smithi130:> sudo chmod 777 /var/log/ceph 2024-09-12T12:54:54.964 INFO:tasks.ceph:Disabling ceph logrotate... 2024-09-12T12:54:54.964 DEBUG:teuthology.orchestra.run.smithi060:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-12T12:54:54.968 DEBUG:teuthology.orchestra.run.smithi130:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-12T12:54:55.022 INFO:tasks.ceph:Creating extra log directories... 2024-09-12T12:54:55.022 DEBUG:teuthology.orchestra.run.smithi060:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-12T12:54:55.026 DEBUG:teuthology.orchestra.run.smithi130:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-12T12:54:55.091 INFO:tasks.ceph:Creating ceph cluster cluster1... 2024-09-12T12:54:55.091 INFO:tasks.ceph:config {'conf': {'client': {'rbd default map options': 'ms_mode=legacy,rxbounce'}, 'global': {'mon warn on pool no app': False, 'ms die on skipped message': False}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)'], 'cpu_profile': set(), 'cluster': 'cluster1', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-09-12T12:54:55.091 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/teuthology-2024-09-11_21:48:03-krbd-squid-testing-default-smithi/7901765', 'branch': 'squid', 'description': 'krbd/mirror/{bluestore-bitmap clusters/2-node conf install/ceph ms_mode$/{legacy-rxbounce} tasks/compare-mirror-images}', 'email': 'ceph-qa@ceph.com', 'first_in_suite': False, 'job_id': '7901765', 'kernel': {'flavor': 'default', 'kdb': 1, 'sha1': '2ca44a2f35ca8f36983aaaa46d49312239ef519a'}, 'last_in_suite': False, 'machine_type': 'smithi', 'meta': [{'desc': '2 ceph clusters with 1 mon, 1 mgr and 3 osd each'}], 'name': 'teuthology-2024-09-11_21:48:03-krbd-squid-testing-default-smithi', 'no_nested_subset': False, 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'squid'}, 'ceph': {'conf': {'client': {'rbd default map options': 'ms_mode=legacy,rxbounce'}, 'global': {'mon warn on pool no app': False, 'ms die on skipped message': False}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)'], 'sha1': '086c737fa75c22d521a3932c59b31c5e1d1c9ef4'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '086c737fa75c22d521a3932c59b31c5e1d1c9ef4'}, 'extra_system_packages': ['pv']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'squid', 'sha1': '086c737fa75c22d521a3932c59b31c5e1d1c9ef4'}}, 'owner': 'scheduled_teuthology@teuthology', 'priority': 100, 'repo': 'https://git.ceph.com/ceph.git', 'roles': [['cluster1.mon.a', 'cluster1.mgr.x', 'cluster1.osd.0', 'cluster1.osd.1', 'cluster1.osd.2'], ['cluster2.mon.a', 'cluster2.mgr.x', 'cluster2.osd.0', 'cluster2.osd.1', 'cluster2.osd.2', 'cluster1.client.mirror', 'cluster1.client.mirror.0', 'cluster2.client.mirror', 'cluster2.client.mirror.0']], 'seed': 6642, 'sha1': '086c737fa75c22d521a3932c59b31c5e1d1c9ef4', 'sleep_before_teardown': 0, 'subset': '2/4', 'suite': 'krbd', 'suite_branch': 'squid', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph_086c737fa75c22d521a3932c59b31c5e1d1c9ef4/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph.git', 'suite_sha1': '086c737fa75c22d521a3932c59b31c5e1d1c9ef4', 'targets': {'smithi060.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIFOvl+hCSxBzLOW+W0g8RdwY9HvLI1Bc6w8kIMW64qum4rkb3wrve2G3MNnVVKdcGrJpzRuajn03zqmcHad4HA=', 'smithi130.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAP6S2zoN+SHzbLXCy7LSlxt94/nNlTvcxp6VIom6mCS2EdmiV6WkzCK7pkJ6lvDtvlYwqfk7CacbxjINJQq9RU='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'flavor': 'default', 'kdb': 1, 'sha1': '2ca44a2f35ca8f36983aaaa46d49312239ef519a'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': ['rbd-mirror'], 'flavor': 'default', 'sha1': '086c737fa75c22d521a3932c59b31c5e1d1c9ef4', 'extra_system_packages': ['pv']}}, {'ceph': {'cluster': 'cluster1', 'conf': {'client': {'rbd default map options': 'ms_mode=legacy,rxbounce'}, 'global': {'mon warn on pool no app': False, 'ms die on skipped message': False}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)'], 'sha1': '086c737fa75c22d521a3932c59b31c5e1d1c9ef4'}}, {'ceph': {'cluster': 'cluster2'}}, {'rbd-mirror': {'client': 'cluster1.client.mirror.0', 'thrash': False}}, {'rbd-mirror': {'client': 'cluster2.client.mirror.0', 'thrash': False}}, {'workunit': {'clients': {'cluster1.client.mirror': ['rbd/compare_mirror_images.sh']}, 'env': {'RBD_DEVICE_TYPE': 'krbd', 'RBD_MIRROR_USE_RBD_MIRROR': '1'}, 'timeout': '3h'}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a', 'timestamp': '2024-09-11_21:48:03', 'tube': 'smithi', 'user': 'teuthology', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856'} 2024-09-12T12:54:55.092 DEBUG:teuthology.orchestra.run.smithi060:> install -d -m0755 -- /home/ubuntu/cephtest/cluster1.data 2024-09-12T12:54:55.095 DEBUG:teuthology.orchestra.run.smithi130:> install -d -m0755 -- /home/ubuntu/cephtest/cluster1.data 2024-09-12T12:54:55.131 DEBUG:teuthology.orchestra.run.smithi060:> sudo install -d -m0777 -- /var/run/ceph 2024-09-12T12:54:55.171 DEBUG:teuthology.orchestra.run.smithi130:> sudo install -d -m0777 -- /var/run/ceph 2024-09-12T12:54:55.226 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-09-12T12:54:55.226 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/scratch_devs of=/dev/stdout 2024-09-12T12:54:55.261 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-12T12:54:55.262 DEBUG:teuthology.orchestra.run.smithi060:> stat /dev/vg_nvme/lv_1 2024-09-12T12:54:55.336 INFO:teuthology.orchestra.run.smithi060.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-12T12:54:55.337 INFO:teuthology.orchestra.run.smithi060.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T12:54:55.337 INFO:teuthology.orchestra.run.smithi060.stdout:Device: 6h/6d Inode: 632 Links: 1 2024-09-12T12:54:55.337 INFO:teuthology.orchestra.run.smithi060.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T12:54:55.337 INFO:teuthology.orchestra.run.smithi060.stdout:Context: system_u:object_r:device_t:s0 2024-09-12T12:54:55.337 INFO:teuthology.orchestra.run.smithi060.stdout:Access: 2024-09-12 12:54:45.618061402 +0000 2024-09-12T12:54:55.337 INFO:teuthology.orchestra.run.smithi060.stdout:Modify: 2024-09-12 12:52:52.192005953 +0000 2024-09-12T12:54:55.337 INFO:teuthology.orchestra.run.smithi060.stdout:Change: 2024-09-12 12:52:52.192005953 +0000 2024-09-12T12:54:55.337 INFO:teuthology.orchestra.run.smithi060.stdout: Birth: 2024-09-12 12:52:52.192005953 +0000 2024-09-12T12:54:55.337 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-12T12:54:55.430 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-09-12T12:54:55.430 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-09-12T12:54:55.431 INFO:teuthology.orchestra.run.smithi060.stderr:512 bytes copied, 0.000173774 s, 2.9 MB/s 2024-09-12T12:54:55.433 DEBUG:teuthology.orchestra.run.smithi060:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-12T12:54:55.470 DEBUG:teuthology.orchestra.run.smithi060:> stat /dev/vg_nvme/lv_2 2024-09-12T12:54:55.545 INFO:teuthology.orchestra.run.smithi060.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-12T12:54:55.545 INFO:teuthology.orchestra.run.smithi060.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T12:54:55.545 INFO:teuthology.orchestra.run.smithi060.stdout:Device: 6h/6d Inode: 653 Links: 1 2024-09-12T12:54:55.545 INFO:teuthology.orchestra.run.smithi060.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T12:54:55.545 INFO:teuthology.orchestra.run.smithi060.stdout:Context: system_u:object_r:device_t:s0 2024-09-12T12:54:55.545 INFO:teuthology.orchestra.run.smithi060.stdout:Access: 2024-09-12 12:54:45.619061392 +0000 2024-09-12T12:54:55.545 INFO:teuthology.orchestra.run.smithi060.stdout:Modify: 2024-09-12 12:52:52.651001579 +0000 2024-09-12T12:54:55.545 INFO:teuthology.orchestra.run.smithi060.stdout:Change: 2024-09-12 12:52:52.651001579 +0000 2024-09-12T12:54:55.546 INFO:teuthology.orchestra.run.smithi060.stdout: Birth: 2024-09-12 12:52:52.651001579 +0000 2024-09-12T12:54:55.546 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-12T12:54:55.637 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-09-12T12:54:55.637 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-09-12T12:54:55.637 INFO:teuthology.orchestra.run.smithi060.stderr:512 bytes copied, 0.000157857 s, 3.2 MB/s 2024-09-12T12:54:55.639 DEBUG:teuthology.orchestra.run.smithi060:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-12T12:54:55.676 DEBUG:teuthology.orchestra.run.smithi060:> stat /dev/vg_nvme/lv_3 2024-09-12T12:54:55.751 INFO:teuthology.orchestra.run.smithi060.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-12T12:54:55.751 INFO:teuthology.orchestra.run.smithi060.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T12:54:55.751 INFO:teuthology.orchestra.run.smithi060.stdout:Device: 6h/6d Inode: 671 Links: 1 2024-09-12T12:54:55.751 INFO:teuthology.orchestra.run.smithi060.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T12:54:55.751 INFO:teuthology.orchestra.run.smithi060.stdout:Context: system_u:object_r:device_t:s0 2024-09-12T12:54:55.751 INFO:teuthology.orchestra.run.smithi060.stdout:Access: 2024-09-12 12:54:45.619061392 +0000 2024-09-12T12:54:55.752 INFO:teuthology.orchestra.run.smithi060.stdout:Modify: 2024-09-12 12:52:53.090997385 +0000 2024-09-12T12:54:55.752 INFO:teuthology.orchestra.run.smithi060.stdout:Change: 2024-09-12 12:52:53.090997385 +0000 2024-09-12T12:54:55.752 INFO:teuthology.orchestra.run.smithi060.stdout: Birth: 2024-09-12 12:52:53.090997385 +0000 2024-09-12T12:54:55.752 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-12T12:54:55.843 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-09-12T12:54:55.844 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-09-12T12:54:55.844 INFO:teuthology.orchestra.run.smithi060.stderr:512 bytes copied, 0.000178129 s, 2.9 MB/s 2024-09-12T12:54:55.846 DEBUG:teuthology.orchestra.run.smithi060:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-12T12:54:55.882 DEBUG:teuthology.orchestra.run.smithi060:> stat /dev/vg_nvme/lv_4 2024-09-12T12:54:55.957 INFO:teuthology.orchestra.run.smithi060.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-12T12:54:55.957 INFO:teuthology.orchestra.run.smithi060.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T12:54:55.957 INFO:teuthology.orchestra.run.smithi060.stdout:Device: 6h/6d Inode: 689 Links: 1 2024-09-12T12:54:55.957 INFO:teuthology.orchestra.run.smithi060.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T12:54:55.957 INFO:teuthology.orchestra.run.smithi060.stdout:Context: system_u:object_r:device_t:s0 2024-09-12T12:54:55.957 INFO:teuthology.orchestra.run.smithi060.stdout:Access: 2024-09-12 12:54:45.620061382 +0000 2024-09-12T12:54:55.957 INFO:teuthology.orchestra.run.smithi060.stdout:Modify: 2024-09-12 12:52:53.568992829 +0000 2024-09-12T12:54:55.957 INFO:teuthology.orchestra.run.smithi060.stdout:Change: 2024-09-12 12:52:53.568992829 +0000 2024-09-12T12:54:55.957 INFO:teuthology.orchestra.run.smithi060.stdout: Birth: 2024-09-12 12:52:53.568992829 +0000 2024-09-12T12:54:55.958 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-12T12:54:56.050 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-09-12T12:54:56.050 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-09-12T12:54:56.050 INFO:teuthology.orchestra.run.smithi060.stderr:512 bytes copied, 0.000170647 s, 3.0 MB/s 2024-09-12T12:54:56.052 DEBUG:teuthology.orchestra.run.smithi060:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-12T12:54:56.089 INFO:tasks.ceph:osd dev map: {'cluster1.osd.0': '/dev/vg_nvme/lv_1', 'cluster1.osd.1': '/dev/vg_nvme/lv_2', 'cluster1.osd.2': '/dev/vg_nvme/lv_3'} 2024-09-12T12:54:56.089 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi060.front.sepia.ceph.com'): {'cluster1.osd.0': '/dev/vg_nvme/lv_1', 'cluster1.osd.1': '/dev/vg_nvme/lv_2', 'cluster1.osd.2': '/dev/vg_nvme/lv_3'}} 2024-09-12T12:54:56.089 INFO:tasks.ceph:Generating config... 2024-09-12T12:54:56.090 INFO:tasks.ceph:[client] rbd default map options = ms_mode=legacy,rxbounce 2024-09-12T12:54:56.091 INFO:tasks.ceph:[global] mon warn on pool no app = False 2024-09-12T12:54:56.091 INFO:tasks.ceph:[global] ms die on skipped message = False 2024-09-12T12:54:56.091 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-09-12T12:54:56.091 INFO:tasks.ceph:[mgr] debug ms = 1 2024-09-12T12:54:56.091 INFO:tasks.ceph:[mon] debug mon = 20 2024-09-12T12:54:56.091 INFO:tasks.ceph:[mon] debug ms = 1 2024-09-12T12:54:56.091 INFO:tasks.ceph:[mon] debug paxos = 20 2024-09-12T12:54:56.091 INFO:tasks.ceph:[osd] bdev async discard = True 2024-09-12T12:54:56.091 INFO:tasks.ceph:[osd] bdev enable discard = True 2024-09-12T12:54:56.091 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2024-09-12T12:54:56.091 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-09-12T12:54:56.091 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-09-12T12:54:56.091 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-09-12T12:54:56.092 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-09-12T12:54:56.092 INFO:tasks.ceph:[osd] debug ms = 1 2024-09-12T12:54:56.092 INFO:tasks.ceph:[osd] debug osd = 20 2024-09-12T12:54:56.092 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-09-12T12:54:56.092 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-09-12T12:54:56.092 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-09-12T12:54:56.092 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-09-12T12:54:56.092 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-09-12T12:54:56.092 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-09-12T12:54:56.092 INFO:tasks.ceph:Setting up cluster1.mon.a... 2024-09-12T12:54:56.092 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/cluster1.keyring 2024-09-12T12:54:56.218 INFO:teuthology.orchestra.run.smithi060.stdout:creating /etc/ceph/cluster1.keyring 2024-09-12T12:54:56.225 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/cluster1.keyring 2024-09-12T12:54:56.310 DEBUG:teuthology.orchestra.run.smithi060:> sudo chmod 0644 /etc/ceph/cluster1.keyring 2024-09-12T12:54:56.365 DEBUG:tasks.ceph:Ceph mon addresses: [('cluster1.mon.a', '172.21.15.60')] 2024-09-12T12:54:56.366 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.60', 'mon warn on pool no app': False, 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false'}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'rbd default map options': 'ms_mode=legacy,rxbounce'}, 'mon.a': {}} 2024-09-12T12:54:56.366 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-09-12T12:54:56.367 DEBUG:teuthology.orchestra.run.smithi060:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-09-12T12:54:56.401 DEBUG:teuthology.orchestra.run.smithi060:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.60 --print /home/ubuntu/cephtest/cluster1.monmap 2024-09-12T12:54:56.511 INFO:teuthology.orchestra.run.smithi060.stdout:monmaptool: monmap file /home/ubuntu/cephtest/cluster1.monmap 2024-09-12T12:54:56.512 INFO:teuthology.orchestra.run.smithi060.stdout:monmaptool: generated fsid 2a63e29a-4caf-4cc6-962c-13f8b91a9d2b 2024-09-12T12:54:56.512 INFO:teuthology.orchestra.run.smithi060.stdout:setting min_mon_release = quincy 2024-09-12T12:54:56.512 INFO:teuthology.orchestra.run.smithi060.stdout:epoch 0 2024-09-12T12:54:56.512 INFO:teuthology.orchestra.run.smithi060.stdout:fsid 2a63e29a-4caf-4cc6-962c-13f8b91a9d2b 2024-09-12T12:54:56.512 INFO:teuthology.orchestra.run.smithi060.stdout:last_changed 2024-09-12T12:54:56.512646+0000 2024-09-12T12:54:56.512 INFO:teuthology.orchestra.run.smithi060.stdout:created 2024-09-12T12:54:56.512646+0000 2024-09-12T12:54:56.512 INFO:teuthology.orchestra.run.smithi060.stdout:min_mon_release 17 (quincy) 2024-09-12T12:54:56.512 INFO:teuthology.orchestra.run.smithi060.stdout:election_strategy: 1 2024-09-12T12:54:56.512 INFO:teuthology.orchestra.run.smithi060.stdout:0: [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] mon.a 2024-09-12T12:54:56.512 INFO:teuthology.orchestra.run.smithi060.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/cluster1.monmap (1 monitors) 2024-09-12T12:54:56.514 DEBUG:teuthology.orchestra.run.smithi060:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-09-12T12:54:56.549 INFO:tasks.ceph:Writing /etc/ceph/cluster1.conf for FSID 2a63e29a-4caf-4cc6-962c-13f8b91a9d2b... 2024-09-12T12:54:56.549 DEBUG:teuthology.orchestra.run.smithi060:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/cluster1.conf && sudo chmod 0644 /etc/ceph/cluster1.conf > /dev/null 2024-09-12T12:54:56.593 DEBUG:teuthology.orchestra.run.smithi130:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/cluster1.conf && sudo chmod 0644 /etc/ceph/cluster1.conf > /dev/null 2024-09-12T12:54:56.686 INFO:teuthology.orchestra.run.smithi060.stdout:[global] 2024-09-12T12:54:56.686 INFO:teuthology.orchestra.run.smithi060.stdout: chdir = "" 2024-09-12T12:54:56.686 INFO:teuthology.orchestra.run.smithi060.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-12T12:54:56.686 INFO:teuthology.orchestra.run.smithi060.stdout: auth supported = cephx 2024-09-12T12:54:56.686 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-12T12:54:56.686 INFO:teuthology.orchestra.run.smithi060.stdout: filestore xattr use omap = true 2024-09-12T12:54:56.686 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-12T12:54:56.686 INFO:teuthology.orchestra.run.smithi060.stdout: mon clock drift allowed = 1.000 2024-09-12T12:54:56.686 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-12T12:54:56.686 INFO:teuthology.orchestra.run.smithi060.stdout: osd crush chooseleaf type = 0 2024-09-12T12:54:56.687 INFO:teuthology.orchestra.run.smithi060.stdout: auth debug = true 2024-09-12T12:54:56.687 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-12T12:54:56.687 INFO:teuthology.orchestra.run.smithi060.stdout: ms die on old message = true 2024-09-12T12:54:56.687 INFO:teuthology.orchestra.run.smithi060.stdout: ms die on bug = true 2024-09-12T12:54:56.687 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-12T12:54:56.687 INFO:teuthology.orchestra.run.smithi060.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-12T12:54:56.687 INFO:teuthology.orchestra.run.smithi060.stdout: mon pg warn max object skew = 0 2024-09-12T12:54:56.687 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-12T12:54:56.687 INFO:teuthology.orchestra.run.smithi060.stdout: # disable pg_autoscaler by default for new pools 2024-09-12T12:54:56.687 INFO:teuthology.orchestra.run.smithi060.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-12T12:54:56.687 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-12T12:54:56.687 INFO:teuthology.orchestra.run.smithi060.stdout: osd pool default size = 2 2024-09-12T12:54:56.687 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-12T12:54:56.687 INFO:teuthology.orchestra.run.smithi060.stdout: mon osd allow primary affinity = true 2024-09-12T12:54:56.688 INFO:teuthology.orchestra.run.smithi060.stdout: mon osd allow pg remap = true 2024-09-12T12:54:56.688 INFO:teuthology.orchestra.run.smithi060.stdout: mon warn on legacy crush tunables = false 2024-09-12T12:54:56.688 INFO:teuthology.orchestra.run.smithi060.stdout: mon warn on crush straw calc version zero = false 2024-09-12T12:54:56.688 INFO:teuthology.orchestra.run.smithi060.stdout: mon warn on no sortbitwise = false 2024-09-12T12:54:56.688 INFO:teuthology.orchestra.run.smithi060.stdout: mon warn on osd down out interval zero = false 2024-09-12T12:54:56.688 INFO:teuthology.orchestra.run.smithi060.stdout: mon warn on too few osds = false 2024-09-12T12:54:56.688 INFO:teuthology.orchestra.run.smithi060.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-12T12:54:56.688 INFO:teuthology.orchestra.run.smithi060.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-12T12:54:56.688 INFO:teuthology.orchestra.run.smithi060.stdout: mon_allow_pool_size_one = true 2024-09-12T12:54:56.688 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-12T12:54:56.688 INFO:teuthology.orchestra.run.smithi060.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-12T12:54:56.689 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-12T12:54:56.689 INFO:teuthology.orchestra.run.smithi060.stdout: osd default data pool replay window = 5 2024-09-12T12:54:56.689 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-12T12:54:56.689 INFO:teuthology.orchestra.run.smithi060.stdout: mon allow pool delete = true 2024-09-12T12:54:56.689 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-12T12:54:56.689 INFO:teuthology.orchestra.run.smithi060.stdout: mon cluster log file level = debug 2024-09-12T12:54:56.689 INFO:teuthology.orchestra.run.smithi060.stdout: debug asserts on shutdown = true 2024-09-12T12:54:56.689 INFO:teuthology.orchestra.run.smithi060.stdout: mon health detail to clog = false 2024-09-12T12:54:56.689 INFO:teuthology.orchestra.run.smithi060.stdout: mon host = 172.21.15.60 2024-09-12T12:54:56.689 INFO:teuthology.orchestra.run.smithi060.stdout: mon warn on pool no app = False 2024-09-12T12:54:56.689 INFO:teuthology.orchestra.run.smithi060.stdout: ms die on skipped message = False 2024-09-12T12:54:56.689 INFO:teuthology.orchestra.run.smithi060.stdout: fsid = 2a63e29a-4caf-4cc6-962c-13f8b91a9d2b 2024-09-12T12:54:56.689 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-12T12:54:56.690 INFO:teuthology.orchestra.run.smithi060.stdout:[osd] 2024-09-12T12:54:56.690 INFO:teuthology.orchestra.run.smithi060.stdout: osd journal size = 100 2024-09-12T12:54:56.690 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-12T12:54:56.690 INFO:teuthology.orchestra.run.smithi060.stdout: osd scrub load threshold = 5.0 2024-09-12T12:54:56.690 INFO:teuthology.orchestra.run.smithi060.stdout: osd scrub max interval = 600 2024-09-12T12:54:56.690 INFO:teuthology.orchestra.run.smithi060.stdout: osd mclock profile = high_recovery_ops 2024-09-12T12:54:56.690 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-12T12:54:56.690 INFO:teuthology.orchestra.run.smithi060.stdout: osd recover clone overlap = true 2024-09-12T12:54:56.690 INFO:teuthology.orchestra.run.smithi060.stdout: osd recovery max chunk = 1048576 2024-09-12T12:54:56.690 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-12T12:54:56.690 INFO:teuthology.orchestra.run.smithi060.stdout: osd debug shutdown = true 2024-09-12T12:54:56.690 INFO:teuthology.orchestra.run.smithi060.stdout: osd debug op order = true 2024-09-12T12:54:56.690 INFO:teuthology.orchestra.run.smithi060.stdout: osd debug verify stray on activate = true 2024-09-12T12:54:56.691 INFO:teuthology.orchestra.run.smithi060.stdout: osd debug trim objects = true 2024-09-12T12:54:56.691 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-12T12:54:56.691 INFO:teuthology.orchestra.run.smithi060.stdout: osd open classes on start = true 2024-09-12T12:54:56.691 INFO:teuthology.orchestra.run.smithi060.stdout: osd debug pg log writeout = true 2024-09-12T12:54:56.691 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-12T12:54:56.691 INFO:teuthology.orchestra.run.smithi060.stdout: osd deep scrub update digest min age = 30 2024-09-12T12:54:56.691 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-12T12:54:56.691 INFO:teuthology.orchestra.run.smithi060.stdout: osd map max advance = 10 2024-09-12T12:54:56.691 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-12T12:54:56.691 INFO:teuthology.orchestra.run.smithi060.stdout: journal zero on create = true 2024-09-12T12:54:56.691 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-12T12:54:56.691 INFO:teuthology.orchestra.run.smithi060.stdout: filestore ondisk finisher threads = 3 2024-09-12T12:54:56.691 INFO:teuthology.orchestra.run.smithi060.stdout: filestore apply finisher threads = 3 2024-09-12T12:54:56.692 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-12T12:54:56.692 INFO:teuthology.orchestra.run.smithi060.stdout: bdev debug aio = true 2024-09-12T12:54:56.692 INFO:teuthology.orchestra.run.smithi060.stdout: osd debug misdirected ops = true 2024-09-12T12:54:56.692 INFO:teuthology.orchestra.run.smithi060.stdout: bdev async discard = True 2024-09-12T12:54:56.692 INFO:teuthology.orchestra.run.smithi060.stdout: bdev enable discard = True 2024-09-12T12:54:56.692 INFO:teuthology.orchestra.run.smithi060.stdout: bluestore allocator = bitmap 2024-09-12T12:54:56.692 INFO:teuthology.orchestra.run.smithi060.stdout: bluestore block size = 96636764160 2024-09-12T12:54:56.692 INFO:teuthology.orchestra.run.smithi060.stdout: bluestore fsck on mount = True 2024-09-12T12:54:56.692 INFO:teuthology.orchestra.run.smithi060.stdout: debug bluefs = 1/20 2024-09-12T12:54:56.692 INFO:teuthology.orchestra.run.smithi060.stdout: debug bluestore = 1/20 2024-09-12T12:54:56.692 INFO:teuthology.orchestra.run.smithi060.stdout: debug ms = 1 2024-09-12T12:54:56.692 INFO:teuthology.orchestra.run.smithi060.stdout: debug osd = 20 2024-09-12T12:54:56.692 INFO:teuthology.orchestra.run.smithi060.stdout: debug rocksdb = 4/10 2024-09-12T12:54:56.693 INFO:teuthology.orchestra.run.smithi060.stdout: mon osd backfillfull_ratio = 0.85 2024-09-12T12:54:56.693 INFO:teuthology.orchestra.run.smithi060.stdout: mon osd full ratio = 0.9 2024-09-12T12:54:56.693 INFO:teuthology.orchestra.run.smithi060.stdout: mon osd nearfull ratio = 0.8 2024-09-12T12:54:56.693 INFO:teuthology.orchestra.run.smithi060.stdout: osd failsafe full ratio = 0.95 2024-09-12T12:54:56.693 INFO:teuthology.orchestra.run.smithi060.stdout: osd objectstore = bluestore 2024-09-12T12:54:56.693 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-12T12:54:56.693 INFO:teuthology.orchestra.run.smithi060.stdout:[mgr] 2024-09-12T12:54:56.693 INFO:teuthology.orchestra.run.smithi060.stdout: debug ms = 1 2024-09-12T12:54:56.693 INFO:teuthology.orchestra.run.smithi060.stdout: debug mgr = 20 2024-09-12T12:54:56.693 INFO:teuthology.orchestra.run.smithi060.stdout: debug mon = 20 2024-09-12T12:54:56.693 INFO:teuthology.orchestra.run.smithi060.stdout: debug auth = 20 2024-09-12T12:54:56.693 INFO:teuthology.orchestra.run.smithi060.stdout: mon reweight min pgs per osd = 4 2024-09-12T12:54:56.693 INFO:teuthology.orchestra.run.smithi060.stdout: mon reweight min bytes per osd = 10 2024-09-12T12:54:56.694 INFO:teuthology.orchestra.run.smithi060.stdout: mgr/telemetry/nag = false 2024-09-12T12:54:56.694 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-12T12:54:56.694 INFO:teuthology.orchestra.run.smithi060.stdout:[mon] 2024-09-12T12:54:56.694 INFO:teuthology.orchestra.run.smithi060.stdout: debug ms = 1 2024-09-12T12:54:56.694 INFO:teuthology.orchestra.run.smithi060.stdout: debug mon = 20 2024-09-12T12:54:56.694 INFO:teuthology.orchestra.run.smithi060.stdout: debug paxos = 20 2024-09-12T12:54:56.694 INFO:teuthology.orchestra.run.smithi060.stdout: debug auth = 20 2024-09-12T12:54:56.694 INFO:teuthology.orchestra.run.smithi060.stdout: mon data avail warn = 5 2024-09-12T12:54:56.694 INFO:teuthology.orchestra.run.smithi060.stdout: mon mgr mkfs grace = 240 2024-09-12T12:54:56.694 INFO:teuthology.orchestra.run.smithi060.stdout: mon reweight min pgs per osd = 4 2024-09-12T12:54:56.694 INFO:teuthology.orchestra.run.smithi060.stdout: mon osd reporter subtree level = osd 2024-09-12T12:54:56.694 INFO:teuthology.orchestra.run.smithi060.stdout: mon osd prime pg temp = true 2024-09-12T12:54:56.694 INFO:teuthology.orchestra.run.smithi060.stdout: mon reweight min bytes per osd = 10 2024-09-12T12:54:56.695 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-12T12:54:56.695 INFO:teuthology.orchestra.run.smithi060.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-12T12:54:56.695 INFO:teuthology.orchestra.run.smithi060.stdout: auth mon ticket ttl = 660 # 11m 2024-09-12T12:54:56.695 INFO:teuthology.orchestra.run.smithi060.stdout: auth service ticket ttl = 240 # 4m 2024-09-12T12:54:56.695 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-12T12:54:56.695 INFO:teuthology.orchestra.run.smithi060.stdout: # don't complain about insecure global_id in the test suite 2024-09-12T12:54:56.695 INFO:teuthology.orchestra.run.smithi060.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-12T12:54:56.695 INFO:teuthology.orchestra.run.smithi060.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-12T12:54:56.695 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-12T12:54:56.695 INFO:teuthology.orchestra.run.smithi060.stdout: # 1m isn't quite enough 2024-09-12T12:54:56.695 INFO:teuthology.orchestra.run.smithi060.stdout: mon_down_mkfs_grace = 2m 2024-09-12T12:54:56.695 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-12T12:54:56.695 INFO:teuthology.orchestra.run.smithi060.stdout: mon_warn_on_filestore_osds = false 2024-09-12T12:54:56.695 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-12T12:54:56.696 INFO:teuthology.orchestra.run.smithi060.stdout:[client] 2024-09-12T12:54:56.696 INFO:teuthology.orchestra.run.smithi060.stdout: rgw cache enabled = true 2024-09-12T12:54:56.696 INFO:teuthology.orchestra.run.smithi060.stdout: rgw enable ops log = true 2024-09-12T12:54:56.696 INFO:teuthology.orchestra.run.smithi060.stdout: rgw enable usage log = true 2024-09-12T12:54:56.696 INFO:teuthology.orchestra.run.smithi060.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-12T12:54:56.696 INFO:teuthology.orchestra.run.smithi060.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-12T12:54:56.696 INFO:teuthology.orchestra.run.smithi060.stdout: rbd default map options = "ms_mode=legacy,rxbounce" 2024-09-12T12:54:56.696 INFO:teuthology.orchestra.run.smithi060.stdout:[mon.a] 2024-09-12T12:54:56.698 INFO:teuthology.orchestra.run.smithi130.stdout:[global] 2024-09-12T12:54:56.698 INFO:teuthology.orchestra.run.smithi130.stdout: chdir = "" 2024-09-12T12:54:56.698 INFO:teuthology.orchestra.run.smithi130.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-12T12:54:56.698 INFO:teuthology.orchestra.run.smithi130.stdout: auth supported = cephx 2024-09-12T12:54:56.698 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-09-12T12:54:56.698 INFO:teuthology.orchestra.run.smithi130.stdout: filestore xattr use omap = true 2024-09-12T12:54:56.698 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-09-12T12:54:56.698 INFO:teuthology.orchestra.run.smithi130.stdout: mon clock drift allowed = 1.000 2024-09-12T12:54:56.698 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-09-12T12:54:56.698 INFO:teuthology.orchestra.run.smithi130.stdout: osd crush chooseleaf type = 0 2024-09-12T12:54:56.698 INFO:teuthology.orchestra.run.smithi130.stdout: auth debug = true 2024-09-12T12:54:56.698 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-09-12T12:54:56.699 INFO:teuthology.orchestra.run.smithi130.stdout: ms die on old message = true 2024-09-12T12:54:56.699 INFO:teuthology.orchestra.run.smithi130.stdout: ms die on bug = true 2024-09-12T12:54:56.699 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-09-12T12:54:56.699 INFO:teuthology.orchestra.run.smithi130.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-12T12:54:56.699 INFO:teuthology.orchestra.run.smithi130.stdout: mon pg warn max object skew = 0 2024-09-12T12:54:56.699 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-09-12T12:54:56.699 INFO:teuthology.orchestra.run.smithi130.stdout: # disable pg_autoscaler by default for new pools 2024-09-12T12:54:56.699 INFO:teuthology.orchestra.run.smithi130.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-12T12:54:56.699 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-09-12T12:54:56.699 INFO:teuthology.orchestra.run.smithi130.stdout: osd pool default size = 2 2024-09-12T12:54:56.699 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-09-12T12:54:56.699 INFO:teuthology.orchestra.run.smithi130.stdout: mon osd allow primary affinity = true 2024-09-12T12:54:56.699 INFO:teuthology.orchestra.run.smithi130.stdout: mon osd allow pg remap = true 2024-09-12T12:54:56.700 INFO:teuthology.orchestra.run.smithi130.stdout: mon warn on legacy crush tunables = false 2024-09-12T12:54:56.700 INFO:teuthology.orchestra.run.smithi130.stdout: mon warn on crush straw calc version zero = false 2024-09-12T12:54:56.700 INFO:teuthology.orchestra.run.smithi130.stdout: mon warn on no sortbitwise = false 2024-09-12T12:54:56.700 INFO:teuthology.orchestra.run.smithi130.stdout: mon warn on osd down out interval zero = false 2024-09-12T12:54:56.700 INFO:teuthology.orchestra.run.smithi130.stdout: mon warn on too few osds = false 2024-09-12T12:54:56.700 INFO:teuthology.orchestra.run.smithi130.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-12T12:54:56.700 INFO:teuthology.orchestra.run.smithi130.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-12T12:54:56.700 INFO:teuthology.orchestra.run.smithi130.stdout: mon_allow_pool_size_one = true 2024-09-12T12:54:56.700 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-09-12T12:54:56.700 INFO:teuthology.orchestra.run.smithi130.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-12T12:54:56.700 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-09-12T12:54:56.700 INFO:teuthology.orchestra.run.smithi130.stdout: osd default data pool replay window = 5 2024-09-12T12:54:56.701 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-09-12T12:54:56.701 INFO:teuthology.orchestra.run.smithi130.stdout: mon allow pool delete = true 2024-09-12T12:54:56.701 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-09-12T12:54:56.701 INFO:teuthology.orchestra.run.smithi130.stdout: mon cluster log file level = debug 2024-09-12T12:54:56.701 INFO:teuthology.orchestra.run.smithi130.stdout: debug asserts on shutdown = true 2024-09-12T12:54:56.701 INFO:teuthology.orchestra.run.smithi130.stdout: mon health detail to clog = false 2024-09-12T12:54:56.701 INFO:teuthology.orchestra.run.smithi130.stdout: mon host = 172.21.15.60 2024-09-12T12:54:56.701 INFO:teuthology.orchestra.run.smithi130.stdout: mon warn on pool no app = False 2024-09-12T12:54:56.701 INFO:teuthology.orchestra.run.smithi130.stdout: ms die on skipped message = False 2024-09-12T12:54:56.701 INFO:teuthology.orchestra.run.smithi130.stdout: fsid = 2a63e29a-4caf-4cc6-962c-13f8b91a9d2b 2024-09-12T12:54:56.701 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-09-12T12:54:56.701 INFO:teuthology.orchestra.run.smithi130.stdout:[osd] 2024-09-12T12:54:56.701 INFO:teuthology.orchestra.run.smithi130.stdout: osd journal size = 100 2024-09-12T12:54:56.702 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-09-12T12:54:56.702 INFO:teuthology.orchestra.run.smithi130.stdout: osd scrub load threshold = 5.0 2024-09-12T12:54:56.702 INFO:teuthology.orchestra.run.smithi130.stdout: osd scrub max interval = 600 2024-09-12T12:54:56.702 INFO:teuthology.orchestra.run.smithi130.stdout: osd mclock profile = high_recovery_ops 2024-09-12T12:54:56.702 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-09-12T12:54:56.702 INFO:teuthology.orchestra.run.smithi130.stdout: osd recover clone overlap = true 2024-09-12T12:54:56.702 INFO:teuthology.orchestra.run.smithi130.stdout: osd recovery max chunk = 1048576 2024-09-12T12:54:56.702 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-09-12T12:54:56.702 INFO:teuthology.orchestra.run.smithi130.stdout: osd debug shutdown = true 2024-09-12T12:54:56.702 INFO:teuthology.orchestra.run.smithi130.stdout: osd debug op order = true 2024-09-12T12:54:56.702 INFO:teuthology.orchestra.run.smithi130.stdout: osd debug verify stray on activate = true 2024-09-12T12:54:56.702 INFO:teuthology.orchestra.run.smithi130.stdout: osd debug trim objects = true 2024-09-12T12:54:56.702 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-09-12T12:54:56.703 INFO:teuthology.orchestra.run.smithi130.stdout: osd open classes on start = true 2024-09-12T12:54:56.703 INFO:teuthology.orchestra.run.smithi130.stdout: osd debug pg log writeout = true 2024-09-12T12:54:56.703 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-09-12T12:54:56.703 INFO:teuthology.orchestra.run.smithi130.stdout: osd deep scrub update digest min age = 30 2024-09-12T12:54:56.703 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-09-12T12:54:56.703 INFO:teuthology.orchestra.run.smithi130.stdout: osd map max advance = 10 2024-09-12T12:54:56.703 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-09-12T12:54:56.703 INFO:teuthology.orchestra.run.smithi130.stdout: journal zero on create = true 2024-09-12T12:54:56.703 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-09-12T12:54:56.703 INFO:teuthology.orchestra.run.smithi130.stdout: filestore ondisk finisher threads = 3 2024-09-12T12:54:56.703 INFO:teuthology.orchestra.run.smithi130.stdout: filestore apply finisher threads = 3 2024-09-12T12:54:56.703 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-09-12T12:54:56.703 INFO:teuthology.orchestra.run.smithi130.stdout: bdev debug aio = true 2024-09-12T12:54:56.703 INFO:teuthology.orchestra.run.smithi130.stdout: osd debug misdirected ops = true 2024-09-12T12:54:56.704 INFO:teuthology.orchestra.run.smithi130.stdout: bdev async discard = True 2024-09-12T12:54:56.704 INFO:teuthology.orchestra.run.smithi130.stdout: bdev enable discard = True 2024-09-12T12:54:56.704 INFO:teuthology.orchestra.run.smithi130.stdout: bluestore allocator = bitmap 2024-09-12T12:54:56.704 INFO:teuthology.orchestra.run.smithi130.stdout: bluestore block size = 96636764160 2024-09-12T12:54:56.704 INFO:teuthology.orchestra.run.smithi130.stdout: bluestore fsck on mount = True 2024-09-12T12:54:56.704 INFO:teuthology.orchestra.run.smithi130.stdout: debug bluefs = 1/20 2024-09-12T12:54:56.704 INFO:teuthology.orchestra.run.smithi130.stdout: debug bluestore = 1/20 2024-09-12T12:54:56.704 INFO:teuthology.orchestra.run.smithi130.stdout: debug ms = 1 2024-09-12T12:54:56.704 INFO:teuthology.orchestra.run.smithi130.stdout: debug osd = 20 2024-09-12T12:54:56.704 INFO:teuthology.orchestra.run.smithi130.stdout: debug rocksdb = 4/10 2024-09-12T12:54:56.704 INFO:teuthology.orchestra.run.smithi130.stdout: mon osd backfillfull_ratio = 0.85 2024-09-12T12:54:56.704 INFO:teuthology.orchestra.run.smithi130.stdout: mon osd full ratio = 0.9 2024-09-12T12:54:56.704 INFO:teuthology.orchestra.run.smithi130.stdout: mon osd nearfull ratio = 0.8 2024-09-12T12:54:56.704 INFO:teuthology.orchestra.run.smithi130.stdout: osd failsafe full ratio = 0.95 2024-09-12T12:54:56.705 INFO:teuthology.orchestra.run.smithi130.stdout: osd objectstore = bluestore 2024-09-12T12:54:56.705 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-09-12T12:54:56.705 INFO:teuthology.orchestra.run.smithi130.stdout:[mgr] 2024-09-12T12:54:56.705 INFO:teuthology.orchestra.run.smithi130.stdout: debug ms = 1 2024-09-12T12:54:56.705 INFO:teuthology.orchestra.run.smithi130.stdout: debug mgr = 20 2024-09-12T12:54:56.705 INFO:teuthology.orchestra.run.smithi130.stdout: debug mon = 20 2024-09-12T12:54:56.705 INFO:teuthology.orchestra.run.smithi130.stdout: debug auth = 20 2024-09-12T12:54:56.705 INFO:teuthology.orchestra.run.smithi130.stdout: mon reweight min pgs per osd = 4 2024-09-12T12:54:56.705 INFO:teuthology.orchestra.run.smithi130.stdout: mon reweight min bytes per osd = 10 2024-09-12T12:54:56.705 INFO:teuthology.orchestra.run.smithi130.stdout: mgr/telemetry/nag = false 2024-09-12T12:54:56.705 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-09-12T12:54:56.705 INFO:teuthology.orchestra.run.smithi130.stdout:[mon] 2024-09-12T12:54:56.705 INFO:teuthology.orchestra.run.smithi130.stdout: debug ms = 1 2024-09-12T12:54:56.706 INFO:teuthology.orchestra.run.smithi130.stdout: debug mon = 20 2024-09-12T12:54:56.706 INFO:teuthology.orchestra.run.smithi130.stdout: debug paxos = 20 2024-09-12T12:54:56.706 INFO:teuthology.orchestra.run.smithi130.stdout: debug auth = 20 2024-09-12T12:54:56.706 INFO:teuthology.orchestra.run.smithi130.stdout: mon data avail warn = 5 2024-09-12T12:54:56.706 INFO:teuthology.orchestra.run.smithi130.stdout: mon mgr mkfs grace = 240 2024-09-12T12:54:56.706 INFO:teuthology.orchestra.run.smithi130.stdout: mon reweight min pgs per osd = 4 2024-09-12T12:54:56.706 INFO:teuthology.orchestra.run.smithi130.stdout: mon osd reporter subtree level = osd 2024-09-12T12:54:56.706 INFO:teuthology.orchestra.run.smithi130.stdout: mon osd prime pg temp = true 2024-09-12T12:54:56.706 INFO:teuthology.orchestra.run.smithi130.stdout: mon reweight min bytes per osd = 10 2024-09-12T12:54:56.706 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-09-12T12:54:56.706 INFO:teuthology.orchestra.run.smithi130.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-12T12:54:56.706 INFO:teuthology.orchestra.run.smithi130.stdout: auth mon ticket ttl = 660 # 11m 2024-09-12T12:54:56.706 INFO:teuthology.orchestra.run.smithi130.stdout: auth service ticket ttl = 240 # 4m 2024-09-12T12:54:56.707 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-09-12T12:54:56.707 INFO:teuthology.orchestra.run.smithi130.stdout: # don't complain about insecure global_id in the test suite 2024-09-12T12:54:56.707 INFO:teuthology.orchestra.run.smithi130.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-12T12:54:56.707 INFO:teuthology.orchestra.run.smithi130.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-12T12:54:56.707 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-09-12T12:54:56.707 INFO:teuthology.orchestra.run.smithi130.stdout: # 1m isn't quite enough 2024-09-12T12:54:56.707 INFO:teuthology.orchestra.run.smithi130.stdout: mon_down_mkfs_grace = 2m 2024-09-12T12:54:56.707 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-09-12T12:54:56.707 INFO:teuthology.orchestra.run.smithi130.stdout: mon_warn_on_filestore_osds = false 2024-09-12T12:54:56.707 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-09-12T12:54:56.707 INFO:teuthology.orchestra.run.smithi130.stdout:[client] 2024-09-12T12:54:56.707 INFO:teuthology.orchestra.run.smithi130.stdout: rgw cache enabled = true 2024-09-12T12:54:56.707 INFO:teuthology.orchestra.run.smithi130.stdout: rgw enable ops log = true 2024-09-12T12:54:56.707 INFO:teuthology.orchestra.run.smithi130.stdout: rgw enable usage log = true 2024-09-12T12:54:56.708 INFO:teuthology.orchestra.run.smithi130.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-12T12:54:56.708 INFO:teuthology.orchestra.run.smithi130.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-12T12:54:56.708 INFO:teuthology.orchestra.run.smithi130.stdout: rbd default map options = "ms_mode=legacy,rxbounce" 2024-09-12T12:54:56.708 INFO:teuthology.orchestra.run.smithi130.stdout:[mon.a] 2024-09-12T12:54:56.724 INFO:tasks.ceph:Creating admin key on cluster1.mon.a... 2024-09-12T12:54:56.724 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/cluster1.keyring 2024-09-12T12:54:56.821 INFO:tasks.ceph:Copying monmap to all nodes... 2024-09-12T12:54:56.821 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-09-12T12:54:56.821 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/etc/ceph/cluster1.keyring of=/dev/stdout 2024-09-12T12:54:56.857 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-09-12T12:54:56.857 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/home/ubuntu/cephtest/cluster1.monmap of=/dev/stdout 2024-09-12T12:54:56.932 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi060.front.sepia.ceph.com 2024-09-12T12:54:56.932 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-09-12T12:54:56.932 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd of=/etc/ceph/cluster1.keyring 2024-09-12T12:54:56.932 DEBUG:teuthology.orchestra.run.smithi060:> sudo chmod 0644 /etc/ceph/cluster1.keyring 2024-09-12T12:54:57.050 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-09-12T12:54:57.051 DEBUG:teuthology.orchestra.run.smithi060:> dd of=/home/ubuntu/cephtest/cluster1.monmap 2024-09-12T12:54:57.126 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi130.front.sepia.ceph.com 2024-09-12T12:54:57.126 DEBUG:teuthology.orchestra.run.smithi130:> set -ex 2024-09-12T12:54:57.126 DEBUG:teuthology.orchestra.run.smithi130:> sudo dd of=/etc/ceph/cluster1.keyring 2024-09-12T12:54:57.126 DEBUG:teuthology.orchestra.run.smithi130:> sudo chmod 0644 /etc/ceph/cluster1.keyring 2024-09-12T12:54:57.206 DEBUG:teuthology.orchestra.run.smithi130:> set -ex 2024-09-12T12:54:57.206 DEBUG:teuthology.orchestra.run.smithi130:> dd of=/home/ubuntu/cephtest/cluster1.monmap 2024-09-12T12:54:57.282 INFO:tasks.ceph:Setting up mon nodes... 2024-09-12T12:54:57.282 INFO:tasks.ceph:Setting up mgr nodes... 2024-09-12T12:54:57.282 DEBUG:teuthology.orchestra.run.smithi060:> sudo mkdir -p /var/lib/ceph/mgr/cluster1-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/cluster1-x/keyring 2024-09-12T12:54:57.386 INFO:teuthology.orchestra.run.smithi060.stdout:creating /var/lib/ceph/mgr/cluster1-x/keyring 2024-09-12T12:54:57.392 INFO:tasks.ceph:Setting up mds nodes... 2024-09-12T12:54:57.392 INFO:tasks.ceph_client:Setting up client nodes... 2024-09-12T12:54:57.392 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.mirror /etc/ceph/cluster1.client.mirror.keyring && sudo chmod 0644 /etc/ceph/cluster1.client.mirror.keyring 2024-09-12T12:54:57.488 INFO:teuthology.orchestra.run.smithi130.stdout:creating /etc/ceph/cluster1.client.mirror.keyring 2024-09-12T12:54:57.517 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.mirror.0 /etc/ceph/cluster1.client.mirror.0.keyring && sudo chmod 0644 /etc/ceph/cluster1.client.mirror.0.keyring 2024-09-12T12:54:57.596 INFO:teuthology.orchestra.run.smithi130.stdout:creating /etc/ceph/cluster1.client.mirror.0.keyring 2024-09-12T12:54:57.623 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-09-12T12:54:57.623 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi060.front.sepia.ceph.com'): {'cluster1.osd.0': '/dev/vg_nvme/lv_1', 'cluster1.osd.1': '/dev/vg_nvme/lv_2', 'cluster1.osd.2': '/dev/vg_nvme/lv_3'}} 2024-09-12T12:54:57.624 DEBUG:teuthology.orchestra.run.smithi060:> sudo mkdir -p /var/lib/ceph/osd/cluster1-0 2024-09-12T12:54:57.680 INFO:tasks.ceph:roles_to_devs: {'cluster1.osd.0': '/dev/vg_nvme/lv_1', 'cluster1.osd.1': '/dev/vg_nvme/lv_2', 'cluster1.osd.2': '/dev/vg_nvme/lv_3'} 2024-09-12T12:54:57.680 INFO:tasks.ceph:role: cluster1.osd.0 2024-09-12T12:54:57.680 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi060.front.sepia.ceph.com 2024-09-12T12:54:57.680 DEBUG:teuthology.orchestra.run.smithi060:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-12T12:54:57.735 INFO:teuthology.orchestra.run.smithi060.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-12T12:54:57.735 INFO:teuthology.orchestra.run.smithi060.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-12T12:54:57.735 INFO:teuthology.orchestra.run.smithi060.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-12T12:54:57.735 INFO:teuthology.orchestra.run.smithi060.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-12T12:54:57.735 INFO:teuthology.orchestra.run.smithi060.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-12T12:54:57.735 INFO:teuthology.orchestra.run.smithi060.stdout: = sunit=0 swidth=0 blks 2024-09-12T12:54:57.735 INFO:teuthology.orchestra.run.smithi060.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-12T12:54:57.735 INFO:teuthology.orchestra.run.smithi060.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-12T12:54:57.735 INFO:teuthology.orchestra.run.smithi060.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-12T12:54:57.735 INFO:teuthology.orchestra.run.smithi060.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-12T12:54:57.771 INFO:teuthology.orchestra.run.smithi060.stdout:Discarding blocks...Done. 2024-09-12T12:54:57.774 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi060.front.sepia.ceph.com -o noatime 2024-09-12T12:54:57.775 DEBUG:teuthology.orchestra.run.smithi060:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/cluster1-0 2024-09-12T12:54:57.856 DEBUG:teuthology.orchestra.run.smithi060:> sudo /sbin/restorecon /var/lib/ceph/osd/cluster1-0 2024-09-12T12:54:57.917 DEBUG:teuthology.orchestra.run.smithi060:> sudo mkdir -p /var/lib/ceph/osd/cluster1-1 2024-09-12T12:54:57.973 INFO:tasks.ceph:roles_to_devs: {'cluster1.osd.0': '/dev/vg_nvme/lv_1', 'cluster1.osd.1': '/dev/vg_nvme/lv_2', 'cluster1.osd.2': '/dev/vg_nvme/lv_3'} 2024-09-12T12:54:57.973 INFO:tasks.ceph:role: cluster1.osd.1 2024-09-12T12:54:57.973 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi060.front.sepia.ceph.com 2024-09-12T12:54:57.974 DEBUG:teuthology.orchestra.run.smithi060:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-12T12:54:58.028 INFO:teuthology.orchestra.run.smithi060.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-12T12:54:58.028 INFO:teuthology.orchestra.run.smithi060.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-12T12:54:58.028 INFO:teuthology.orchestra.run.smithi060.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-12T12:54:58.028 INFO:teuthology.orchestra.run.smithi060.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-12T12:54:58.029 INFO:teuthology.orchestra.run.smithi060.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-12T12:54:58.029 INFO:teuthology.orchestra.run.smithi060.stdout: = sunit=0 swidth=0 blks 2024-09-12T12:54:58.029 INFO:teuthology.orchestra.run.smithi060.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-12T12:54:58.029 INFO:teuthology.orchestra.run.smithi060.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-12T12:54:58.029 INFO:teuthology.orchestra.run.smithi060.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-12T12:54:58.029 INFO:teuthology.orchestra.run.smithi060.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-12T12:54:58.063 INFO:teuthology.orchestra.run.smithi060.stdout:Discarding blocks...Done. 2024-09-12T12:54:58.067 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi060.front.sepia.ceph.com -o noatime 2024-09-12T12:54:58.067 DEBUG:teuthology.orchestra.run.smithi060:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/cluster1-1 2024-09-12T12:54:58.147 DEBUG:teuthology.orchestra.run.smithi060:> sudo /sbin/restorecon /var/lib/ceph/osd/cluster1-1 2024-09-12T12:54:58.209 DEBUG:teuthology.orchestra.run.smithi060:> sudo mkdir -p /var/lib/ceph/osd/cluster1-2 2024-09-12T12:54:58.265 INFO:tasks.ceph:roles_to_devs: {'cluster1.osd.0': '/dev/vg_nvme/lv_1', 'cluster1.osd.1': '/dev/vg_nvme/lv_2', 'cluster1.osd.2': '/dev/vg_nvme/lv_3'} 2024-09-12T12:54:58.265 INFO:tasks.ceph:role: cluster1.osd.2 2024-09-12T12:54:58.265 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi060.front.sepia.ceph.com 2024-09-12T12:54:58.265 DEBUG:teuthology.orchestra.run.smithi060:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-12T12:54:58.319 INFO:teuthology.orchestra.run.smithi060.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-12T12:54:58.320 INFO:teuthology.orchestra.run.smithi060.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-12T12:54:58.320 INFO:teuthology.orchestra.run.smithi060.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-12T12:54:58.320 INFO:teuthology.orchestra.run.smithi060.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-12T12:54:58.320 INFO:teuthology.orchestra.run.smithi060.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-12T12:54:58.320 INFO:teuthology.orchestra.run.smithi060.stdout: = sunit=0 swidth=0 blks 2024-09-12T12:54:58.320 INFO:teuthology.orchestra.run.smithi060.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-12T12:54:58.320 INFO:teuthology.orchestra.run.smithi060.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-12T12:54:58.320 INFO:teuthology.orchestra.run.smithi060.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-12T12:54:58.320 INFO:teuthology.orchestra.run.smithi060.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-12T12:54:58.352 INFO:teuthology.orchestra.run.smithi060.stdout:Discarding blocks...Done. 2024-09-12T12:54:58.357 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi060.front.sepia.ceph.com -o noatime 2024-09-12T12:54:58.357 DEBUG:teuthology.orchestra.run.smithi060:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/cluster1-2 2024-09-12T12:54:58.438 DEBUG:teuthology.orchestra.run.smithi060:> sudo /sbin/restorecon /var/lib/ceph/osd/cluster1-2 2024-09-12T12:54:58.550 DEBUG:teuthology.orchestra.run.smithi060:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster cluster1 --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/cluster1.monmap 2024-09-12T12:54:58.660 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-12T12:54:58.660+0000 7f08ddc64640 -1 auth: error reading file: /var/lib/ceph/osd/cluster1-0/keyring: can't open /var/lib/ceph/osd/cluster1-0/keyring: (2) No such file or directory 2024-09-12T12:54:58.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-12T12:54:58.661+0000 7f08ddc64640 -1 created new key in keyring /var/lib/ceph/osd/cluster1-0/keyring 2024-09-12T12:54:58.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-12T12:54:58.661+0000 7f08ddc64640 -1 bdev(0x55857fb51800 /var/lib/ceph/osd/cluster1-0/block) open stat got: (1) Operation not permitted 2024-09-12T12:54:58.661 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-12T12:54:58.661+0000 7f08ddc64640 -1 bluestore(/var/lib/ceph/osd/cluster1-0) _read_fsid unparsable uuid 2024-09-12T12:55:02.674 DEBUG:teuthology.orchestra.run.smithi060:> sudo chown -R ceph:ceph /var/lib/ceph/osd/cluster1-0 2024-09-12T12:55:02.736 DEBUG:teuthology.orchestra.run.smithi060:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster cluster1 --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/cluster1.monmap 2024-09-12T12:55:02.829 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-12T12:55:02.829+0000 7ff0a48d8640 -1 auth: error reading file: /var/lib/ceph/osd/cluster1-1/keyring: can't open /var/lib/ceph/osd/cluster1-1/keyring: (2) No such file or directory 2024-09-12T12:55:02.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-12T12:55:02.830+0000 7ff0a48d8640 -1 created new key in keyring /var/lib/ceph/osd/cluster1-1/keyring 2024-09-12T12:55:02.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-12T12:55:02.830+0000 7ff0a48d8640 -1 bdev(0x56206d245800 /var/lib/ceph/osd/cluster1-1/block) open stat got: (1) Operation not permitted 2024-09-12T12:55:02.830 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-12T12:55:02.830+0000 7ff0a48d8640 -1 bluestore(/var/lib/ceph/osd/cluster1-1) _read_fsid unparsable uuid 2024-09-12T12:55:06.834 DEBUG:teuthology.orchestra.run.smithi060:> sudo chown -R ceph:ceph /var/lib/ceph/osd/cluster1-1 2024-09-12T12:55:06.894 DEBUG:teuthology.orchestra.run.smithi060:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster cluster1 --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/cluster1.monmap 2024-09-12T12:55:06.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-12T12:55:06.987+0000 7fdc0b81c640 -1 auth: error reading file: /var/lib/ceph/osd/cluster1-2/keyring: can't open /var/lib/ceph/osd/cluster1-2/keyring: (2) No such file or directory 2024-09-12T12:55:06.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-12T12:55:06.987+0000 7fdc0b81c640 -1 created new key in keyring /var/lib/ceph/osd/cluster1-2/keyring 2024-09-12T12:55:06.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-12T12:55:06.987+0000 7fdc0b81c640 -1 bdev(0x5635d2a1f800 /var/lib/ceph/osd/cluster1-2/block) open stat got: (1) Operation not permitted 2024-09-12T12:55:06.988 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-12T12:55:06.988+0000 7fdc0b81c640 -1 bluestore(/var/lib/ceph/osd/cluster1-2) _read_fsid unparsable uuid 2024-09-12T12:55:10.985 DEBUG:teuthology.orchestra.run.smithi060:> sudo chown -R ceph:ceph /var/lib/ceph/osd/cluster1-2 2024-09-12T12:55:11.046 INFO:tasks.ceph:Reading keys from all nodes... 2024-09-12T12:55:11.046 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-09-12T12:55:11.046 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd if=/var/lib/ceph/mgr/cluster1-x/keyring of=/dev/stdout 2024-09-12T12:55:11.104 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-09-12T12:55:11.104 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd if=/var/lib/ceph/osd/cluster1-0/keyring of=/dev/stdout 2024-09-12T12:55:11.200 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-09-12T12:55:11.201 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd if=/var/lib/ceph/osd/cluster1-1/keyring of=/dev/stdout 2024-09-12T12:55:11.296 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-09-12T12:55:11.296 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd if=/var/lib/ceph/osd/cluster1-2/keyring of=/dev/stdout 2024-09-12T12:55:11.391 DEBUG:teuthology.orchestra.run.smithi130:> set -ex 2024-09-12T12:55:11.392 DEBUG:teuthology.orchestra.run.smithi130:> dd if=/etc/ceph/cluster1.client.mirror.keyring of=/dev/stdout 2024-09-12T12:55:11.431 DEBUG:teuthology.orchestra.run.smithi130:> set -ex 2024-09-12T12:55:11.432 DEBUG:teuthology.orchestra.run.smithi130:> dd if=/etc/ceph/cluster1.client.mirror.0.keyring of=/dev/stdout 2024-09-12T12:55:11.509 INFO:tasks.ceph:Adding keys to all mons... 2024-09-12T12:55:11.509 DEBUG:teuthology.orchestra.run.smithi060:> sudo tee -a /etc/ceph/cluster1.keyring 2024-09-12T12:55:11.562 INFO:teuthology.orchestra.run.smithi060.stdout:[mgr.x] 2024-09-12T12:55:11.562 INFO:teuthology.orchestra.run.smithi060.stdout: key = AQCh5OJmY8UXFxAAIsNqAoUXMd7kyhemuc2dRA== 2024-09-12T12:55:11.562 INFO:teuthology.orchestra.run.smithi060.stdout:[osd.0] 2024-09-12T12:55:11.562 INFO:teuthology.orchestra.run.smithi060.stdout: key = AQCi5OJmqFlwJxAAI0qEXjbpWLRGuWS1xNEWdQ== 2024-09-12T12:55:11.562 INFO:teuthology.orchestra.run.smithi060.stdout:[osd.1] 2024-09-12T12:55:11.562 INFO:teuthology.orchestra.run.smithi060.stdout: key = AQCm5OJmadGCMRAAMZ3aU4yPJOwMpD4PBH1x5w== 2024-09-12T12:55:11.562 INFO:teuthology.orchestra.run.smithi060.stdout:[osd.2] 2024-09-12T12:55:11.562 INFO:teuthology.orchestra.run.smithi060.stdout: key = AQCq5OJmyRHtOhAAE3EQHIYR5SCk/j9EXybYpw== 2024-09-12T12:55:11.562 INFO:teuthology.orchestra.run.smithi060.stdout:[client.mirror] 2024-09-12T12:55:11.562 INFO:teuthology.orchestra.run.smithi060.stdout: key = AQCh5OJmcZEvHRAA6gjbf9E3fCY4aTWAQAuGpg== 2024-09-12T12:55:11.562 INFO:teuthology.orchestra.run.smithi060.stdout:[client.mirror.0] 2024-09-12T12:55:11.563 INFO:teuthology.orchestra.run.smithi060.stdout: key = AQCh5OJmlrSeIxAAbsMU0s1Rlnr+Ga3ZYNSd2A== 2024-09-12T12:55:11.564 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster1.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-12T12:55:11.697 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster1.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-12T12:55:11.791 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster1.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-12T12:55:11.886 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster1.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-12T12:55:11.979 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster1.keyring --name=client.mirror --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-12T12:55:12.076 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster1.keyring --name=client.mirror.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-12T12:55:12.172 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-09-12T12:55:12.173 DEBUG:teuthology.orchestra.run.smithi060:> sudo mkdir -p /var/lib/ceph/mon/cluster1-a 2024-09-12T12:55:12.228 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster cluster1 --mkfs -i a --monmap /home/ubuntu/cephtest/cluster1.monmap --keyring /etc/ceph/cluster1.keyring 2024-09-12T12:55:12.335 DEBUG:teuthology.orchestra.run.smithi060:> sudo chown -R ceph:ceph /var/lib/ceph/mon/cluster1-a 2024-09-12T12:55:12.391 DEBUG:teuthology.orchestra.run.smithi060:> rm -- /home/ubuntu/cephtest/cluster1.monmap 2024-09-12T12:55:12.425 INFO:tasks.ceph:Starting mon daemons in cluster cluster1... 2024-09-12T12:55:12.425 INFO:tasks.ceph.cluster1.mon.a:Restarting daemon 2024-09-12T12:55:12.426 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster cluster1 -i a 2024-09-12T12:55:12.470 INFO:tasks.ceph.cluster1.mon.a:Started 2024-09-12T12:55:12.471 INFO:tasks.ceph:Starting mgr daemons in cluster cluster1... 2024-09-12T12:55:12.471 INFO:tasks.ceph.cluster1.mgr.x:Restarting daemon 2024-09-12T12:55:12.471 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster cluster1 -i x 2024-09-12T12:55:12.474 INFO:tasks.ceph.cluster1.mgr.x:Started 2024-09-12T12:55:12.474 INFO:tasks.ceph:Setting crush tunables to default 2024-09-12T12:55:12.475 DEBUG:teuthology.orchestra.run.smithi060:> sudo ceph --cluster cluster1 osd crush tunables default 2024-09-12T12:55:12.709 INFO:teuthology.orchestra.run.smithi060.stderr:adjusted tunables profile to default 2024-09-12T12:55:12.727 INFO:tasks.ceph:check_enable_crimson: False 2024-09-12T12:55:12.727 INFO:tasks.ceph:Starting osd daemons in cluster cluster1... 2024-09-12T12:55:12.727 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-09-12T12:55:12.727 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd if=/var/lib/ceph/osd/cluster1-0/fsid of=/dev/stdout 2024-09-12T12:55:12.783 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-09-12T12:55:12.783 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd if=/var/lib/ceph/osd/cluster1-1/fsid of=/dev/stdout 2024-09-12T12:55:12.879 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-09-12T12:55:12.879 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd if=/var/lib/ceph/osd/cluster1-2/fsid of=/dev/stdout 2024-09-12T12:55:12.930 INFO:tasks.ceph.cluster1.mgr.x.smithi060.stderr:2024-09-12T12:55:12.930+0000 7f11e6f2f040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-12T12:55:12.979 DEBUG:teuthology.orchestra.run.smithi060:> sudo ceph --cluster cluster1 osd new 8a419ad1-c999-4a27-8870-2d6e505f6398 0 2024-09-12T12:55:12.983 INFO:tasks.ceph.cluster1.mgr.x.smithi060.stderr:2024-09-12T12:55:12.983+0000 7f11e6f2f040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-12T12:55:13.165 INFO:tasks.ceph.cluster1.mgr.x.smithi060.stderr:2024-09-12T12:55:13.165+0000 7f11e6f2f040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-12T12:55:13.219 INFO:teuthology.orchestra.run.smithi060.stdout:0 2024-09-12T12:55:13.228 INFO:tasks.ceph.cluster1.mgr.x.smithi060.stderr:2024-09-12T12:55:13.228+0000 7f11e6f2f040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-12T12:55:13.235 DEBUG:teuthology.orchestra.run.smithi060:> sudo ceph --cluster cluster1 osd new 2b198f71-f471-4b42-9d29-f25f9f69a1e4 1 2024-09-12T12:55:13.428 INFO:tasks.ceph.cluster1.mgr.x.smithi060.stderr:2024-09-12T12:55:13.428+0000 7f11e6f2f040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-12T12:55:13.473 INFO:teuthology.orchestra.run.smithi060.stdout:1 2024-09-12T12:55:13.490 DEBUG:teuthology.orchestra.run.smithi060:> sudo ceph --cluster cluster1 osd new 39c37b7a-038d-49f6-b67c-78a015c7966c 2 2024-09-12T12:55:13.565 INFO:tasks.ceph.cluster1.mgr.x.smithi060.stderr:2024-09-12T12:55:13.565+0000 7f11e6f2f040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-12T12:55:13.729 INFO:teuthology.orchestra.run.smithi060.stdout:2 2024-09-12T12:55:13.745 INFO:tasks.ceph.cluster1.osd.0:Restarting daemon 2024-09-12T12:55:13.745 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster cluster1 -i 0 2024-09-12T12:55:13.749 INFO:tasks.ceph.cluster1.osd.0:Started 2024-09-12T12:55:13.749 INFO:tasks.ceph.cluster1.osd.1:Restarting daemon 2024-09-12T12:55:13.749 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster cluster1 -i 1 2024-09-12T12:55:13.753 INFO:tasks.ceph.cluster1.osd.1:Started 2024-09-12T12:55:13.753 INFO:tasks.ceph.cluster1.osd.2:Restarting daemon 2024-09-12T12:55:13.753 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster cluster1 -i 2 2024-09-12T12:55:13.757 INFO:tasks.ceph.cluster1.osd.2:Started 2024-09-12T12:55:13.757 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster1 osd dump --format=json 2024-09-12T12:55:13.982 INFO:tasks.ceph.cluster1.mgr.x.smithi060.stderr:2024-09-12T12:55:13.982+0000 7f11e6f2f040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-12T12:55:14.027 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-12T12:55:14.027 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":5,"fsid":"2a63e29a-4caf-4cc6-962c-13f8b91a9d2b","created":"2024-09-12T12:55:12.622095+0000","modified":"2024-09-12T12:55:13.728922+0000","last_up_change":"0.000000","last_in_change":"2024-09-12T12:55:13.728922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"8a419ad1-c999-4a27-8870-2d6e505f6398","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"2b198f71-f471-4b42-9d29-f25f9f69a1e4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"39c37b7a-038d-49f6-b67c-78a015c7966c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T12:55:14.041 INFO:tasks.ceph.cluster1.mgr.x.smithi060.stderr:2024-09-12T12:55:14.040+0000 7f11e6f2f040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-12T12:55:14.050 INFO:tasks.ceph.ceph_manager.cluster1:[] 2024-09-12T12:55:14.050 INFO:tasks.ceph:Waiting for OSDs to come up 2024-09-12T12:55:14.219 INFO:tasks.ceph.cluster1.mgr.x.smithi060.stderr:2024-09-12T12:55:14.218+0000 7f11e6f2f040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-12T12:55:14.357 DEBUG:teuthology.orchestra.run.smithi060:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster cluster1 osd dump --format=json 2024-09-12T12:55:14.567 INFO:teuthology.misc.health.smithi060.stdout: 2024-09-12T12:55:14.567 INFO:teuthology.misc.health.smithi060.stdout:{"epoch":5,"fsid":"2a63e29a-4caf-4cc6-962c-13f8b91a9d2b","created":"2024-09-12T12:55:12.622095+0000","modified":"2024-09-12T12:55:13.728922+0000","last_up_change":"0.000000","last_in_change":"2024-09-12T12:55:13.728922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"8a419ad1-c999-4a27-8870-2d6e505f6398","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"2b198f71-f471-4b42-9d29-f25f9f69a1e4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"39c37b7a-038d-49f6-b67c-78a015c7966c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T12:55:14.580 DEBUG:teuthology.misc:0 of 3 OSDs are up 2024-09-12T12:55:15.411 INFO:tasks.ceph.cluster1.mgr.x.smithi060.stderr:2024-09-12T12:55:15.411+0000 7f11e6f2f040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-12T12:55:15.476 INFO:tasks.ceph.cluster1.mgr.x.smithi060.stderr:2024-09-12T12:55:15.476+0000 7f11e6f2f040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-12T12:55:15.579 INFO:tasks.ceph.cluster1.osd.1.smithi060.stderr:2024-09-12T12:55:15.579+0000 7f3b2bf27640 -1 Falling back to public interface 2024-09-12T12:55:15.798 INFO:tasks.ceph.cluster1.osd.0.smithi060.stderr:2024-09-12T12:55:15.798+0000 7ff4de21e640 -1 Falling back to public interface 2024-09-12T12:55:15.849 INFO:tasks.ceph.cluster1.osd.2.smithi060.stderr:2024-09-12T12:55:15.849+0000 7f7eb33a8640 -1 Falling back to public interface 2024-09-12T12:55:15.938 INFO:tasks.ceph.cluster1.mgr.x.smithi060.stderr:2024-09-12T12:55:15.937+0000 7f11e6f2f040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-12T12:55:16.040 INFO:tasks.ceph.cluster1.mgr.x.smithi060.stderr:2024-09-12T12:55:16.040+0000 7f11e6f2f040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-12T12:55:16.167 INFO:tasks.ceph.cluster1.mgr.x.smithi060.stderr:2024-09-12T12:55:16.167+0000 7f11e6f2f040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-12T12:55:16.224 INFO:tasks.ceph.cluster1.mgr.x.smithi060.stderr:2024-09-12T12:55:16.224+0000 7f11e6f2f040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-12T12:55:16.357 INFO:tasks.ceph.cluster1.mgr.x.smithi060.stderr:2024-09-12T12:55:16.357+0000 7f11e6f2f040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-12T12:55:16.424 INFO:tasks.ceph.cluster1.mgr.x.smithi060.stderr:2024-09-12T12:55:16.423+0000 7f11e6f2f040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-12T12:55:16.481 INFO:tasks.ceph.cluster1.mgr.x.smithi060.stderr:2024-09-12T12:55:16.479+0000 7f11e6f2f040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-12T12:55:16.611 INFO:tasks.ceph.cluster1.mgr.x.smithi060.stderr:2024-09-12T12:55:16.611+0000 7f11e6f2f040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-12T12:55:16.673 INFO:tasks.ceph.cluster1.mgr.x.smithi060.stderr:2024-09-12T12:55:16.672+0000 7f11e6f2f040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-12T12:55:16.736 INFO:tasks.ceph.cluster1.mgr.x.smithi060.stderr:2024-09-12T12:55:16.736+0000 7f11e6f2f040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-12T12:55:16.818 INFO:tasks.ceph.cluster1.mgr.x.smithi060.stderr:2024-09-12T12:55:16.818+0000 7f11e6f2f040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-12T12:55:16.873 INFO:tasks.ceph.cluster1.mgr.x.smithi060.stderr:2024-09-12T12:55:16.872+0000 7f11e6f2f040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-12T12:55:16.927 INFO:tasks.ceph.cluster1.mgr.x.smithi060.stderr:2024-09-12T12:55:16.927+0000 7f11e6f2f040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-12T12:55:17.703 INFO:tasks.ceph.cluster1.mgr.x.smithi060.stderr:2024-09-12T12:55:17.703+0000 7f11e6f2f040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-12T12:55:18.716 INFO:tasks.ceph.cluster1.mgr.x.smithi060.stderr:2024-09-12T12:55:18.715+0000 7f1186800640 -1 mgr.server handle_report got status from non-daemon mon.a 2024-09-12T12:55:20.250 INFO:tasks.ceph.cluster1.osd.1.smithi060.stderr:2024-09-12T12:55:20.250+0000 7f3b2bf27640 -1 osd.1 0 log_to_monitors true 2024-09-12T12:55:20.417 INFO:tasks.ceph.cluster1.osd.0.smithi060.stderr:2024-09-12T12:55:20.416+0000 7ff4de21e640 -1 osd.0 0 log_to_monitors true 2024-09-12T12:55:20.580 INFO:tasks.ceph.cluster1.osd.2.smithi060.stderr:2024-09-12T12:55:20.579+0000 7f7eb33a8640 -1 osd.2 0 log_to_monitors true 2024-09-12T12:55:20.886 DEBUG:teuthology.orchestra.run.smithi060:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster cluster1 osd dump --format=json 2024-09-12T12:55:21.222 INFO:teuthology.misc.health.smithi060.stdout: 2024-09-12T12:55:21.223 INFO:teuthology.misc.health.smithi060.stdout:{"epoch":6,"fsid":"2a63e29a-4caf-4cc6-962c-13f8b91a9d2b","created":"2024-09-12T12:55:12.622095+0000","modified":"2024-09-12T12:55:20.714337+0000","last_up_change":"0.000000","last_in_change":"2024-09-12T12:55:13.728922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":3,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"8a419ad1-c999-4a27-8870-2d6e505f6398","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"2b198f71-f471-4b42-9d29-f25f9f69a1e4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"39c37b7a-038d-49f6-b67c-78a015c7966c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T12:55:21.237 DEBUG:teuthology.misc:0 of 3 OSDs are up 2024-09-12T12:55:23.126 INFO:tasks.ceph.cluster1.osd.1.smithi060.stderr:2024-09-12T12:55:23.126+0000 7f3b27800640 -1 osd.1 0 waiting for initial osdmap 2024-09-12T12:55:23.131 INFO:tasks.ceph.cluster1.osd.2.smithi060.stderr:2024-09-12T12:55:23.131+0000 7f7ead800640 -1 osd.2 0 waiting for initial osdmap 2024-09-12T12:55:23.134 INFO:tasks.ceph.cluster1.osd.2.smithi060.stderr:2024-09-12T12:55:23.134+0000 7f7ea4200640 -1 osd.2 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T12:55:23.136 INFO:tasks.ceph.cluster1.osd.1.smithi060.stderr:2024-09-12T12:55:23.136+0000 7f3b1ce00640 -1 osd.1 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T12:55:23.162 INFO:tasks.ceph.cluster1.osd.0.smithi060.stderr:2024-09-12T12:55:23.162+0000 7ff4d8200640 -1 osd.0 0 waiting for initial osdmap 2024-09-12T12:55:23.169 INFO:tasks.ceph.cluster1.osd.0.smithi060.stderr:2024-09-12T12:55:23.169+0000 7ff4cec00640 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T12:55:27.540 DEBUG:teuthology.orchestra.run.smithi060:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster cluster1 osd dump --format=json 2024-09-12T12:55:27.893 INFO:teuthology.misc.health.smithi060.stdout: 2024-09-12T12:55:27.894 INFO:teuthology.misc.health.smithi060.stdout:{"epoch":11,"fsid":"2a63e29a-4caf-4cc6-962c-13f8b91a9d2b","created":"2024-09-12T12:55:12.622095+0000","modified":"2024-09-12T12:55:27.134667+0000","last_up_change":"2024-09-12T12:55:24.130456+0000","last_in_change":"2024-09-12T12:55:13.728922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T12:55:25.770242+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8a419ad1-c999-4a27-8870-2d6e505f6398","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":156850008},{"type":"v1","addr":"172.21.15.60:6809","nonce":156850008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":156850008},{"type":"v1","addr":"172.21.15.60:6811","nonce":156850008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":156850008},{"type":"v1","addr":"172.21.15.60:6815","nonce":156850008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":156850008},{"type":"v1","addr":"172.21.15.60:6813","nonce":156850008}]},"public_addr":"172.21.15.60:6809/156850008","cluster_addr":"172.21.15.60:6811/156850008","heartbeat_back_addr":"172.21.15.60:6815/156850008","heartbeat_front_addr":"172.21.15.60:6813/156850008","state":["exists","up"]},{"osd":1,"uuid":"2b198f71-f471-4b42-9d29-f25f9f69a1e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":10,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1750992701},{"type":"v1","addr":"172.21.15.60:6801","nonce":1750992701}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":1750992701},{"type":"v1","addr":"172.21.15.60:6803","nonce":1750992701}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":1750992701},{"type":"v1","addr":"172.21.15.60:6807","nonce":1750992701}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":1750992701},{"type":"v1","addr":"172.21.15.60:6805","nonce":1750992701}]},"public_addr":"172.21.15.60:6801/1750992701","cluster_addr":"172.21.15.60:6803/1750992701","heartbeat_back_addr":"172.21.15.60:6807/1750992701","heartbeat_front_addr":"172.21.15.60:6805/1750992701","state":["exists","up"]},{"osd":2,"uuid":"39c37b7a-038d-49f6-b67c-78a015c7966c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":37652449},{"type":"v1","addr":"172.21.15.60:6817","nonce":37652449}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":37652449},{"type":"v1","addr":"172.21.15.60:6819","nonce":37652449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":37652449},{"type":"v1","addr":"172.21.15.60:6823","nonce":37652449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":37652449},{"type":"v1","addr":"172.21.15.60:6821","nonce":37652449}]},"public_addr":"172.21.15.60:6817/37652449","cluster_addr":"172.21.15.60:6819/37652449","heartbeat_back_addr":"172.21.15.60:6823/37652449","heartbeat_front_addr":"172.21.15.60:6821/37652449","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:55:21.424833+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:55:21.274503+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:55:21.542972+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T12:55:27.909 DEBUG:teuthology.misc:3 of 3 OSDs are up 2024-09-12T12:55:27.909 INFO:tasks.ceph:Creating RBD pool 2024-09-12T12:55:27.909 DEBUG:teuthology.orchestra.run.smithi060:> sudo ceph --cluster cluster1 osd pool create rbd 8 2024-09-12T12:55:29.142 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'rbd' created 2024-09-12T12:55:29.163 DEBUG:teuthology.orchestra.run.smithi060:> rbd --cluster cluster1 pool init rbd 2024-09-12T12:55:32.177 INFO:tasks.ceph:Starting mds daemons in cluster cluster1... 2024-09-12T12:55:32.178 INFO:tasks.ceph:Waiting until cluster1 daemons up and pgs clean... 2024-09-12T12:55:32.178 INFO:tasks.ceph.ceph_manager.cluster1:waiting for mgr available 2024-09-12T12:55:32.178 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster1 mgr dump --format=json 2024-09-12T12:55:32.179 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-09-12T12:55:32.588 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-12T12:55:32.604 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":5,"flags":0,"active_gid":4101,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":2246967692},{"type":"v1","addr":"172.21.15.60:6825","nonce":2246967692}]},"active_addr":"172.21.15.60:6825/2246967692","active_change":"2024-09-12T12:55:17.706175+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.60:0","nonce":2964950491}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.60:0","nonce":1233360461}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.60:0","nonce":2241032092}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.60:0","nonce":1861318167}]}]} 2024-09-12T12:55:32.606 INFO:tasks.ceph.ceph_manager.cluster1:mgr available! 2024-09-12T12:55:32.606 INFO:tasks.ceph.ceph_manager.cluster1:waiting for all up 2024-09-12T12:55:32.606 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster1 osd dump --format=json 2024-09-12T12:55:32.959 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-12T12:55:32.959 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":16,"fsid":"2a63e29a-4caf-4cc6-962c-13f8b91a9d2b","created":"2024-09-12T12:55:12.622095+0000","modified":"2024-09-12T12:55:32.149860+0000","last_up_change":"2024-09-12T12:55:24.130456+0000","last_in_change":"2024-09-12T12:55:13.728922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T12:55:25.770242+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-12T12:55:28.339452+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":16,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8a419ad1-c999-4a27-8870-2d6e505f6398","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":156850008},{"type":"v1","addr":"172.21.15.60:6809","nonce":156850008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":156850008},{"type":"v1","addr":"172.21.15.60:6811","nonce":156850008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":156850008},{"type":"v1","addr":"172.21.15.60:6815","nonce":156850008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":156850008},{"type":"v1","addr":"172.21.15.60:6813","nonce":156850008}]},"public_addr":"172.21.15.60:6809/156850008","cluster_addr":"172.21.15.60:6811/156850008","heartbeat_back_addr":"172.21.15.60:6815/156850008","heartbeat_front_addr":"172.21.15.60:6813/156850008","state":["exists","up"]},{"osd":1,"uuid":"2b198f71-f471-4b42-9d29-f25f9f69a1e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1750992701},{"type":"v1","addr":"172.21.15.60:6801","nonce":1750992701}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":1750992701},{"type":"v1","addr":"172.21.15.60:6803","nonce":1750992701}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":1750992701},{"type":"v1","addr":"172.21.15.60:6807","nonce":1750992701}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":1750992701},{"type":"v1","addr":"172.21.15.60:6805","nonce":1750992701}]},"public_addr":"172.21.15.60:6801/1750992701","cluster_addr":"172.21.15.60:6803/1750992701","heartbeat_back_addr":"172.21.15.60:6807/1750992701","heartbeat_front_addr":"172.21.15.60:6805/1750992701","state":["exists","up"]},{"osd":2,"uuid":"39c37b7a-038d-49f6-b67c-78a015c7966c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":37652449},{"type":"v1","addr":"172.21.15.60:6817","nonce":37652449}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":37652449},{"type":"v1","addr":"172.21.15.60:6819","nonce":37652449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":37652449},{"type":"v1","addr":"172.21.15.60:6823","nonce":37652449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":37652449},{"type":"v1","addr":"172.21.15.60:6821","nonce":37652449}]},"public_addr":"172.21.15.60:6817/37652449","cluster_addr":"172.21.15.60:6819/37652449","heartbeat_back_addr":"172.21.15.60:6823/37652449","heartbeat_front_addr":"172.21.15.60:6821/37652449","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:55:21.424833+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:55:21.274503+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:55:21.542972+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T12:55:32.976 INFO:tasks.ceph.ceph_manager.cluster1:all up! 2024-09-12T12:55:32.976 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster1 osd dump --format=json 2024-09-12T12:55:33.328 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-12T12:55:33.328 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":16,"fsid":"2a63e29a-4caf-4cc6-962c-13f8b91a9d2b","created":"2024-09-12T12:55:12.622095+0000","modified":"2024-09-12T12:55:32.149860+0000","last_up_change":"2024-09-12T12:55:24.130456+0000","last_in_change":"2024-09-12T12:55:13.728922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T12:55:25.770242+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-12T12:55:28.339452+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":16,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8a419ad1-c999-4a27-8870-2d6e505f6398","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":156850008},{"type":"v1","addr":"172.21.15.60:6809","nonce":156850008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":156850008},{"type":"v1","addr":"172.21.15.60:6811","nonce":156850008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":156850008},{"type":"v1","addr":"172.21.15.60:6815","nonce":156850008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":156850008},{"type":"v1","addr":"172.21.15.60:6813","nonce":156850008}]},"public_addr":"172.21.15.60:6809/156850008","cluster_addr":"172.21.15.60:6811/156850008","heartbeat_back_addr":"172.21.15.60:6815/156850008","heartbeat_front_addr":"172.21.15.60:6813/156850008","state":["exists","up"]},{"osd":1,"uuid":"2b198f71-f471-4b42-9d29-f25f9f69a1e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1750992701},{"type":"v1","addr":"172.21.15.60:6801","nonce":1750992701}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":1750992701},{"type":"v1","addr":"172.21.15.60:6803","nonce":1750992701}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":1750992701},{"type":"v1","addr":"172.21.15.60:6807","nonce":1750992701}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":1750992701},{"type":"v1","addr":"172.21.15.60:6805","nonce":1750992701}]},"public_addr":"172.21.15.60:6801/1750992701","cluster_addr":"172.21.15.60:6803/1750992701","heartbeat_back_addr":"172.21.15.60:6807/1750992701","heartbeat_front_addr":"172.21.15.60:6805/1750992701","state":["exists","up"]},{"osd":2,"uuid":"39c37b7a-038d-49f6-b67c-78a015c7966c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":37652449},{"type":"v1","addr":"172.21.15.60:6817","nonce":37652449}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":37652449},{"type":"v1","addr":"172.21.15.60:6819","nonce":37652449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":37652449},{"type":"v1","addr":"172.21.15.60:6823","nonce":37652449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":37652449},{"type":"v1","addr":"172.21.15.60:6821","nonce":37652449}]},"public_addr":"172.21.15.60:6817/37652449","cluster_addr":"172.21.15.60:6819/37652449","heartbeat_back_addr":"172.21.15.60:6823/37652449","heartbeat_front_addr":"172.21.15.60:6821/37652449","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:55:21.424833+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:55:21.274503+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:55:21.542972+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T12:55:33.346 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster1 tell osd.0 flush_pg_stats 2024-09-12T12:55:33.346 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster1 tell osd.1 flush_pg_stats 2024-09-12T12:55:33.347 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster1 tell osd.2 flush_pg_stats 2024-09-12T12:55:33.551 INFO:teuthology.orchestra.run.smithi060.stdout:34359738372 2024-09-12T12:55:33.551 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster1 osd last-stat-seq osd.2 2024-09-12T12:55:33.570 INFO:teuthology.orchestra.run.smithi060.stdout:34359738373 2024-09-12T12:55:33.570 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster1 osd last-stat-seq osd.0 2024-09-12T12:55:33.571 INFO:teuthology.orchestra.run.smithi060.stdout:34359738372 2024-09-12T12:55:33.572 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster1 osd last-stat-seq osd.1 2024-09-12T12:55:33.914 INFO:teuthology.orchestra.run.smithi060.stdout:34359738371 2024-09-12T12:55:33.933 INFO:tasks.ceph.ceph_manager.cluster1:need seq 34359738372 got 34359738371 for osd.2 2024-09-12T12:55:33.981 INFO:teuthology.orchestra.run.smithi060.stdout:34359738371 2024-09-12T12:55:33.984 INFO:teuthology.orchestra.run.smithi060.stdout:34359738372 2024-09-12T12:55:34.005 INFO:tasks.ceph.ceph_manager.cluster1:need seq 34359738372 got 34359738371 for osd.1 2024-09-12T12:55:34.007 INFO:tasks.ceph.ceph_manager.cluster1:need seq 34359738373 got 34359738372 for osd.0 2024-09-12T12:55:34.934 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster1 osd last-stat-seq osd.2 2024-09-12T12:55:35.007 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster1 osd last-stat-seq osd.1 2024-09-12T12:55:35.008 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster1 osd last-stat-seq osd.0 2024-09-12T12:55:35.361 INFO:teuthology.orchestra.run.smithi060.stdout:34359738372 2024-09-12T12:55:35.375 INFO:teuthology.orchestra.run.smithi060.stdout:34359738372 2024-09-12T12:55:35.380 INFO:tasks.ceph.ceph_manager.cluster1:need seq 34359738372 got 34359738372 for osd.2 2024-09-12T12:55:35.380 DEBUG:teuthology.parallel:result is None 2024-09-12T12:55:35.395 INFO:tasks.ceph.ceph_manager.cluster1:need seq 34359738372 got 34359738372 for osd.1 2024-09-12T12:55:35.395 DEBUG:teuthology.parallel:result is None 2024-09-12T12:55:35.432 INFO:teuthology.orchestra.run.smithi060.stdout:34359738373 2024-09-12T12:55:35.449 INFO:tasks.ceph.ceph_manager.cluster1:need seq 34359738373 got 34359738373 for osd.0 2024-09-12T12:55:35.449 DEBUG:teuthology.parallel:result is None 2024-09-12T12:55:35.449 INFO:tasks.ceph.ceph_manager.cluster1:waiting for clean 2024-09-12T12:55:35.450 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster1 pg dump --format=json 2024-09-12T12:55:35.838 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-12T12:55:35.839 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2024-09-12T12:55:35.855 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":20,"stamp":"2024-09-12T12:55:35.716347+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":139,"num_write_kb":1896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":100,"ondisk_log_size":100,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":95144,"kb_used_data":1576,"kb_used_omap":4,"kb_used_meta":93499,"kb_avail":283020376,"statfs":{"total":289910292480,"available":289812865024,"internally_reserved":0,"allocated":1613824,"data_stored":1303162,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4785,"internal_metadata":95743311},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.570098"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":19,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-12T12:55:32.158921+0000","last_change":"2024-09-12T12:55:32.159226+0000","last_active":"2024-09-12T12:55:32.158921+0000","last_peered":"2024-09-12T12:55:32.158921+0000","last_clean":"2024-09-12T12:55:32.158921+0000","last_became_active":"2024-09-12T12:55:30.158364+0000","last_became_peered":"2024-09-12T12:55:30.158364+0000","last_unstale":"2024-09-12T12:55:32.158921+0000","last_undegraded":"2024-09-12T12:55:32.158921+0000","last_fullsized":"2024-09-12T12:55:32.158921+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:55:29.139739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:55:29.139739+0000","last_clean_scrub_stamp":"2024-09-12T12:55:29.139739+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:47:27.317250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095102900000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":19,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-12T12:55:32.158432+0000","last_change":"2024-09-12T12:55:32.158781+0000","last_active":"2024-09-12T12:55:32.158432+0000","last_peered":"2024-09-12T12:55:32.158432+0000","last_clean":"2024-09-12T12:55:32.158432+0000","last_became_active":"2024-09-12T12:55:30.154763+0000","last_became_peered":"2024-09-12T12:55:30.154763+0000","last_unstale":"2024-09-12T12:55:32.158432+0000","last_undegraded":"2024-09-12T12:55:32.158432+0000","last_fullsized":"2024-09-12T12:55:32.158432+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:55:29.139739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:55:29.139739+0000","last_clean_scrub_stamp":"2024-09-12T12:55:29.139739+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:59:24.414161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062363399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-12T12:55:32.159300+0000","last_change":"2024-09-12T12:55:32.159553+0000","last_active":"2024-09-12T12:55:32.159300+0000","last_peered":"2024-09-12T12:55:32.159300+0000","last_clean":"2024-09-12T12:55:32.159300+0000","last_became_active":"2024-09-12T12:55:30.154871+0000","last_became_peered":"2024-09-12T12:55:30.154871+0000","last_unstale":"2024-09-12T12:55:32.159300+0000","last_undegraded":"2024-09-12T12:55:32.159300+0000","last_fullsized":"2024-09-12T12:55:32.159300+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:55:29.139739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:55:29.139739+0000","last_clean_scrub_stamp":"2024-09-12T12:55:29.139739+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:07:25.204569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010629509999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-12T12:55:32.158879+0000","last_change":"2024-09-12T12:55:32.159226+0000","last_active":"2024-09-12T12:55:32.158879+0000","last_peered":"2024-09-12T12:55:32.158879+0000","last_clean":"2024-09-12T12:55:32.158879+0000","last_became_active":"2024-09-12T12:55:30.154781+0000","last_became_peered":"2024-09-12T12:55:30.154781+0000","last_unstale":"2024-09-12T12:55:32.158879+0000","last_undegraded":"2024-09-12T12:55:32.158879+0000","last_fullsized":"2024-09-12T12:55:32.158879+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:55:29.139739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:55:29.139739+0000","last_clean_scrub_stamp":"2024-09-12T12:55:29.139739+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:45:34.855250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010971749999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"16'2","reported_seq":21,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-12T12:55:32.154984+0000","last_change":"2024-09-12T12:55:32.154984+0000","last_active":"2024-09-12T12:55:32.154984+0000","last_peered":"2024-09-12T12:55:32.154984+0000","last_clean":"2024-09-12T12:55:32.154984+0000","last_became_active":"2024-09-12T12:55:30.157400+0000","last_became_peered":"2024-09-12T12:55:30.157400+0000","last_unstale":"2024-09-12T12:55:32.154984+0000","last_undegraded":"2024-09-12T12:55:32.154984+0000","last_fullsized":"2024-09-12T12:55:32.154984+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:55:29.139739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:55:29.139739+0000","last_clean_scrub_stamp":"2024-09-12T12:55:29.139739+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:00:30.935389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051984500000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":19,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-12T12:55:32.687476+0000","last_change":"2024-09-12T12:55:32.687689+0000","last_active":"2024-09-12T12:55:32.687476+0000","last_peered":"2024-09-12T12:55:32.687476+0000","last_clean":"2024-09-12T12:55:32.687476+0000","last_became_active":"2024-09-12T12:55:30.157447+0000","last_became_peered":"2024-09-12T12:55:30.157447+0000","last_unstale":"2024-09-12T12:55:32.687476+0000","last_undegraded":"2024-09-12T12:55:32.687476+0000","last_fullsized":"2024-09-12T12:55:32.687476+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:55:29.139739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:55:29.139739+0000","last_clean_scrub_stamp":"2024-09-12T12:55:29.139739+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:11:12.106599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047483699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":19,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-12T12:55:32.687421+0000","last_change":"2024-09-12T12:55:32.687614+0000","last_active":"2024-09-12T12:55:32.687421+0000","last_peered":"2024-09-12T12:55:32.687421+0000","last_clean":"2024-09-12T12:55:32.687421+0000","last_became_active":"2024-09-12T12:55:30.156802+0000","last_became_peered":"2024-09-12T12:55:30.156802+0000","last_unstale":"2024-09-12T12:55:32.687421+0000","last_undegraded":"2024-09-12T12:55:32.687421+0000","last_fullsized":"2024-09-12T12:55:32.687421+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:55:29.139739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:55:29.139739+0000","last_clean_scrub_stamp":"2024-09-12T12:55:29.139739+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T00:47:04.052477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039869699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"14'1","reported_seq":20,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-12T12:55:32.158493+0000","last_change":"2024-09-12T12:55:32.158817+0000","last_active":"2024-09-12T12:55:32.158493+0000","last_peered":"2024-09-12T12:55:32.158493+0000","last_clean":"2024-09-12T12:55:32.158493+0000","last_became_active":"2024-09-12T12:55:30.154063+0000","last_became_peered":"2024-09-12T12:55:30.154063+0000","last_unstale":"2024-09-12T12:55:32.158493+0000","last_undegraded":"2024-09-12T12:55:32.158493+0000","last_fullsized":"2024-09-12T12:55:32.158493+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:55:29.139739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:55:29.139739+0000","last_clean_scrub_stamp":"2024-09-12T12:55:29.139739+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:40:17.400915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062050300000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"11'97","reported_seq":146,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-12T12:55:32.158399+0000","last_change":"2024-09-12T12:55:27.318143+0000","last_active":"2024-09-12T12:55:32.158399+0000","last_peered":"2024-09-12T12:55:32.158399+0000","last_clean":"2024-09-12T12:55:32.158399+0000","last_became_active":"2024-09-12T12:55:27.317743+0000","last_became_peered":"2024-09-12T12:55:27.317743+0000","last_unstale":"2024-09-12T12:55:32.158399+0000","last_undegraded":"2024-09-12T12:55:32.158399+0000","last_fullsized":"2024-09-12T12:55:32.158399+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:55:26.132558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:55:26.132558+0000","last_clean_scrub_stamp":"2024-09-12T12:55:26.132558+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:19:44.340577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738373,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31328,"kb_used_data":136,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340512,"statfs":{"total":96636764160,"available":96604684288,"internally_reserved":0,"allocated":139264,"data_stored":40796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":8,"seq":34359738373,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31908,"kb_used_data":720,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339932,"statfs":{"total":96636764160,"available":96604090368,"internally_reserved":0,"allocated":737280,"data_stored":631183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738374,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31908,"kb_used_data":720,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339932,"statfs":{"total":96636764160,"available":96604090368,"internally_reserved":0,"allocated":737280,"data_stored":631183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T12:55:35.856 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster1 pg dump --format=json 2024-09-12T12:55:36.207 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-12T12:55:36.208 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2024-09-12T12:55:36.224 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":20,"stamp":"2024-09-12T12:55:35.716347+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":139,"num_write_kb":1896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":100,"ondisk_log_size":100,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":95144,"kb_used_data":1576,"kb_used_omap":4,"kb_used_meta":93499,"kb_avail":283020376,"statfs":{"total":289910292480,"available":289812865024,"internally_reserved":0,"allocated":1613824,"data_stored":1303162,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4785,"internal_metadata":95743311},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.570098"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":19,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-12T12:55:32.158921+0000","last_change":"2024-09-12T12:55:32.159226+0000","last_active":"2024-09-12T12:55:32.158921+0000","last_peered":"2024-09-12T12:55:32.158921+0000","last_clean":"2024-09-12T12:55:32.158921+0000","last_became_active":"2024-09-12T12:55:30.158364+0000","last_became_peered":"2024-09-12T12:55:30.158364+0000","last_unstale":"2024-09-12T12:55:32.158921+0000","last_undegraded":"2024-09-12T12:55:32.158921+0000","last_fullsized":"2024-09-12T12:55:32.158921+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:55:29.139739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:55:29.139739+0000","last_clean_scrub_stamp":"2024-09-12T12:55:29.139739+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:47:27.317250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095102900000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":19,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-12T12:55:32.158432+0000","last_change":"2024-09-12T12:55:32.158781+0000","last_active":"2024-09-12T12:55:32.158432+0000","last_peered":"2024-09-12T12:55:32.158432+0000","last_clean":"2024-09-12T12:55:32.158432+0000","last_became_active":"2024-09-12T12:55:30.154763+0000","last_became_peered":"2024-09-12T12:55:30.154763+0000","last_unstale":"2024-09-12T12:55:32.158432+0000","last_undegraded":"2024-09-12T12:55:32.158432+0000","last_fullsized":"2024-09-12T12:55:32.158432+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:55:29.139739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:55:29.139739+0000","last_clean_scrub_stamp":"2024-09-12T12:55:29.139739+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:59:24.414161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062363399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-12T12:55:32.159300+0000","last_change":"2024-09-12T12:55:32.159553+0000","last_active":"2024-09-12T12:55:32.159300+0000","last_peered":"2024-09-12T12:55:32.159300+0000","last_clean":"2024-09-12T12:55:32.159300+0000","last_became_active":"2024-09-12T12:55:30.154871+0000","last_became_peered":"2024-09-12T12:55:30.154871+0000","last_unstale":"2024-09-12T12:55:32.159300+0000","last_undegraded":"2024-09-12T12:55:32.159300+0000","last_fullsized":"2024-09-12T12:55:32.159300+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:55:29.139739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:55:29.139739+0000","last_clean_scrub_stamp":"2024-09-12T12:55:29.139739+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:07:25.204569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010629509999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-12T12:55:32.158879+0000","last_change":"2024-09-12T12:55:32.159226+0000","last_active":"2024-09-12T12:55:32.158879+0000","last_peered":"2024-09-12T12:55:32.158879+0000","last_clean":"2024-09-12T12:55:32.158879+0000","last_became_active":"2024-09-12T12:55:30.154781+0000","last_became_peered":"2024-09-12T12:55:30.154781+0000","last_unstale":"2024-09-12T12:55:32.158879+0000","last_undegraded":"2024-09-12T12:55:32.158879+0000","last_fullsized":"2024-09-12T12:55:32.158879+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:55:29.139739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:55:29.139739+0000","last_clean_scrub_stamp":"2024-09-12T12:55:29.139739+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:45:34.855250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010971749999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"16'2","reported_seq":21,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-12T12:55:32.154984+0000","last_change":"2024-09-12T12:55:32.154984+0000","last_active":"2024-09-12T12:55:32.154984+0000","last_peered":"2024-09-12T12:55:32.154984+0000","last_clean":"2024-09-12T12:55:32.154984+0000","last_became_active":"2024-09-12T12:55:30.157400+0000","last_became_peered":"2024-09-12T12:55:30.157400+0000","last_unstale":"2024-09-12T12:55:32.154984+0000","last_undegraded":"2024-09-12T12:55:32.154984+0000","last_fullsized":"2024-09-12T12:55:32.154984+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:55:29.139739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:55:29.139739+0000","last_clean_scrub_stamp":"2024-09-12T12:55:29.139739+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:00:30.935389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051984500000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":19,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-12T12:55:32.687476+0000","last_change":"2024-09-12T12:55:32.687689+0000","last_active":"2024-09-12T12:55:32.687476+0000","last_peered":"2024-09-12T12:55:32.687476+0000","last_clean":"2024-09-12T12:55:32.687476+0000","last_became_active":"2024-09-12T12:55:30.157447+0000","last_became_peered":"2024-09-12T12:55:30.157447+0000","last_unstale":"2024-09-12T12:55:32.687476+0000","last_undegraded":"2024-09-12T12:55:32.687476+0000","last_fullsized":"2024-09-12T12:55:32.687476+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:55:29.139739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:55:29.139739+0000","last_clean_scrub_stamp":"2024-09-12T12:55:29.139739+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:11:12.106599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047483699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":19,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-12T12:55:32.687421+0000","last_change":"2024-09-12T12:55:32.687614+0000","last_active":"2024-09-12T12:55:32.687421+0000","last_peered":"2024-09-12T12:55:32.687421+0000","last_clean":"2024-09-12T12:55:32.687421+0000","last_became_active":"2024-09-12T12:55:30.156802+0000","last_became_peered":"2024-09-12T12:55:30.156802+0000","last_unstale":"2024-09-12T12:55:32.687421+0000","last_undegraded":"2024-09-12T12:55:32.687421+0000","last_fullsized":"2024-09-12T12:55:32.687421+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:55:29.139739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:55:29.139739+0000","last_clean_scrub_stamp":"2024-09-12T12:55:29.139739+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T00:47:04.052477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039869699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"14'1","reported_seq":20,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-12T12:55:32.158493+0000","last_change":"2024-09-12T12:55:32.158817+0000","last_active":"2024-09-12T12:55:32.158493+0000","last_peered":"2024-09-12T12:55:32.158493+0000","last_clean":"2024-09-12T12:55:32.158493+0000","last_became_active":"2024-09-12T12:55:30.154063+0000","last_became_peered":"2024-09-12T12:55:30.154063+0000","last_unstale":"2024-09-12T12:55:32.158493+0000","last_undegraded":"2024-09-12T12:55:32.158493+0000","last_fullsized":"2024-09-12T12:55:32.158493+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:55:29.139739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:55:29.139739+0000","last_clean_scrub_stamp":"2024-09-12T12:55:29.139739+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:40:17.400915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062050300000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"11'97","reported_seq":146,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-12T12:55:32.158399+0000","last_change":"2024-09-12T12:55:27.318143+0000","last_active":"2024-09-12T12:55:32.158399+0000","last_peered":"2024-09-12T12:55:32.158399+0000","last_clean":"2024-09-12T12:55:32.158399+0000","last_became_active":"2024-09-12T12:55:27.317743+0000","last_became_peered":"2024-09-12T12:55:27.317743+0000","last_unstale":"2024-09-12T12:55:32.158399+0000","last_undegraded":"2024-09-12T12:55:32.158399+0000","last_fullsized":"2024-09-12T12:55:32.158399+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:55:26.132558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:55:26.132558+0000","last_clean_scrub_stamp":"2024-09-12T12:55:26.132558+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:19:44.340577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738373,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31328,"kb_used_data":136,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340512,"statfs":{"total":96636764160,"available":96604684288,"internally_reserved":0,"allocated":139264,"data_stored":40796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":8,"seq":34359738373,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31908,"kb_used_data":720,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339932,"statfs":{"total":96636764160,"available":96604090368,"internally_reserved":0,"allocated":737280,"data_stored":631183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738374,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31908,"kb_used_data":720,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339932,"statfs":{"total":96636764160,"available":96604090368,"internally_reserved":0,"allocated":737280,"data_stored":631183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T12:55:36.225 INFO:tasks.ceph.ceph_manager.cluster1:clean! 2024-09-12T12:55:36.225 INFO:tasks.ceph:Waiting until ceph cluster cluster1 is healthy... 2024-09-12T12:55:36.225 INFO:tasks.ceph.ceph_manager.cluster1:wait_until_healthy 2024-09-12T12:55:36.225 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster1 health --format=json 2024-09-12T12:55:36.618 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-12T12:55:36.618 INFO:teuthology.orchestra.run.smithi060.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-12T12:55:36.635 INFO:tasks.ceph.ceph_manager.cluster1:wait_until_healthy done 2024-09-12T12:55:36.635 INFO:teuthology.run_tasks:Running task ceph... 2024-09-12T12:55:36.644 INFO:tasks.ceph:Creating ceph cluster cluster2... 2024-09-12T12:55:36.644 INFO:tasks.ceph:config {'conf': {'client': {'rbd default map options': 'ms_mode=legacy,rxbounce'}, 'global': {'mon warn on pool no app': False, 'ms die on skipped message': False}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)'], 'cpu_profile': set(), 'cluster': 'cluster2', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-09-12T12:55:36.644 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/teuthology-2024-09-11_21:48:03-krbd-squid-testing-default-smithi/7901765', 'branch': 'squid', 'description': 'krbd/mirror/{bluestore-bitmap clusters/2-node conf install/ceph ms_mode$/{legacy-rxbounce} tasks/compare-mirror-images}', 'email': 'ceph-qa@ceph.com', 'first_in_suite': False, 'job_id': '7901765', 'kernel': {'flavor': 'default', 'kdb': 1, 'sha1': '2ca44a2f35ca8f36983aaaa46d49312239ef519a'}, 'last_in_suite': False, 'machine_type': 'smithi', 'meta': [{'desc': '2 ceph clusters with 1 mon, 1 mgr and 3 osd each'}], 'name': 'teuthology-2024-09-11_21:48:03-krbd-squid-testing-default-smithi', 'no_nested_subset': False, 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'squid'}, 'ceph': {'conf': {'client': {'rbd default map options': 'ms_mode=legacy,rxbounce'}, 'global': {'mon warn on pool no app': False, 'ms die on skipped message': False}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)'], 'sha1': '086c737fa75c22d521a3932c59b31c5e1d1c9ef4'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '086c737fa75c22d521a3932c59b31c5e1d1c9ef4'}, 'extra_system_packages': ['pv']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'squid', 'sha1': '086c737fa75c22d521a3932c59b31c5e1d1c9ef4'}}, 'owner': 'scheduled_teuthology@teuthology', 'priority': 100, 'repo': 'https://git.ceph.com/ceph.git', 'roles': [['cluster1.mon.a', 'cluster1.mgr.x', 'cluster1.osd.0', 'cluster1.osd.1', 'cluster1.osd.2'], ['cluster2.mon.a', 'cluster2.mgr.x', 'cluster2.osd.0', 'cluster2.osd.1', 'cluster2.osd.2', 'cluster1.client.mirror', 'cluster1.client.mirror.0', 'cluster2.client.mirror', 'cluster2.client.mirror.0']], 'seed': 6642, 'sha1': '086c737fa75c22d521a3932c59b31c5e1d1c9ef4', 'sleep_before_teardown': 0, 'subset': '2/4', 'suite': 'krbd', 'suite_branch': 'squid', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph_086c737fa75c22d521a3932c59b31c5e1d1c9ef4/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph.git', 'suite_sha1': '086c737fa75c22d521a3932c59b31c5e1d1c9ef4', 'targets': {'smithi060.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIFOvl+hCSxBzLOW+W0g8RdwY9HvLI1Bc6w8kIMW64qum4rkb3wrve2G3MNnVVKdcGrJpzRuajn03zqmcHad4HA=', 'smithi130.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAP6S2zoN+SHzbLXCy7LSlxt94/nNlTvcxp6VIom6mCS2EdmiV6WkzCK7pkJ6lvDtvlYwqfk7CacbxjINJQq9RU='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'flavor': 'default', 'kdb': 1, 'sha1': '2ca44a2f35ca8f36983aaaa46d49312239ef519a'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': ['rbd-mirror'], 'flavor': 'default', 'sha1': '086c737fa75c22d521a3932c59b31c5e1d1c9ef4', 'extra_system_packages': ['pv']}}, {'ceph': {'cluster': 'cluster1', 'conf': {'client': {'rbd default map options': 'ms_mode=legacy,rxbounce'}, 'global': {'mon warn on pool no app': False, 'ms die on skipped message': False}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)'], 'sha1': '086c737fa75c22d521a3932c59b31c5e1d1c9ef4'}}, {'ceph': {'cluster': 'cluster2', 'conf': {'client': {'rbd default map options': 'ms_mode=legacy,rxbounce'}, 'global': {'mon warn on pool no app': False, 'ms die on skipped message': False}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)'], 'sha1': '086c737fa75c22d521a3932c59b31c5e1d1c9ef4'}}, {'rbd-mirror': {'client': 'cluster1.client.mirror.0', 'thrash': False}}, {'rbd-mirror': {'client': 'cluster2.client.mirror.0', 'thrash': False}}, {'workunit': {'clients': {'cluster1.client.mirror': ['rbd/compare_mirror_images.sh']}, 'env': {'RBD_DEVICE_TYPE': 'krbd', 'RBD_MIRROR_USE_RBD_MIRROR': '1'}, 'timeout': '3h'}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a', 'timestamp': '2024-09-11_21:48:03', 'tube': 'smithi', 'user': 'teuthology', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856'} 2024-09-12T12:55:36.645 DEBUG:teuthology.orchestra.run.smithi060:> install -d -m0755 -- /home/ubuntu/cephtest/cluster2.data 2024-09-12T12:55:36.649 DEBUG:teuthology.orchestra.run.smithi130:> install -d -m0755 -- /home/ubuntu/cephtest/cluster2.data 2024-09-12T12:55:36.688 DEBUG:teuthology.orchestra.run.smithi060:> sudo install -d -m0777 -- /var/run/ceph 2024-09-12T12:55:36.724 DEBUG:teuthology.orchestra.run.smithi130:> sudo install -d -m0777 -- /var/run/ceph 2024-09-12T12:55:36.788 DEBUG:teuthology.orchestra.run.smithi130:> set -ex 2024-09-12T12:55:36.788 DEBUG:teuthology.orchestra.run.smithi130:> dd if=/scratch_devs of=/dev/stdout 2024-09-12T12:55:36.824 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-12T12:55:36.824 DEBUG:teuthology.orchestra.run.smithi130:> stat /dev/vg_nvme/lv_1 2024-09-12T12:55:36.900 INFO:teuthology.orchestra.run.smithi130.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-12T12:55:36.901 INFO:teuthology.orchestra.run.smithi130.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T12:55:36.901 INFO:teuthology.orchestra.run.smithi130.stdout:Device: 6h/6d Inode: 632 Links: 1 2024-09-12T12:55:36.901 INFO:teuthology.orchestra.run.smithi130.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T12:55:36.901 INFO:teuthology.orchestra.run.smithi130.stdout:Context: system_u:object_r:device_t:s0 2024-09-12T12:55:36.901 INFO:teuthology.orchestra.run.smithi130.stdout:Access: 2024-09-12 12:54:40.740897301 +0000 2024-09-12T12:55:36.901 INFO:teuthology.orchestra.run.smithi130.stdout:Modify: 2024-09-12 12:52:46.677509870 +0000 2024-09-12T12:55:36.901 INFO:teuthology.orchestra.run.smithi130.stdout:Change: 2024-09-12 12:52:46.677509870 +0000 2024-09-12T12:55:36.901 INFO:teuthology.orchestra.run.smithi130.stdout: Birth: 2024-09-12 12:52:46.677509870 +0000 2024-09-12T12:55:36.901 DEBUG:teuthology.orchestra.run.smithi130:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-12T12:55:36.993 INFO:teuthology.orchestra.run.smithi130.stderr:1+0 records in 2024-09-12T12:55:36.993 INFO:teuthology.orchestra.run.smithi130.stderr:1+0 records out 2024-09-12T12:55:36.993 INFO:teuthology.orchestra.run.smithi130.stderr:512 bytes copied, 0.000339044 s, 1.5 MB/s 2024-09-12T12:55:36.995 DEBUG:teuthology.orchestra.run.smithi130:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-12T12:55:37.031 DEBUG:teuthology.orchestra.run.smithi130:> stat /dev/vg_nvme/lv_2 2024-09-12T12:55:37.108 INFO:teuthology.orchestra.run.smithi130.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-12T12:55:37.108 INFO:teuthology.orchestra.run.smithi130.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T12:55:37.108 INFO:teuthology.orchestra.run.smithi130.stdout:Device: 6h/6d Inode: 652 Links: 1 2024-09-12T12:55:37.108 INFO:teuthology.orchestra.run.smithi130.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T12:55:37.108 INFO:teuthology.orchestra.run.smithi130.stdout:Context: system_u:object_r:device_t:s0 2024-09-12T12:55:37.108 INFO:teuthology.orchestra.run.smithi130.stdout:Access: 2024-09-12 12:54:40.741897283 +0000 2024-09-12T12:55:37.108 INFO:teuthology.orchestra.run.smithi130.stdout:Modify: 2024-09-12 12:52:47.152501383 +0000 2024-09-12T12:55:37.108 INFO:teuthology.orchestra.run.smithi130.stdout:Change: 2024-09-12 12:52:47.152501383 +0000 2024-09-12T12:55:37.109 INFO:teuthology.orchestra.run.smithi130.stdout: Birth: 2024-09-12 12:52:47.152501383 +0000 2024-09-12T12:55:37.109 DEBUG:teuthology.orchestra.run.smithi130:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-12T12:55:37.200 INFO:teuthology.orchestra.run.smithi130.stderr:1+0 records in 2024-09-12T12:55:37.200 INFO:teuthology.orchestra.run.smithi130.stderr:1+0 records out 2024-09-12T12:55:37.200 INFO:teuthology.orchestra.run.smithi130.stderr:512 bytes copied, 0.000285663 s, 1.8 MB/s 2024-09-12T12:55:37.203 DEBUG:teuthology.orchestra.run.smithi130:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-12T12:55:37.242 DEBUG:teuthology.orchestra.run.smithi130:> stat /dev/vg_nvme/lv_3 2024-09-12T12:55:37.319 INFO:teuthology.orchestra.run.smithi130.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-12T12:55:37.320 INFO:teuthology.orchestra.run.smithi130.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T12:55:37.320 INFO:teuthology.orchestra.run.smithi130.stdout:Device: 6h/6d Inode: 669 Links: 1 2024-09-12T12:55:37.320 INFO:teuthology.orchestra.run.smithi130.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T12:55:37.320 INFO:teuthology.orchestra.run.smithi130.stdout:Context: system_u:object_r:device_t:s0 2024-09-12T12:55:37.320 INFO:teuthology.orchestra.run.smithi130.stdout:Access: 2024-09-12 12:54:40.741897283 +0000 2024-09-12T12:55:37.320 INFO:teuthology.orchestra.run.smithi130.stdout:Modify: 2024-09-12 12:52:47.627492897 +0000 2024-09-12T12:55:37.320 INFO:teuthology.orchestra.run.smithi130.stdout:Change: 2024-09-12 12:52:47.627492897 +0000 2024-09-12T12:55:37.320 INFO:teuthology.orchestra.run.smithi130.stdout: Birth: 2024-09-12 12:52:47.627492897 +0000 2024-09-12T12:55:37.320 DEBUG:teuthology.orchestra.run.smithi130:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-12T12:55:37.414 INFO:teuthology.orchestra.run.smithi130.stderr:1+0 records in 2024-09-12T12:55:37.414 INFO:teuthology.orchestra.run.smithi130.stderr:1+0 records out 2024-09-12T12:55:37.414 INFO:teuthology.orchestra.run.smithi130.stderr:512 bytes copied, 0.000304081 s, 1.7 MB/s 2024-09-12T12:55:37.417 DEBUG:teuthology.orchestra.run.smithi130:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-12T12:55:37.455 DEBUG:teuthology.orchestra.run.smithi130:> stat /dev/vg_nvme/lv_4 2024-09-12T12:55:37.532 INFO:teuthology.orchestra.run.smithi130.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-12T12:55:37.532 INFO:teuthology.orchestra.run.smithi130.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T12:55:37.532 INFO:teuthology.orchestra.run.smithi130.stdout:Device: 6h/6d Inode: 688 Links: 1 2024-09-12T12:55:37.533 INFO:teuthology.orchestra.run.smithi130.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T12:55:37.533 INFO:teuthology.orchestra.run.smithi130.stdout:Context: system_u:object_r:device_t:s0 2024-09-12T12:55:37.533 INFO:teuthology.orchestra.run.smithi130.stdout:Access: 2024-09-12 12:54:40.742897265 +0000 2024-09-12T12:55:37.533 INFO:teuthology.orchestra.run.smithi130.stdout:Modify: 2024-09-12 12:52:48.098484481 +0000 2024-09-12T12:55:37.533 INFO:teuthology.orchestra.run.smithi130.stdout:Change: 2024-09-12 12:52:48.098484481 +0000 2024-09-12T12:55:37.533 INFO:teuthology.orchestra.run.smithi130.stdout: Birth: 2024-09-12 12:52:48.098484481 +0000 2024-09-12T12:55:37.533 DEBUG:teuthology.orchestra.run.smithi130:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-12T12:55:37.629 INFO:teuthology.orchestra.run.smithi130.stderr:1+0 records in 2024-09-12T12:55:37.629 INFO:teuthology.orchestra.run.smithi130.stderr:1+0 records out 2024-09-12T12:55:37.629 INFO:teuthology.orchestra.run.smithi130.stderr:512 bytes copied, 0.000311003 s, 1.6 MB/s 2024-09-12T12:55:37.632 DEBUG:teuthology.orchestra.run.smithi130:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-12T12:55:37.670 INFO:tasks.ceph:osd dev map: {'cluster2.osd.0': '/dev/vg_nvme/lv_1', 'cluster2.osd.1': '/dev/vg_nvme/lv_2', 'cluster2.osd.2': '/dev/vg_nvme/lv_3'} 2024-09-12T12:55:37.670 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi130.front.sepia.ceph.com'): {'cluster2.osd.0': '/dev/vg_nvme/lv_1', 'cluster2.osd.1': '/dev/vg_nvme/lv_2', 'cluster2.osd.2': '/dev/vg_nvme/lv_3'}} 2024-09-12T12:55:37.670 INFO:tasks.ceph:Generating config... 2024-09-12T12:55:37.671 INFO:tasks.ceph:[client] rbd default map options = ms_mode=legacy,rxbounce 2024-09-12T12:55:37.671 INFO:tasks.ceph:[global] mon warn on pool no app = False 2024-09-12T12:55:37.671 INFO:tasks.ceph:[global] ms die on skipped message = False 2024-09-12T12:55:37.672 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-09-12T12:55:37.672 INFO:tasks.ceph:[mgr] debug ms = 1 2024-09-12T12:55:37.672 INFO:tasks.ceph:[mon] debug mon = 20 2024-09-12T12:55:37.672 INFO:tasks.ceph:[mon] debug ms = 1 2024-09-12T12:55:37.672 INFO:tasks.ceph:[mon] debug paxos = 20 2024-09-12T12:55:37.672 INFO:tasks.ceph:[osd] bdev async discard = True 2024-09-12T12:55:37.672 INFO:tasks.ceph:[osd] bdev enable discard = True 2024-09-12T12:55:37.672 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2024-09-12T12:55:37.672 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-09-12T12:55:37.672 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-09-12T12:55:37.672 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-09-12T12:55:37.672 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-09-12T12:55:37.672 INFO:tasks.ceph:[osd] debug ms = 1 2024-09-12T12:55:37.673 INFO:tasks.ceph:[osd] debug osd = 20 2024-09-12T12:55:37.673 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-09-12T12:55:37.673 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-09-12T12:55:37.673 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-09-12T12:55:37.673 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-09-12T12:55:37.673 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-09-12T12:55:37.673 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-09-12T12:55:37.673 INFO:tasks.ceph:Setting up cluster2.mon.a... 2024-09-12T12:55:37.673 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/cluster2.keyring 2024-09-12T12:55:37.791 INFO:teuthology.orchestra.run.smithi130.stdout:creating /etc/ceph/cluster2.keyring 2024-09-12T12:55:37.797 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/cluster2.keyring 2024-09-12T12:55:37.886 DEBUG:teuthology.orchestra.run.smithi130:> sudo chmod 0644 /etc/ceph/cluster2.keyring 2024-09-12T12:55:37.941 DEBUG:tasks.ceph:Ceph mon addresses: [('cluster2.mon.a', '172.21.15.130')] 2024-09-12T12:55:37.942 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.130', 'mon warn on pool no app': False, 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false'}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'rbd default map options': 'ms_mode=legacy,rxbounce'}, 'mon.a': {}} 2024-09-12T12:55:37.942 DEBUG:teuthology.orchestra.run.smithi130:> set -ex 2024-09-12T12:55:37.942 DEBUG:teuthology.orchestra.run.smithi130:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-09-12T12:55:37.977 DEBUG:teuthology.orchestra.run.smithi130:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.130 --print /home/ubuntu/cephtest/cluster2.monmap 2024-09-12T12:55:38.089 INFO:teuthology.orchestra.run.smithi130.stdout:monmaptool: monmap file /home/ubuntu/cephtest/cluster2.monmap 2024-09-12T12:55:38.089 INFO:teuthology.orchestra.run.smithi130.stdout:monmaptool: generated fsid 76d48f4f-9f5b-4bde-b048-7f763396fe3f 2024-09-12T12:55:38.089 INFO:teuthology.orchestra.run.smithi130.stdout:setting min_mon_release = quincy 2024-09-12T12:55:38.089 INFO:teuthology.orchestra.run.smithi130.stdout:epoch 0 2024-09-12T12:55:38.089 INFO:teuthology.orchestra.run.smithi130.stdout:fsid 76d48f4f-9f5b-4bde-b048-7f763396fe3f 2024-09-12T12:55:38.089 INFO:teuthology.orchestra.run.smithi130.stdout:last_changed 2024-09-12T12:55:38.090100+0000 2024-09-12T12:55:38.089 INFO:teuthology.orchestra.run.smithi130.stdout:created 2024-09-12T12:55:38.090100+0000 2024-09-12T12:55:38.090 INFO:teuthology.orchestra.run.smithi130.stdout:min_mon_release 17 (quincy) 2024-09-12T12:55:38.090 INFO:teuthology.orchestra.run.smithi130.stdout:election_strategy: 1 2024-09-12T12:55:38.090 INFO:teuthology.orchestra.run.smithi130.stdout:0: [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] mon.a 2024-09-12T12:55:38.090 INFO:teuthology.orchestra.run.smithi130.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/cluster2.monmap (1 monitors) 2024-09-12T12:55:38.092 DEBUG:teuthology.orchestra.run.smithi130:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-09-12T12:55:38.126 INFO:tasks.ceph:Writing /etc/ceph/cluster2.conf for FSID 76d48f4f-9f5b-4bde-b048-7f763396fe3f... 2024-09-12T12:55:38.127 DEBUG:teuthology.orchestra.run.smithi060:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/cluster2.conf && sudo chmod 0644 /etc/ceph/cluster2.conf > /dev/null 2024-09-12T12:55:38.132 DEBUG:teuthology.orchestra.run.smithi130:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/cluster2.conf && sudo chmod 0644 /etc/ceph/cluster2.conf > /dev/null 2024-09-12T12:55:38.234 INFO:teuthology.orchestra.run.smithi060.stdout:[global] 2024-09-12T12:55:38.234 INFO:teuthology.orchestra.run.smithi060.stdout: chdir = "" 2024-09-12T12:55:38.235 INFO:teuthology.orchestra.run.smithi060.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-12T12:55:38.235 INFO:teuthology.orchestra.run.smithi060.stdout: auth supported = cephx 2024-09-12T12:55:38.235 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-12T12:55:38.235 INFO:teuthology.orchestra.run.smithi060.stdout: filestore xattr use omap = true 2024-09-12T12:55:38.235 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-12T12:55:38.235 INFO:teuthology.orchestra.run.smithi060.stdout: mon clock drift allowed = 1.000 2024-09-12T12:55:38.235 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-12T12:55:38.235 INFO:teuthology.orchestra.run.smithi060.stdout: osd crush chooseleaf type = 0 2024-09-12T12:55:38.235 INFO:teuthology.orchestra.run.smithi060.stdout: auth debug = true 2024-09-12T12:55:38.235 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-12T12:55:38.235 INFO:teuthology.orchestra.run.smithi060.stdout: ms die on old message = true 2024-09-12T12:55:38.235 INFO:teuthology.orchestra.run.smithi060.stdout: ms die on bug = true 2024-09-12T12:55:38.235 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-12T12:55:38.236 INFO:teuthology.orchestra.run.smithi060.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-12T12:55:38.236 INFO:teuthology.orchestra.run.smithi060.stdout: mon pg warn max object skew = 0 2024-09-12T12:55:38.236 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-12T12:55:38.236 INFO:teuthology.orchestra.run.smithi060.stdout: # disable pg_autoscaler by default for new pools 2024-09-12T12:55:38.236 INFO:teuthology.orchestra.run.smithi060.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-12T12:55:38.236 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-12T12:55:38.236 INFO:teuthology.orchestra.run.smithi060.stdout: osd pool default size = 2 2024-09-12T12:55:38.236 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-12T12:55:38.236 INFO:teuthology.orchestra.run.smithi060.stdout: mon osd allow primary affinity = true 2024-09-12T12:55:38.236 INFO:teuthology.orchestra.run.smithi060.stdout: mon osd allow pg remap = true 2024-09-12T12:55:38.236 INFO:teuthology.orchestra.run.smithi060.stdout: mon warn on legacy crush tunables = false 2024-09-12T12:55:38.236 INFO:teuthology.orchestra.run.smithi060.stdout: mon warn on crush straw calc version zero = false 2024-09-12T12:55:38.237 INFO:teuthology.orchestra.run.smithi060.stdout: mon warn on no sortbitwise = false 2024-09-12T12:55:38.237 INFO:teuthology.orchestra.run.smithi060.stdout: mon warn on osd down out interval zero = false 2024-09-12T12:55:38.237 INFO:teuthology.orchestra.run.smithi060.stdout: mon warn on too few osds = false 2024-09-12T12:55:38.237 INFO:teuthology.orchestra.run.smithi060.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-12T12:55:38.237 INFO:teuthology.orchestra.run.smithi060.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-12T12:55:38.237 INFO:teuthology.orchestra.run.smithi060.stdout: mon_allow_pool_size_one = true 2024-09-12T12:55:38.237 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-12T12:55:38.237 INFO:teuthology.orchestra.run.smithi060.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-12T12:55:38.237 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-12T12:55:38.237 INFO:teuthology.orchestra.run.smithi060.stdout: osd default data pool replay window = 5 2024-09-12T12:55:38.237 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-12T12:55:38.237 INFO:teuthology.orchestra.run.smithi060.stdout: mon allow pool delete = true 2024-09-12T12:55:38.237 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-12T12:55:38.238 INFO:teuthology.orchestra.run.smithi060.stdout: mon cluster log file level = debug 2024-09-12T12:55:38.238 INFO:teuthology.orchestra.run.smithi060.stdout: debug asserts on shutdown = true 2024-09-12T12:55:38.238 INFO:teuthology.orchestra.run.smithi060.stdout: mon health detail to clog = false 2024-09-12T12:55:38.238 INFO:teuthology.orchestra.run.smithi060.stdout: mon host = 172.21.15.130 2024-09-12T12:55:38.238 INFO:teuthology.orchestra.run.smithi060.stdout: mon warn on pool no app = False 2024-09-12T12:55:38.238 INFO:teuthology.orchestra.run.smithi060.stdout: ms die on skipped message = False 2024-09-12T12:55:38.238 INFO:teuthology.orchestra.run.smithi060.stdout: fsid = 76d48f4f-9f5b-4bde-b048-7f763396fe3f 2024-09-12T12:55:38.238 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-12T12:55:38.238 INFO:teuthology.orchestra.run.smithi060.stdout:[osd] 2024-09-12T12:55:38.238 INFO:teuthology.orchestra.run.smithi060.stdout: osd journal size = 100 2024-09-12T12:55:38.238 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-12T12:55:38.238 INFO:teuthology.orchestra.run.smithi060.stdout: osd scrub load threshold = 5.0 2024-09-12T12:55:38.238 INFO:teuthology.orchestra.run.smithi060.stdout: osd scrub max interval = 600 2024-09-12T12:55:38.238 INFO:teuthology.orchestra.run.smithi060.stdout: osd mclock profile = high_recovery_ops 2024-09-12T12:55:38.239 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-12T12:55:38.239 INFO:teuthology.orchestra.run.smithi060.stdout: osd recover clone overlap = true 2024-09-12T12:55:38.239 INFO:teuthology.orchestra.run.smithi060.stdout: osd recovery max chunk = 1048576 2024-09-12T12:55:38.239 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-12T12:55:38.239 INFO:teuthology.orchestra.run.smithi060.stdout: osd debug shutdown = true 2024-09-12T12:55:38.239 INFO:teuthology.orchestra.run.smithi060.stdout: osd debug op order = true 2024-09-12T12:55:38.239 INFO:teuthology.orchestra.run.smithi060.stdout: osd debug verify stray on activate = true 2024-09-12T12:55:38.239 INFO:teuthology.orchestra.run.smithi060.stdout: osd debug trim objects = true 2024-09-12T12:55:38.239 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-12T12:55:38.239 INFO:teuthology.orchestra.run.smithi060.stdout: osd open classes on start = true 2024-09-12T12:55:38.239 INFO:teuthology.orchestra.run.smithi060.stdout: osd debug pg log writeout = true 2024-09-12T12:55:38.239 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-12T12:55:38.239 INFO:teuthology.orchestra.run.smithi060.stdout: osd deep scrub update digest min age = 30 2024-09-12T12:55:38.240 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-12T12:55:38.240 INFO:teuthology.orchestra.run.smithi060.stdout: osd map max advance = 10 2024-09-12T12:55:38.240 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-12T12:55:38.240 INFO:teuthology.orchestra.run.smithi060.stdout: journal zero on create = true 2024-09-12T12:55:38.240 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-12T12:55:38.240 INFO:teuthology.orchestra.run.smithi060.stdout: filestore ondisk finisher threads = 3 2024-09-12T12:55:38.240 INFO:teuthology.orchestra.run.smithi060.stdout: filestore apply finisher threads = 3 2024-09-12T12:55:38.240 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-12T12:55:38.240 INFO:teuthology.orchestra.run.smithi060.stdout: bdev debug aio = true 2024-09-12T12:55:38.240 INFO:teuthology.orchestra.run.smithi060.stdout: osd debug misdirected ops = true 2024-09-12T12:55:38.240 INFO:teuthology.orchestra.run.smithi060.stdout: bdev async discard = True 2024-09-12T12:55:38.240 INFO:teuthology.orchestra.run.smithi060.stdout: bdev enable discard = True 2024-09-12T12:55:38.240 INFO:teuthology.orchestra.run.smithi060.stdout: bluestore allocator = bitmap 2024-09-12T12:55:38.241 INFO:teuthology.orchestra.run.smithi060.stdout: bluestore block size = 96636764160 2024-09-12T12:55:38.241 INFO:teuthology.orchestra.run.smithi060.stdout: bluestore fsck on mount = True 2024-09-12T12:55:38.241 INFO:teuthology.orchestra.run.smithi060.stdout: debug bluefs = 1/20 2024-09-12T12:55:38.241 INFO:teuthology.orchestra.run.smithi060.stdout: debug bluestore = 1/20 2024-09-12T12:55:38.241 INFO:teuthology.orchestra.run.smithi060.stdout: debug ms = 1 2024-09-12T12:55:38.241 INFO:teuthology.orchestra.run.smithi060.stdout: debug osd = 20 2024-09-12T12:55:38.241 INFO:teuthology.orchestra.run.smithi060.stdout: debug rocksdb = 4/10 2024-09-12T12:55:38.241 INFO:teuthology.orchestra.run.smithi060.stdout: mon osd backfillfull_ratio = 0.85 2024-09-12T12:55:38.241 INFO:teuthology.orchestra.run.smithi060.stdout: mon osd full ratio = 0.9 2024-09-12T12:55:38.241 INFO:teuthology.orchestra.run.smithi060.stdout: mon osd nearfull ratio = 0.8 2024-09-12T12:55:38.241 INFO:teuthology.orchestra.run.smithi060.stdout: osd failsafe full ratio = 0.95 2024-09-12T12:55:38.241 INFO:teuthology.orchestra.run.smithi060.stdout: osd objectstore = bluestore 2024-09-12T12:55:38.241 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-12T12:55:38.242 INFO:teuthology.orchestra.run.smithi060.stdout:[mgr] 2024-09-12T12:55:38.242 INFO:teuthology.orchestra.run.smithi060.stdout: debug ms = 1 2024-09-12T12:55:38.242 INFO:teuthology.orchestra.run.smithi060.stdout: debug mgr = 20 2024-09-12T12:55:38.242 INFO:teuthology.orchestra.run.smithi060.stdout: debug mon = 20 2024-09-12T12:55:38.242 INFO:teuthology.orchestra.run.smithi060.stdout: debug auth = 20 2024-09-12T12:55:38.242 INFO:teuthology.orchestra.run.smithi060.stdout: mon reweight min pgs per osd = 4 2024-09-12T12:55:38.242 INFO:teuthology.orchestra.run.smithi060.stdout: mon reweight min bytes per osd = 10 2024-09-12T12:55:38.242 INFO:teuthology.orchestra.run.smithi060.stdout: mgr/telemetry/nag = false 2024-09-12T12:55:38.242 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-12T12:55:38.242 INFO:teuthology.orchestra.run.smithi060.stdout:[mon] 2024-09-12T12:55:38.242 INFO:teuthology.orchestra.run.smithi060.stdout: debug ms = 1 2024-09-12T12:55:38.242 INFO:teuthology.orchestra.run.smithi060.stdout: debug mon = 20 2024-09-12T12:55:38.242 INFO:teuthology.orchestra.run.smithi060.stdout: debug paxos = 20 2024-09-12T12:55:38.242 INFO:teuthology.orchestra.run.smithi060.stdout: debug auth = 20 2024-09-12T12:55:38.243 INFO:teuthology.orchestra.run.smithi060.stdout: mon data avail warn = 5 2024-09-12T12:55:38.243 INFO:teuthology.orchestra.run.smithi060.stdout: mon mgr mkfs grace = 240 2024-09-12T12:55:38.243 INFO:teuthology.orchestra.run.smithi060.stdout: mon reweight min pgs per osd = 4 2024-09-12T12:55:38.243 INFO:teuthology.orchestra.run.smithi060.stdout: mon osd reporter subtree level = osd 2024-09-12T12:55:38.243 INFO:teuthology.orchestra.run.smithi060.stdout: mon osd prime pg temp = true 2024-09-12T12:55:38.243 INFO:teuthology.orchestra.run.smithi060.stdout: mon reweight min bytes per osd = 10 2024-09-12T12:55:38.243 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-12T12:55:38.243 INFO:teuthology.orchestra.run.smithi060.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-12T12:55:38.243 INFO:teuthology.orchestra.run.smithi060.stdout: auth mon ticket ttl = 660 # 11m 2024-09-12T12:55:38.243 INFO:teuthology.orchestra.run.smithi060.stdout: auth service ticket ttl = 240 # 4m 2024-09-12T12:55:38.243 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-12T12:55:38.243 INFO:teuthology.orchestra.run.smithi060.stdout: # don't complain about insecure global_id in the test suite 2024-09-12T12:55:38.243 INFO:teuthology.orchestra.run.smithi060.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-12T12:55:38.243 INFO:teuthology.orchestra.run.smithi060.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-12T12:55:38.244 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-12T12:55:38.244 INFO:teuthology.orchestra.run.smithi060.stdout: # 1m isn't quite enough 2024-09-12T12:55:38.244 INFO:teuthology.orchestra.run.smithi060.stdout: mon_down_mkfs_grace = 2m 2024-09-12T12:55:38.244 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-12T12:55:38.244 INFO:teuthology.orchestra.run.smithi060.stdout: mon_warn_on_filestore_osds = false 2024-09-12T12:55:38.244 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-12T12:55:38.244 INFO:teuthology.orchestra.run.smithi060.stdout:[client] 2024-09-12T12:55:38.244 INFO:teuthology.orchestra.run.smithi060.stdout: rgw cache enabled = true 2024-09-12T12:55:38.244 INFO:teuthology.orchestra.run.smithi060.stdout: rgw enable ops log = true 2024-09-12T12:55:38.244 INFO:teuthology.orchestra.run.smithi060.stdout: rgw enable usage log = true 2024-09-12T12:55:38.244 INFO:teuthology.orchestra.run.smithi060.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-12T12:55:38.244 INFO:teuthology.orchestra.run.smithi060.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-12T12:55:38.244 INFO:teuthology.orchestra.run.smithi060.stdout: rbd default map options = "ms_mode=legacy,rxbounce" 2024-09-12T12:55:38.245 INFO:teuthology.orchestra.run.smithi060.stdout:[mon.a] 2024-09-12T12:55:38.265 INFO:teuthology.orchestra.run.smithi130.stdout:[global] 2024-09-12T12:55:38.265 INFO:teuthology.orchestra.run.smithi130.stdout: chdir = "" 2024-09-12T12:55:38.265 INFO:teuthology.orchestra.run.smithi130.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-12T12:55:38.266 INFO:teuthology.orchestra.run.smithi130.stdout: auth supported = cephx 2024-09-12T12:55:38.266 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-09-12T12:55:38.266 INFO:teuthology.orchestra.run.smithi130.stdout: filestore xattr use omap = true 2024-09-12T12:55:38.266 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-09-12T12:55:38.266 INFO:teuthology.orchestra.run.smithi130.stdout: mon clock drift allowed = 1.000 2024-09-12T12:55:38.266 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-09-12T12:55:38.266 INFO:teuthology.orchestra.run.smithi130.stdout: osd crush chooseleaf type = 0 2024-09-12T12:55:38.266 INFO:teuthology.orchestra.run.smithi130.stdout: auth debug = true 2024-09-12T12:55:38.266 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-09-12T12:55:38.266 INFO:teuthology.orchestra.run.smithi130.stdout: ms die on old message = true 2024-09-12T12:55:38.266 INFO:teuthology.orchestra.run.smithi130.stdout: ms die on bug = true 2024-09-12T12:55:38.266 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-09-12T12:55:38.266 INFO:teuthology.orchestra.run.smithi130.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-12T12:55:38.267 INFO:teuthology.orchestra.run.smithi130.stdout: mon pg warn max object skew = 0 2024-09-12T12:55:38.267 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-09-12T12:55:38.267 INFO:teuthology.orchestra.run.smithi130.stdout: # disable pg_autoscaler by default for new pools 2024-09-12T12:55:38.267 INFO:teuthology.orchestra.run.smithi130.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-12T12:55:38.267 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-09-12T12:55:38.267 INFO:teuthology.orchestra.run.smithi130.stdout: osd pool default size = 2 2024-09-12T12:55:38.267 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-09-12T12:55:38.267 INFO:teuthology.orchestra.run.smithi130.stdout: mon osd allow primary affinity = true 2024-09-12T12:55:38.267 INFO:teuthology.orchestra.run.smithi130.stdout: mon osd allow pg remap = true 2024-09-12T12:55:38.267 INFO:teuthology.orchestra.run.smithi130.stdout: mon warn on legacy crush tunables = false 2024-09-12T12:55:38.267 INFO:teuthology.orchestra.run.smithi130.stdout: mon warn on crush straw calc version zero = false 2024-09-12T12:55:38.267 INFO:teuthology.orchestra.run.smithi130.stdout: mon warn on no sortbitwise = false 2024-09-12T12:55:38.267 INFO:teuthology.orchestra.run.smithi130.stdout: mon warn on osd down out interval zero = false 2024-09-12T12:55:38.268 INFO:teuthology.orchestra.run.smithi130.stdout: mon warn on too few osds = false 2024-09-12T12:55:38.268 INFO:teuthology.orchestra.run.smithi130.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-12T12:55:38.268 INFO:teuthology.orchestra.run.smithi130.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-12T12:55:38.268 INFO:teuthology.orchestra.run.smithi130.stdout: mon_allow_pool_size_one = true 2024-09-12T12:55:38.268 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-09-12T12:55:38.268 INFO:teuthology.orchestra.run.smithi130.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-12T12:55:38.268 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-09-12T12:55:38.268 INFO:teuthology.orchestra.run.smithi130.stdout: osd default data pool replay window = 5 2024-09-12T12:55:38.268 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-09-12T12:55:38.268 INFO:teuthology.orchestra.run.smithi130.stdout: mon allow pool delete = true 2024-09-12T12:55:38.268 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-09-12T12:55:38.268 INFO:teuthology.orchestra.run.smithi130.stdout: mon cluster log file level = debug 2024-09-12T12:55:38.269 INFO:teuthology.orchestra.run.smithi130.stdout: debug asserts on shutdown = true 2024-09-12T12:55:38.269 INFO:teuthology.orchestra.run.smithi130.stdout: mon health detail to clog = false 2024-09-12T12:55:38.269 INFO:teuthology.orchestra.run.smithi130.stdout: mon host = 172.21.15.130 2024-09-12T12:55:38.269 INFO:teuthology.orchestra.run.smithi130.stdout: mon warn on pool no app = False 2024-09-12T12:55:38.269 INFO:teuthology.orchestra.run.smithi130.stdout: ms die on skipped message = False 2024-09-12T12:55:38.269 INFO:teuthology.orchestra.run.smithi130.stdout: fsid = 76d48f4f-9f5b-4bde-b048-7f763396fe3f 2024-09-12T12:55:38.269 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-09-12T12:55:38.269 INFO:teuthology.orchestra.run.smithi130.stdout:[osd] 2024-09-12T12:55:38.269 INFO:teuthology.orchestra.run.smithi130.stdout: osd journal size = 100 2024-09-12T12:55:38.269 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-09-12T12:55:38.269 INFO:teuthology.orchestra.run.smithi130.stdout: osd scrub load threshold = 5.0 2024-09-12T12:55:38.269 INFO:teuthology.orchestra.run.smithi130.stdout: osd scrub max interval = 600 2024-09-12T12:55:38.269 INFO:teuthology.orchestra.run.smithi130.stdout: osd mclock profile = high_recovery_ops 2024-09-12T12:55:38.270 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-09-12T12:55:38.270 INFO:teuthology.orchestra.run.smithi130.stdout: osd recover clone overlap = true 2024-09-12T12:55:38.270 INFO:teuthology.orchestra.run.smithi130.stdout: osd recovery max chunk = 1048576 2024-09-12T12:55:38.270 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-09-12T12:55:38.270 INFO:teuthology.orchestra.run.smithi130.stdout: osd debug shutdown = true 2024-09-12T12:55:38.270 INFO:teuthology.orchestra.run.smithi130.stdout: osd debug op order = true 2024-09-12T12:55:38.270 INFO:teuthology.orchestra.run.smithi130.stdout: osd debug verify stray on activate = true 2024-09-12T12:55:38.270 INFO:teuthology.orchestra.run.smithi130.stdout: osd debug trim objects = true 2024-09-12T12:55:38.270 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-09-12T12:55:38.270 INFO:teuthology.orchestra.run.smithi130.stdout: osd open classes on start = true 2024-09-12T12:55:38.270 INFO:teuthology.orchestra.run.smithi130.stdout: osd debug pg log writeout = true 2024-09-12T12:55:38.270 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-09-12T12:55:38.270 INFO:teuthology.orchestra.run.smithi130.stdout: osd deep scrub update digest min age = 30 2024-09-12T12:55:38.271 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-09-12T12:55:38.271 INFO:teuthology.orchestra.run.smithi130.stdout: osd map max advance = 10 2024-09-12T12:55:38.271 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-09-12T12:55:38.271 INFO:teuthology.orchestra.run.smithi130.stdout: journal zero on create = true 2024-09-12T12:55:38.271 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-09-12T12:55:38.271 INFO:teuthology.orchestra.run.smithi130.stdout: filestore ondisk finisher threads = 3 2024-09-12T12:55:38.271 INFO:teuthology.orchestra.run.smithi130.stdout: filestore apply finisher threads = 3 2024-09-12T12:55:38.271 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-09-12T12:55:38.271 INFO:teuthology.orchestra.run.smithi130.stdout: bdev debug aio = true 2024-09-12T12:55:38.271 INFO:teuthology.orchestra.run.smithi130.stdout: osd debug misdirected ops = true 2024-09-12T12:55:38.271 INFO:teuthology.orchestra.run.smithi130.stdout: bdev async discard = True 2024-09-12T12:55:38.271 INFO:teuthology.orchestra.run.smithi130.stdout: bdev enable discard = True 2024-09-12T12:55:38.271 INFO:teuthology.orchestra.run.smithi130.stdout: bluestore allocator = bitmap 2024-09-12T12:55:38.271 INFO:teuthology.orchestra.run.smithi130.stdout: bluestore block size = 96636764160 2024-09-12T12:55:38.272 INFO:teuthology.orchestra.run.smithi130.stdout: bluestore fsck on mount = True 2024-09-12T12:55:38.272 INFO:teuthology.orchestra.run.smithi130.stdout: debug bluefs = 1/20 2024-09-12T12:55:38.272 INFO:teuthology.orchestra.run.smithi130.stdout: debug bluestore = 1/20 2024-09-12T12:55:38.272 INFO:teuthology.orchestra.run.smithi130.stdout: debug ms = 1 2024-09-12T12:55:38.272 INFO:teuthology.orchestra.run.smithi130.stdout: debug osd = 20 2024-09-12T12:55:38.272 INFO:teuthology.orchestra.run.smithi130.stdout: debug rocksdb = 4/10 2024-09-12T12:55:38.272 INFO:teuthology.orchestra.run.smithi130.stdout: mon osd backfillfull_ratio = 0.85 2024-09-12T12:55:38.272 INFO:teuthology.orchestra.run.smithi130.stdout: mon osd full ratio = 0.9 2024-09-12T12:55:38.272 INFO:teuthology.orchestra.run.smithi130.stdout: mon osd nearfull ratio = 0.8 2024-09-12T12:55:38.272 INFO:teuthology.orchestra.run.smithi130.stdout: osd failsafe full ratio = 0.95 2024-09-12T12:55:38.272 INFO:teuthology.orchestra.run.smithi130.stdout: osd objectstore = bluestore 2024-09-12T12:55:38.272 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-09-12T12:55:38.273 INFO:teuthology.orchestra.run.smithi130.stdout:[mgr] 2024-09-12T12:55:38.273 INFO:teuthology.orchestra.run.smithi130.stdout: debug ms = 1 2024-09-12T12:55:38.273 INFO:teuthology.orchestra.run.smithi130.stdout: debug mgr = 20 2024-09-12T12:55:38.273 INFO:teuthology.orchestra.run.smithi130.stdout: debug mon = 20 2024-09-12T12:55:38.273 INFO:teuthology.orchestra.run.smithi130.stdout: debug auth = 20 2024-09-12T12:55:38.273 INFO:teuthology.orchestra.run.smithi130.stdout: mon reweight min pgs per osd = 4 2024-09-12T12:55:38.273 INFO:teuthology.orchestra.run.smithi130.stdout: mon reweight min bytes per osd = 10 2024-09-12T12:55:38.273 INFO:teuthology.orchestra.run.smithi130.stdout: mgr/telemetry/nag = false 2024-09-12T12:55:38.273 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-09-12T12:55:38.273 INFO:teuthology.orchestra.run.smithi130.stdout:[mon] 2024-09-12T12:55:38.273 INFO:teuthology.orchestra.run.smithi130.stdout: debug ms = 1 2024-09-12T12:55:38.273 INFO:teuthology.orchestra.run.smithi130.stdout: debug mon = 20 2024-09-12T12:55:38.273 INFO:teuthology.orchestra.run.smithi130.stdout: debug paxos = 20 2024-09-12T12:55:38.273 INFO:teuthology.orchestra.run.smithi130.stdout: debug auth = 20 2024-09-12T12:55:38.274 INFO:teuthology.orchestra.run.smithi130.stdout: mon data avail warn = 5 2024-09-12T12:55:38.274 INFO:teuthology.orchestra.run.smithi130.stdout: mon mgr mkfs grace = 240 2024-09-12T12:55:38.274 INFO:teuthology.orchestra.run.smithi130.stdout: mon reweight min pgs per osd = 4 2024-09-12T12:55:38.274 INFO:teuthology.orchestra.run.smithi130.stdout: mon osd reporter subtree level = osd 2024-09-12T12:55:38.274 INFO:teuthology.orchestra.run.smithi130.stdout: mon osd prime pg temp = true 2024-09-12T12:55:38.274 INFO:teuthology.orchestra.run.smithi130.stdout: mon reweight min bytes per osd = 10 2024-09-12T12:55:38.274 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-09-12T12:55:38.274 INFO:teuthology.orchestra.run.smithi130.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-12T12:55:38.274 INFO:teuthology.orchestra.run.smithi130.stdout: auth mon ticket ttl = 660 # 11m 2024-09-12T12:55:38.274 INFO:teuthology.orchestra.run.smithi130.stdout: auth service ticket ttl = 240 # 4m 2024-09-12T12:55:38.274 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-09-12T12:55:38.274 INFO:teuthology.orchestra.run.smithi130.stdout: # don't complain about insecure global_id in the test suite 2024-09-12T12:55:38.274 INFO:teuthology.orchestra.run.smithi130.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-12T12:55:38.274 INFO:teuthology.orchestra.run.smithi130.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-12T12:55:38.275 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-09-12T12:55:38.275 INFO:teuthology.orchestra.run.smithi130.stdout: # 1m isn't quite enough 2024-09-12T12:55:38.275 INFO:teuthology.orchestra.run.smithi130.stdout: mon_down_mkfs_grace = 2m 2024-09-12T12:55:38.275 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-09-12T12:55:38.275 INFO:teuthology.orchestra.run.smithi130.stdout: mon_warn_on_filestore_osds = false 2024-09-12T12:55:38.275 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-09-12T12:55:38.275 INFO:teuthology.orchestra.run.smithi130.stdout:[client] 2024-09-12T12:55:38.275 INFO:teuthology.orchestra.run.smithi130.stdout: rgw cache enabled = true 2024-09-12T12:55:38.275 INFO:teuthology.orchestra.run.smithi130.stdout: rgw enable ops log = true 2024-09-12T12:55:38.275 INFO:teuthology.orchestra.run.smithi130.stdout: rgw enable usage log = true 2024-09-12T12:55:38.275 INFO:teuthology.orchestra.run.smithi130.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-12T12:55:38.275 INFO:teuthology.orchestra.run.smithi130.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-12T12:55:38.275 INFO:teuthology.orchestra.run.smithi130.stdout: rbd default map options = "ms_mode=legacy,rxbounce" 2024-09-12T12:55:38.275 INFO:teuthology.orchestra.run.smithi130.stdout:[mon.a] 2024-09-12T12:55:38.291 INFO:tasks.ceph:Creating admin key on cluster2.mon.a... 2024-09-12T12:55:38.291 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/cluster2.keyring 2024-09-12T12:55:38.389 INFO:tasks.ceph:Copying monmap to all nodes... 2024-09-12T12:55:38.389 DEBUG:teuthology.orchestra.run.smithi130:> set -ex 2024-09-12T12:55:38.390 DEBUG:teuthology.orchestra.run.smithi130:> dd if=/etc/ceph/cluster2.keyring of=/dev/stdout 2024-09-12T12:55:38.425 DEBUG:teuthology.orchestra.run.smithi130:> set -ex 2024-09-12T12:55:38.425 DEBUG:teuthology.orchestra.run.smithi130:> dd if=/home/ubuntu/cephtest/cluster2.monmap of=/dev/stdout 2024-09-12T12:55:38.499 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi060.front.sepia.ceph.com 2024-09-12T12:55:38.500 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-09-12T12:55:38.500 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd of=/etc/ceph/cluster2.keyring 2024-09-12T12:55:38.500 DEBUG:teuthology.orchestra.run.smithi060:> sudo chmod 0644 /etc/ceph/cluster2.keyring 2024-09-12T12:55:38.582 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-09-12T12:55:38.582 DEBUG:teuthology.orchestra.run.smithi060:> dd of=/home/ubuntu/cephtest/cluster2.monmap 2024-09-12T12:55:38.658 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi130.front.sepia.ceph.com 2024-09-12T12:55:38.659 DEBUG:teuthology.orchestra.run.smithi130:> set -ex 2024-09-12T12:55:38.659 DEBUG:teuthology.orchestra.run.smithi130:> sudo dd of=/etc/ceph/cluster2.keyring 2024-09-12T12:55:38.659 DEBUG:teuthology.orchestra.run.smithi130:> sudo chmod 0644 /etc/ceph/cluster2.keyring 2024-09-12T12:55:38.737 DEBUG:teuthology.orchestra.run.smithi130:> set -ex 2024-09-12T12:55:38.737 DEBUG:teuthology.orchestra.run.smithi130:> dd of=/home/ubuntu/cephtest/cluster2.monmap 2024-09-12T12:55:38.811 INFO:tasks.ceph:Setting up mon nodes... 2024-09-12T12:55:38.812 INFO:tasks.ceph:Setting up mgr nodes... 2024-09-12T12:55:38.812 DEBUG:teuthology.orchestra.run.smithi130:> sudo mkdir -p /var/lib/ceph/mgr/cluster2-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/cluster2-x/keyring 2024-09-12T12:55:38.954 INFO:teuthology.orchestra.run.smithi130.stdout:creating /var/lib/ceph/mgr/cluster2-x/keyring 2024-09-12T12:55:38.960 INFO:tasks.ceph:Setting up mds nodes... 2024-09-12T12:55:38.961 INFO:tasks.ceph_client:Setting up client nodes... 2024-09-12T12:55:38.961 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.mirror /etc/ceph/cluster2.client.mirror.keyring && sudo chmod 0644 /etc/ceph/cluster2.client.mirror.keyring 2024-09-12T12:55:39.041 INFO:teuthology.orchestra.run.smithi130.stdout:creating /etc/ceph/cluster2.client.mirror.keyring 2024-09-12T12:55:39.068 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.mirror.0 /etc/ceph/cluster2.client.mirror.0.keyring && sudo chmod 0644 /etc/ceph/cluster2.client.mirror.0.keyring 2024-09-12T12:55:39.149 INFO:teuthology.orchestra.run.smithi130.stdout:creating /etc/ceph/cluster2.client.mirror.0.keyring 2024-09-12T12:55:39.177 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-09-12T12:55:39.177 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi060.front.sepia.ceph.com'): {'cluster1.osd.0': '/dev/vg_nvme/lv_1', 'cluster1.osd.1': '/dev/vg_nvme/lv_2', 'cluster1.osd.2': '/dev/vg_nvme/lv_3'}, Remote(name='ubuntu@smithi130.front.sepia.ceph.com'): {'cluster2.osd.0': '/dev/vg_nvme/lv_1', 'cluster2.osd.1': '/dev/vg_nvme/lv_2', 'cluster2.osd.2': '/dev/vg_nvme/lv_3'}} 2024-09-12T12:55:39.177 DEBUG:teuthology.orchestra.run.smithi130:> sudo mkdir -p /var/lib/ceph/osd/cluster2-0 2024-09-12T12:55:39.232 INFO:tasks.ceph:roles_to_devs: {'cluster2.osd.0': '/dev/vg_nvme/lv_1', 'cluster2.osd.1': '/dev/vg_nvme/lv_2', 'cluster2.osd.2': '/dev/vg_nvme/lv_3'} 2024-09-12T12:55:39.232 INFO:tasks.ceph:role: cluster2.osd.0 2024-09-12T12:55:39.233 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi130.front.sepia.ceph.com 2024-09-12T12:55:39.233 DEBUG:teuthology.orchestra.run.smithi130:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-12T12:55:39.295 INFO:teuthology.orchestra.run.smithi130.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-12T12:55:39.296 INFO:teuthology.orchestra.run.smithi130.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-12T12:55:39.296 INFO:teuthology.orchestra.run.smithi130.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-12T12:55:39.296 INFO:teuthology.orchestra.run.smithi130.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-12T12:55:39.296 INFO:teuthology.orchestra.run.smithi130.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-12T12:55:39.296 INFO:teuthology.orchestra.run.smithi130.stdout: = sunit=0 swidth=0 blks 2024-09-12T12:55:39.296 INFO:teuthology.orchestra.run.smithi130.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-12T12:55:39.296 INFO:teuthology.orchestra.run.smithi130.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-12T12:55:39.296 INFO:teuthology.orchestra.run.smithi130.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-12T12:55:39.296 INFO:teuthology.orchestra.run.smithi130.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-12T12:55:39.704 INFO:teuthology.orchestra.run.smithi130.stdout:Discarding blocks...Done. 2024-09-12T12:55:39.707 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi130.front.sepia.ceph.com -o noatime 2024-09-12T12:55:39.707 DEBUG:teuthology.orchestra.run.smithi130:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/cluster2-0 2024-09-12T12:55:39.853 DEBUG:teuthology.orchestra.run.smithi130:> sudo /sbin/restorecon /var/lib/ceph/osd/cluster2-0 2024-09-12T12:55:39.918 DEBUG:teuthology.orchestra.run.smithi130:> sudo mkdir -p /var/lib/ceph/osd/cluster2-1 2024-09-12T12:55:39.974 INFO:tasks.ceph:roles_to_devs: {'cluster2.osd.0': '/dev/vg_nvme/lv_1', 'cluster2.osd.1': '/dev/vg_nvme/lv_2', 'cluster2.osd.2': '/dev/vg_nvme/lv_3'} 2024-09-12T12:55:39.974 INFO:tasks.ceph:role: cluster2.osd.1 2024-09-12T12:55:39.974 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi130.front.sepia.ceph.com 2024-09-12T12:55:39.975 DEBUG:teuthology.orchestra.run.smithi130:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-12T12:55:40.037 INFO:teuthology.orchestra.run.smithi130.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-12T12:55:40.037 INFO:teuthology.orchestra.run.smithi130.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-12T12:55:40.037 INFO:teuthology.orchestra.run.smithi130.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-12T12:55:40.037 INFO:teuthology.orchestra.run.smithi130.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-12T12:55:40.037 INFO:teuthology.orchestra.run.smithi130.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-12T12:55:40.037 INFO:teuthology.orchestra.run.smithi130.stdout: = sunit=0 swidth=0 blks 2024-09-12T12:55:40.037 INFO:teuthology.orchestra.run.smithi130.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-12T12:55:40.038 INFO:teuthology.orchestra.run.smithi130.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-12T12:55:40.038 INFO:teuthology.orchestra.run.smithi130.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-12T12:55:40.038 INFO:teuthology.orchestra.run.smithi130.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-12T12:55:40.394 INFO:teuthology.orchestra.run.smithi130.stdout:Discarding blocks...Done. 2024-09-12T12:55:40.398 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi130.front.sepia.ceph.com -o noatime 2024-09-12T12:55:40.398 DEBUG:teuthology.orchestra.run.smithi130:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/cluster2-1 2024-09-12T12:55:40.481 DEBUG:teuthology.orchestra.run.smithi130:> sudo /sbin/restorecon /var/lib/ceph/osd/cluster2-1 2024-09-12T12:55:40.542 DEBUG:teuthology.orchestra.run.smithi130:> sudo mkdir -p /var/lib/ceph/osd/cluster2-2 2024-09-12T12:55:40.597 INFO:tasks.ceph:roles_to_devs: {'cluster2.osd.0': '/dev/vg_nvme/lv_1', 'cluster2.osd.1': '/dev/vg_nvme/lv_2', 'cluster2.osd.2': '/dev/vg_nvme/lv_3'} 2024-09-12T12:55:40.597 INFO:tasks.ceph:role: cluster2.osd.2 2024-09-12T12:55:40.598 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi130.front.sepia.ceph.com 2024-09-12T12:55:40.598 DEBUG:teuthology.orchestra.run.smithi130:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-12T12:55:40.660 INFO:teuthology.orchestra.run.smithi130.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-12T12:55:40.660 INFO:teuthology.orchestra.run.smithi130.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-12T12:55:40.660 INFO:teuthology.orchestra.run.smithi130.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-12T12:55:40.660 INFO:teuthology.orchestra.run.smithi130.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-12T12:55:40.660 INFO:teuthology.orchestra.run.smithi130.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-12T12:55:40.660 INFO:teuthology.orchestra.run.smithi130.stdout: = sunit=0 swidth=0 blks 2024-09-12T12:55:40.660 INFO:teuthology.orchestra.run.smithi130.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-12T12:55:40.660 INFO:teuthology.orchestra.run.smithi130.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-12T12:55:40.660 INFO:teuthology.orchestra.run.smithi130.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-12T12:55:40.660 INFO:teuthology.orchestra.run.smithi130.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-12T12:55:41.055 INFO:teuthology.orchestra.run.smithi130.stdout:Discarding blocks...Done. 2024-09-12T12:55:41.059 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi130.front.sepia.ceph.com -o noatime 2024-09-12T12:55:41.059 DEBUG:teuthology.orchestra.run.smithi130:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/cluster2-2 2024-09-12T12:55:41.162 DEBUG:teuthology.orchestra.run.smithi130:> sudo /sbin/restorecon /var/lib/ceph/osd/cluster2-2 2024-09-12T12:55:41.224 DEBUG:teuthology.orchestra.run.smithi130:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster cluster2 --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/cluster2.monmap 2024-09-12T12:55:41.324 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-12T12:55:41.324+0000 7f82655bb640 -1 auth: error reading file: /var/lib/ceph/osd/cluster2-0/keyring: can't open /var/lib/ceph/osd/cluster2-0/keyring: (2) No such file or directory 2024-09-12T12:55:41.328 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-12T12:55:41.327+0000 7f82655bb640 -1 created new key in keyring /var/lib/ceph/osd/cluster2-0/keyring 2024-09-12T12:55:41.328 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-12T12:55:41.327+0000 7f82655bb640 -1 bdev(0x55c5cfc23800 /var/lib/ceph/osd/cluster2-0/block) open stat got: (1) Operation not permitted 2024-09-12T12:55:41.328 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-12T12:55:41.328+0000 7f82655bb640 -1 bluestore(/var/lib/ceph/osd/cluster2-0) _read_fsid unparsable uuid 2024-09-12T12:55:44.847 DEBUG:teuthology.orchestra.run.smithi130:> sudo chown -R ceph:ceph /var/lib/ceph/osd/cluster2-0 2024-09-12T12:55:44.904 DEBUG:teuthology.orchestra.run.smithi130:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster cluster2 --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/cluster2.monmap 2024-09-12T12:55:44.997 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-12T12:55:44.996+0000 7f3fa2060640 -1 auth: error reading file: /var/lib/ceph/osd/cluster2-1/keyring: can't open /var/lib/ceph/osd/cluster2-1/keyring: (2) No such file or directory 2024-09-12T12:55:44.997 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-12T12:55:44.997+0000 7f3fa2060640 -1 created new key in keyring /var/lib/ceph/osd/cluster2-1/keyring 2024-09-12T12:55:44.997 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-12T12:55:44.997+0000 7f3fa2060640 -1 bdev(0x55fc45c49800 /var/lib/ceph/osd/cluster2-1/block) open stat got: (1) Operation not permitted 2024-09-12T12:55:44.997 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-12T12:55:44.997+0000 7f3fa2060640 -1 bluestore(/var/lib/ceph/osd/cluster2-1) _read_fsid unparsable uuid 2024-09-12T12:55:48.981 DEBUG:teuthology.orchestra.run.smithi130:> sudo chown -R ceph:ceph /var/lib/ceph/osd/cluster2-1 2024-09-12T12:55:49.038 DEBUG:teuthology.orchestra.run.smithi130:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster cluster2 --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/cluster2.monmap 2024-09-12T12:55:49.129 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-12T12:55:49.128+0000 7f34287a6640 -1 auth: error reading file: /var/lib/ceph/osd/cluster2-2/keyring: can't open /var/lib/ceph/osd/cluster2-2/keyring: (2) No such file or directory 2024-09-12T12:55:49.129 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-12T12:55:49.129+0000 7f34287a6640 -1 created new key in keyring /var/lib/ceph/osd/cluster2-2/keyring 2024-09-12T12:55:49.129 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-12T12:55:49.129+0000 7f34287a6640 -1 bdev(0x562df6e05800 /var/lib/ceph/osd/cluster2-2/block) open stat got: (1) Operation not permitted 2024-09-12T12:55:49.130 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-12T12:55:49.129+0000 7f34287a6640 -1 bluestore(/var/lib/ceph/osd/cluster2-2) _read_fsid unparsable uuid 2024-09-12T12:55:52.852 DEBUG:teuthology.orchestra.run.smithi130:> sudo chown -R ceph:ceph /var/lib/ceph/osd/cluster2-2 2024-09-12T12:55:52.912 INFO:tasks.ceph:Reading keys from all nodes... 2024-09-12T12:55:52.912 DEBUG:teuthology.orchestra.run.smithi130:> set -ex 2024-09-12T12:55:52.912 DEBUG:teuthology.orchestra.run.smithi130:> sudo dd if=/var/lib/ceph/mgr/cluster2-x/keyring of=/dev/stdout 2024-09-12T12:55:52.970 DEBUG:teuthology.orchestra.run.smithi130:> set -ex 2024-09-12T12:55:52.970 DEBUG:teuthology.orchestra.run.smithi130:> sudo dd if=/var/lib/ceph/osd/cluster2-0/keyring of=/dev/stdout 2024-09-12T12:55:53.066 DEBUG:teuthology.orchestra.run.smithi130:> set -ex 2024-09-12T12:55:53.066 DEBUG:teuthology.orchestra.run.smithi130:> sudo dd if=/var/lib/ceph/osd/cluster2-1/keyring of=/dev/stdout 2024-09-12T12:55:53.161 DEBUG:teuthology.orchestra.run.smithi130:> set -ex 2024-09-12T12:55:53.161 DEBUG:teuthology.orchestra.run.smithi130:> sudo dd if=/var/lib/ceph/osd/cluster2-2/keyring of=/dev/stdout 2024-09-12T12:55:53.257 DEBUG:teuthology.orchestra.run.smithi130:> set -ex 2024-09-12T12:55:53.257 DEBUG:teuthology.orchestra.run.smithi130:> dd if=/etc/ceph/cluster2.client.mirror.keyring of=/dev/stdout 2024-09-12T12:55:53.333 DEBUG:teuthology.orchestra.run.smithi130:> set -ex 2024-09-12T12:55:53.333 DEBUG:teuthology.orchestra.run.smithi130:> dd if=/etc/ceph/cluster2.client.mirror.0.keyring of=/dev/stdout 2024-09-12T12:55:53.408 INFO:tasks.ceph:Adding keys to all mons... 2024-09-12T12:55:53.408 DEBUG:teuthology.orchestra.run.smithi130:> sudo tee -a /etc/ceph/cluster2.keyring 2024-09-12T12:55:53.500 INFO:teuthology.orchestra.run.smithi130.stdout:[mgr.x] 2024-09-12T12:55:53.500 INFO:teuthology.orchestra.run.smithi130.stdout: key = AQDK5OJmOUr3OBAASYH7M752IqVhzUMY4fvjDw== 2024-09-12T12:55:53.500 INFO:teuthology.orchestra.run.smithi130.stdout:[osd.0] 2024-09-12T12:55:53.500 INFO:teuthology.orchestra.run.smithi130.stdout: key = AQDN5OJm8BZkExAA9hePnRRPHLkd5gti9sD8mg== 2024-09-12T12:55:53.500 INFO:teuthology.orchestra.run.smithi130.stdout:[osd.1] 2024-09-12T12:55:53.500 INFO:teuthology.orchestra.run.smithi130.stdout: key = AQDQ5OJmc397OxAAGiGmlh9k54ayLfzcWT8Zow== 2024-09-12T12:55:53.500 INFO:teuthology.orchestra.run.smithi130.stdout:[osd.2] 2024-09-12T12:55:53.501 INFO:teuthology.orchestra.run.smithi130.stdout: key = AQDV5OJmSdnBBxAAnCNpFMmXpVAzgX6Is11Xjw== 2024-09-12T12:55:53.501 INFO:teuthology.orchestra.run.smithi130.stdout:[client.mirror] 2024-09-12T12:55:53.501 INFO:teuthology.orchestra.run.smithi130.stdout: key = AQDL5OJmbm+OAhAAaVfsvc2GTghO5fbXamAgUQ== 2024-09-12T12:55:53.501 INFO:teuthology.orchestra.run.smithi130.stdout:[client.mirror.0] 2024-09-12T12:55:53.501 INFO:teuthology.orchestra.run.smithi130.stdout: key = AQDL5OJmy3X8CBAAMhHj+ZlDaLspyFxisbdI6A== 2024-09-12T12:55:53.502 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster2.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-12T12:55:53.637 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster2.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-12T12:55:53.730 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster2.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-12T12:55:53.823 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster2.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-12T12:55:53.916 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster2.keyring --name=client.mirror --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-12T12:55:54.012 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster2.keyring --name=client.mirror.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-12T12:55:54.107 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-09-12T12:55:54.107 DEBUG:teuthology.orchestra.run.smithi130:> sudo mkdir -p /var/lib/ceph/mon/cluster2-a 2024-09-12T12:55:54.163 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster cluster2 --mkfs -i a --monmap /home/ubuntu/cephtest/cluster2.monmap --keyring /etc/ceph/cluster2.keyring 2024-09-12T12:55:54.271 DEBUG:teuthology.orchestra.run.smithi130:> sudo chown -R ceph:ceph /var/lib/ceph/mon/cluster2-a 2024-09-12T12:55:54.325 DEBUG:teuthology.orchestra.run.smithi130:> rm -- /home/ubuntu/cephtest/cluster2.monmap 2024-09-12T12:55:54.360 INFO:tasks.ceph:Starting mon daemons in cluster cluster2... 2024-09-12T12:55:54.360 INFO:tasks.ceph.cluster2.mon.a:Restarting daemon 2024-09-12T12:55:54.360 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster cluster2 -i a 2024-09-12T12:55:54.404 INFO:tasks.ceph.cluster2.mon.a:Started 2024-09-12T12:55:54.404 INFO:tasks.ceph:Starting mgr daemons in cluster cluster2... 2024-09-12T12:55:54.405 INFO:tasks.ceph.cluster2.mgr.x:Restarting daemon 2024-09-12T12:55:54.405 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster cluster2 -i x 2024-09-12T12:55:54.409 INFO:tasks.ceph.cluster2.mgr.x:Started 2024-09-12T12:55:54.409 INFO:tasks.ceph:Setting crush tunables to default 2024-09-12T12:55:54.409 DEBUG:teuthology.orchestra.run.smithi130:> sudo ceph --cluster cluster2 osd crush tunables default 2024-09-12T12:55:54.637 INFO:teuthology.orchestra.run.smithi130.stderr:adjusted tunables profile to default 2024-09-12T12:55:54.653 INFO:tasks.ceph:check_enable_crimson: False 2024-09-12T12:55:54.653 INFO:tasks.ceph:Starting osd daemons in cluster cluster2... 2024-09-12T12:55:54.653 DEBUG:teuthology.orchestra.run.smithi130:> set -ex 2024-09-12T12:55:54.653 DEBUG:teuthology.orchestra.run.smithi130:> sudo dd if=/var/lib/ceph/osd/cluster2-0/fsid of=/dev/stdout 2024-09-12T12:55:54.708 DEBUG:teuthology.orchestra.run.smithi130:> set -ex 2024-09-12T12:55:54.708 DEBUG:teuthology.orchestra.run.smithi130:> sudo dd if=/var/lib/ceph/osd/cluster2-1/fsid of=/dev/stdout 2024-09-12T12:55:54.804 DEBUG:teuthology.orchestra.run.smithi130:> set -ex 2024-09-12T12:55:54.804 DEBUG:teuthology.orchestra.run.smithi130:> sudo dd if=/var/lib/ceph/osd/cluster2-2/fsid of=/dev/stdout 2024-09-12T12:55:54.873 INFO:tasks.ceph.cluster2.mgr.x.smithi130.stderr:2024-09-12T12:55:54.873+0000 7f7d023ef040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-12T12:55:54.900 DEBUG:teuthology.orchestra.run.smithi130:> sudo ceph --cluster cluster2 osd new 14760318-3ea5-426d-949a-87b78b1c5170 0 2024-09-12T12:55:54.924 INFO:tasks.ceph.cluster2.mgr.x.smithi130.stderr:2024-09-12T12:55:54.924+0000 7f7d023ef040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-12T12:55:55.103 INFO:tasks.ceph.cluster2.mgr.x.smithi130.stderr:2024-09-12T12:55:55.103+0000 7f7d023ef040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-12T12:55:55.155 INFO:teuthology.orchestra.run.smithi130.stdout:0 2024-09-12T12:55:55.165 INFO:tasks.ceph.cluster2.mgr.x.smithi130.stderr:2024-09-12T12:55:55.165+0000 7f7d023ef040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-12T12:55:55.172 DEBUG:teuthology.orchestra.run.smithi130:> sudo ceph --cluster cluster2 osd new 5129799d-ce0e-4fbc-9bcc-2528285fa983 1 2024-09-12T12:55:55.361 INFO:tasks.ceph.cluster2.mgr.x.smithi130.stderr:2024-09-12T12:55:55.361+0000 7f7d023ef040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-12T12:55:55.406 INFO:teuthology.orchestra.run.smithi130.stdout:1 2024-09-12T12:55:55.421 DEBUG:teuthology.orchestra.run.smithi130:> sudo ceph --cluster cluster2 osd new 3321b514-7dac-4085-9fe1-55714a27c720 2 2024-09-12T12:55:55.495 INFO:tasks.ceph.cluster2.mgr.x.smithi130.stderr:2024-09-12T12:55:55.494+0000 7f7d023ef040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-12T12:55:55.652 INFO:teuthology.orchestra.run.smithi130.stdout:2 2024-09-12T12:55:55.669 INFO:tasks.ceph.cluster2.osd.0:Restarting daemon 2024-09-12T12:55:55.669 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster cluster2 -i 0 2024-09-12T12:55:55.673 INFO:tasks.ceph.cluster2.osd.0:Started 2024-09-12T12:55:55.673 INFO:tasks.ceph.cluster2.osd.1:Restarting daemon 2024-09-12T12:55:55.673 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster cluster2 -i 1 2024-09-12T12:55:55.677 INFO:tasks.ceph.cluster2.osd.1:Started 2024-09-12T12:55:55.677 INFO:tasks.ceph.cluster2.osd.2:Restarting daemon 2024-09-12T12:55:55.677 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster cluster2 -i 2 2024-09-12T12:55:55.681 INFO:tasks.ceph.cluster2.osd.2:Started 2024-09-12T12:55:55.681 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster2 osd dump --format=json 2024-09-12T12:55:55.916 INFO:tasks.ceph.cluster2.mgr.x.smithi130.stderr:2024-09-12T12:55:55.916+0000 7f7d023ef040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-12T12:55:55.935 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-09-12T12:55:55.936 INFO:teuthology.orchestra.run.smithi130.stdout:{"epoch":5,"fsid":"76d48f4f-9f5b-4bde-b048-7f763396fe3f","created":"2024-09-12T12:55:54.555731+0000","modified":"2024-09-12T12:55:55.652531+0000","last_up_change":"0.000000","last_in_change":"2024-09-12T12:55:55.652531+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"14760318-3ea5-426d-949a-87b78b1c5170","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"5129799d-ce0e-4fbc-9bcc-2528285fa983","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"3321b514-7dac-4085-9fe1-55714a27c720","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T12:55:55.951 INFO:tasks.ceph.ceph_manager.cluster2:[] 2024-09-12T12:55:55.951 INFO:tasks.ceph:Waiting for OSDs to come up 2024-09-12T12:55:55.980 INFO:tasks.ceph.cluster2.mgr.x.smithi130.stderr:2024-09-12T12:55:55.979+0000 7f7d023ef040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-12T12:55:56.151 INFO:tasks.ceph.cluster2.mgr.x.smithi130.stderr:2024-09-12T12:55:56.150+0000 7f7d023ef040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-12T12:55:56.254 DEBUG:teuthology.orchestra.run.smithi130:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster cluster2 osd dump --format=json 2024-09-12T12:55:56.463 INFO:teuthology.misc.health.smithi130.stdout: 2024-09-12T12:55:56.463 INFO:teuthology.misc.health.smithi130.stdout:{"epoch":5,"fsid":"76d48f4f-9f5b-4bde-b048-7f763396fe3f","created":"2024-09-12T12:55:54.555731+0000","modified":"2024-09-12T12:55:55.652531+0000","last_up_change":"0.000000","last_in_change":"2024-09-12T12:55:55.652531+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"14760318-3ea5-426d-949a-87b78b1c5170","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"5129799d-ce0e-4fbc-9bcc-2528285fa983","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"3321b514-7dac-4085-9fe1-55714a27c720","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T12:55:56.477 DEBUG:teuthology.misc:0 of 3 OSDs are up 2024-09-12T12:55:57.291 INFO:tasks.ceph.cluster2.mgr.x.smithi130.stderr:2024-09-12T12:55:57.291+0000 7f7d023ef040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-12T12:55:57.352 INFO:tasks.ceph.cluster2.mgr.x.smithi130.stderr:2024-09-12T12:55:57.352+0000 7f7d023ef040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-12T12:55:57.752 INFO:tasks.ceph.cluster2.osd.0.smithi130.stderr:2024-09-12T12:55:57.752+0000 7f9ae39a4640 -1 Falling back to public interface 2024-09-12T12:55:57.759 INFO:tasks.ceph.cluster2.osd.1.smithi130.stderr:2024-09-12T12:55:57.759+0000 7fcf61729640 -1 Falling back to public interface 2024-09-12T12:55:57.760 INFO:tasks.ceph.cluster2.osd.2.smithi130.stderr:2024-09-12T12:55:57.760+0000 7f74f4232640 -1 Falling back to public interface 2024-09-12T12:55:57.791 INFO:tasks.ceph.cluster2.mgr.x.smithi130.stderr:2024-09-12T12:55:57.791+0000 7f7d023ef040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-12T12:55:57.889 INFO:tasks.ceph.cluster2.mgr.x.smithi130.stderr:2024-09-12T12:55:57.888+0000 7f7d023ef040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-12T12:55:58.008 INFO:tasks.ceph.cluster2.mgr.x.smithi130.stderr:2024-09-12T12:55:58.008+0000 7f7d023ef040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-12T12:55:58.065 INFO:tasks.ceph.cluster2.mgr.x.smithi130.stderr:2024-09-12T12:55:58.064+0000 7f7d023ef040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-12T12:55:58.194 INFO:tasks.ceph.cluster2.mgr.x.smithi130.stderr:2024-09-12T12:55:58.193+0000 7f7d023ef040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-12T12:55:58.256 INFO:tasks.ceph.cluster2.mgr.x.smithi130.stderr:2024-09-12T12:55:58.255+0000 7f7d023ef040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-12T12:55:58.308 INFO:tasks.ceph.cluster2.mgr.x.smithi130.stderr:2024-09-12T12:55:58.308+0000 7f7d023ef040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-12T12:55:58.436 INFO:tasks.ceph.cluster2.mgr.x.smithi130.stderr:2024-09-12T12:55:58.436+0000 7f7d023ef040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-12T12:55:58.495 INFO:tasks.ceph.cluster2.mgr.x.smithi130.stderr:2024-09-12T12:55:58.495+0000 7f7d023ef040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-12T12:55:58.555 INFO:tasks.ceph.cluster2.mgr.x.smithi130.stderr:2024-09-12T12:55:58.555+0000 7f7d023ef040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-12T12:55:58.635 INFO:tasks.ceph.cluster2.mgr.x.smithi130.stderr:2024-09-12T12:55:58.635+0000 7f7d023ef040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-12T12:55:58.687 INFO:tasks.ceph.cluster2.mgr.x.smithi130.stderr:2024-09-12T12:55:58.687+0000 7f7d023ef040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-12T12:55:58.738 INFO:tasks.ceph.cluster2.mgr.x.smithi130.stderr:2024-09-12T12:55:58.738+0000 7f7d023ef040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-12T12:55:59.483 INFO:tasks.ceph.cluster2.mgr.x.smithi130.stderr:2024-09-12T12:55:59.482+0000 7f7d023ef040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-12T12:56:00.494 INFO:tasks.ceph.cluster2.mgr.x.smithi130.stderr:2024-09-12T12:56:00.494+0000 7f7ca2600640 -1 mgr.server handle_report got status from non-daemon mon.a 2024-09-12T12:56:02.259 INFO:tasks.ceph.cluster2.osd.1.smithi130.stderr:2024-09-12T12:56:02.259+0000 7fcf61729640 -1 osd.1 0 log_to_monitors true 2024-09-12T12:56:02.425 INFO:tasks.ceph.cluster2.osd.0.smithi130.stderr:2024-09-12T12:56:02.425+0000 7f9ae39a4640 -1 osd.0 0 log_to_monitors true 2024-09-12T12:56:02.476 INFO:tasks.ceph.cluster2.osd.2.smithi130.stderr:2024-09-12T12:56:02.476+0000 7f74f4232640 -1 osd.2 0 log_to_monitors true 2024-09-12T12:56:02.780 DEBUG:teuthology.orchestra.run.smithi130:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster cluster2 osd dump --format=json 2024-09-12T12:56:03.104 INFO:teuthology.misc.health.smithi130.stdout: 2024-09-12T12:56:03.105 INFO:teuthology.misc.health.smithi130.stdout:{"epoch":6,"fsid":"76d48f4f-9f5b-4bde-b048-7f763396fe3f","created":"2024-09-12T12:55:54.555731+0000","modified":"2024-09-12T12:56:02.494873+0000","last_up_change":"0.000000","last_in_change":"2024-09-12T12:55:55.652531+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":3,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"14760318-3ea5-426d-949a-87b78b1c5170","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"5129799d-ce0e-4fbc-9bcc-2528285fa983","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"3321b514-7dac-4085-9fe1-55714a27c720","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T12:56:03.119 DEBUG:teuthology.misc:0 of 3 OSDs are up 2024-09-12T12:56:05.251 INFO:tasks.ceph.cluster2.osd.2.smithi130.stderr:2024-09-12T12:56:05.251+0000 7f74edc00640 -1 osd.2 0 waiting for initial osdmap 2024-09-12T12:56:05.257 INFO:tasks.ceph.cluster2.osd.2.smithi130.stderr:2024-09-12T12:56:05.257+0000 7f74e4600640 -1 osd.2 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T12:56:05.285 INFO:tasks.ceph.cluster2.osd.1.smithi130.stderr:2024-09-12T12:56:05.285+0000 7fcf5d000640 -1 osd.1 0 waiting for initial osdmap 2024-09-12T12:56:05.289 INFO:tasks.ceph.cluster2.osd.0.smithi130.stderr:2024-09-12T12:56:05.289+0000 7f9adde00640 -1 osd.0 0 waiting for initial osdmap 2024-09-12T12:56:05.293 INFO:tasks.ceph.cluster2.osd.1.smithi130.stderr:2024-09-12T12:56:05.293+0000 7fcf52600640 -1 osd.1 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T12:56:05.304 INFO:tasks.ceph.cluster2.osd.0.smithi130.stderr:2024-09-12T12:56:05.304+0000 7f9ad4800640 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T12:56:09.422 DEBUG:teuthology.orchestra.run.smithi130:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster cluster2 osd dump --format=json 2024-09-12T12:56:09.802 INFO:teuthology.misc.health.smithi130.stdout: 2024-09-12T12:56:09.802 INFO:teuthology.misc.health.smithi130.stdout:{"epoch":11,"fsid":"76d48f4f-9f5b-4bde-b048-7f763396fe3f","created":"2024-09-12T12:55:54.555731+0000","modified":"2024-09-12T12:56:09.259458+0000","last_up_change":"2024-09-12T12:56:06.254589+0000","last_in_change":"2024-09-12T12:55:55.652531+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T12:56:07.548658+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"14760318-3ea5-426d-949a-87b78b1c5170","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6800","nonce":373021749},{"type":"v1","addr":"172.21.15.130:6801","nonce":373021749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6802","nonce":373021749},{"type":"v1","addr":"172.21.15.130:6803","nonce":373021749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6806","nonce":373021749},{"type":"v1","addr":"172.21.15.130:6807","nonce":373021749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6804","nonce":373021749},{"type":"v1","addr":"172.21.15.130:6805","nonce":373021749}]},"public_addr":"172.21.15.130:6801/373021749","cluster_addr":"172.21.15.130:6803/373021749","heartbeat_back_addr":"172.21.15.130:6807/373021749","heartbeat_front_addr":"172.21.15.130:6805/373021749","state":["exists","up"]},{"osd":1,"uuid":"5129799d-ce0e-4fbc-9bcc-2528285fa983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":10,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6808","nonce":1890016022},{"type":"v1","addr":"172.21.15.130:6809","nonce":1890016022}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6810","nonce":1890016022},{"type":"v1","addr":"172.21.15.130:6811","nonce":1890016022}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6817","nonce":1890016022},{"type":"v1","addr":"172.21.15.130:6819","nonce":1890016022}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6813","nonce":1890016022},{"type":"v1","addr":"172.21.15.130:6815","nonce":1890016022}]},"public_addr":"172.21.15.130:6809/1890016022","cluster_addr":"172.21.15.130:6811/1890016022","heartbeat_back_addr":"172.21.15.130:6819/1890016022","heartbeat_front_addr":"172.21.15.130:6815/1890016022","state":["exists","up"]},{"osd":2,"uuid":"3321b514-7dac-4085-9fe1-55714a27c720","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6812","nonce":2246267968},{"type":"v1","addr":"172.21.15.130:6814","nonce":2246267968}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6816","nonce":2246267968},{"type":"v1","addr":"172.21.15.130:6818","nonce":2246267968}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6822","nonce":2246267968},{"type":"v1","addr":"172.21.15.130:6823","nonce":2246267968}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6820","nonce":2246267968},{"type":"v1","addr":"172.21.15.130:6821","nonce":2246267968}]},"public_addr":"172.21.15.130:6814/2246267968","cluster_addr":"172.21.15.130:6818/2246267968","heartbeat_back_addr":"172.21.15.130:6823/2246267968","heartbeat_front_addr":"172.21.15.130:6821/2246267968","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:56:03.468853+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:56:03.255425+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:56:05.253102+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T12:56:09.817 DEBUG:teuthology.misc:3 of 3 OSDs are up 2024-09-12T12:56:09.817 INFO:tasks.ceph:Creating RBD pool 2024-09-12T12:56:09.817 DEBUG:teuthology.orchestra.run.smithi130:> sudo ceph --cluster cluster2 osd pool create rbd 8 2024-09-12T12:56:10.279 INFO:teuthology.orchestra.run.smithi130.stderr:pool 'rbd' created 2024-09-12T12:56:10.304 DEBUG:teuthology.orchestra.run.smithi130:> rbd --cluster cluster2 pool init rbd 2024-09-12T12:56:13.311 INFO:tasks.ceph:Starting mds daemons in cluster cluster2... 2024-09-12T12:56:13.312 INFO:tasks.ceph:Waiting until cluster2 daemons up and pgs clean... 2024-09-12T12:56:13.312 INFO:tasks.ceph.ceph_manager.cluster2:waiting for mgr available 2024-09-12T12:56:13.312 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster2 mgr dump --format=json 2024-09-12T12:56:13.313 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-09-12T12:56:13.706 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-09-12T12:56:13.722 INFO:teuthology.orchestra.run.smithi130.stdout:{"epoch":5,"flags":0,"active_gid":4101,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6824","nonce":3691954150},{"type":"v1","addr":"172.21.15.130:6825","nonce":3691954150}]},"active_addr":"172.21.15.130:6825/3691954150","active_change":"2024-09-12T12:55:59.485900+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.130:0","nonce":2166618080}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.130:0","nonce":4065063997}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.130:0","nonce":1416358012}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.130:0","nonce":1432800581}]}]} 2024-09-12T12:56:13.725 INFO:tasks.ceph.ceph_manager.cluster2:mgr available! 2024-09-12T12:56:13.725 INFO:tasks.ceph.ceph_manager.cluster2:waiting for all up 2024-09-12T12:56:13.726 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster2 osd dump --format=json 2024-09-12T12:56:14.067 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-09-12T12:56:14.067 INFO:teuthology.orchestra.run.smithi130.stdout:{"epoch":15,"fsid":"76d48f4f-9f5b-4bde-b048-7f763396fe3f","created":"2024-09-12T12:55:54.555731+0000","modified":"2024-09-12T12:56:13.288371+0000","last_up_change":"2024-09-12T12:56:06.254589+0000","last_in_change":"2024-09-12T12:55:55.652531+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T12:56:07.548658+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-12T12:56:10.277057+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":15,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"14760318-3ea5-426d-949a-87b78b1c5170","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6800","nonce":373021749},{"type":"v1","addr":"172.21.15.130:6801","nonce":373021749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6802","nonce":373021749},{"type":"v1","addr":"172.21.15.130:6803","nonce":373021749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6806","nonce":373021749},{"type":"v1","addr":"172.21.15.130:6807","nonce":373021749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6804","nonce":373021749},{"type":"v1","addr":"172.21.15.130:6805","nonce":373021749}]},"public_addr":"172.21.15.130:6801/373021749","cluster_addr":"172.21.15.130:6803/373021749","heartbeat_back_addr":"172.21.15.130:6807/373021749","heartbeat_front_addr":"172.21.15.130:6805/373021749","state":["exists","up"]},{"osd":1,"uuid":"5129799d-ce0e-4fbc-9bcc-2528285fa983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6808","nonce":1890016022},{"type":"v1","addr":"172.21.15.130:6809","nonce":1890016022}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6810","nonce":1890016022},{"type":"v1","addr":"172.21.15.130:6811","nonce":1890016022}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6817","nonce":1890016022},{"type":"v1","addr":"172.21.15.130:6819","nonce":1890016022}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6813","nonce":1890016022},{"type":"v1","addr":"172.21.15.130:6815","nonce":1890016022}]},"public_addr":"172.21.15.130:6809/1890016022","cluster_addr":"172.21.15.130:6811/1890016022","heartbeat_back_addr":"172.21.15.130:6819/1890016022","heartbeat_front_addr":"172.21.15.130:6815/1890016022","state":["exists","up"]},{"osd":2,"uuid":"3321b514-7dac-4085-9fe1-55714a27c720","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6812","nonce":2246267968},{"type":"v1","addr":"172.21.15.130:6814","nonce":2246267968}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6816","nonce":2246267968},{"type":"v1","addr":"172.21.15.130:6818","nonce":2246267968}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6822","nonce":2246267968},{"type":"v1","addr":"172.21.15.130:6823","nonce":2246267968}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6820","nonce":2246267968},{"type":"v1","addr":"172.21.15.130:6821","nonce":2246267968}]},"public_addr":"172.21.15.130:6814/2246267968","cluster_addr":"172.21.15.130:6818/2246267968","heartbeat_back_addr":"172.21.15.130:6823/2246267968","heartbeat_front_addr":"172.21.15.130:6821/2246267968","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:56:03.468853+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:56:03.255425+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:56:05.253102+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T12:56:14.085 INFO:tasks.ceph.ceph_manager.cluster2:all up! 2024-09-12T12:56:14.085 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster2 osd dump --format=json 2024-09-12T12:56:14.428 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-09-12T12:56:14.428 INFO:teuthology.orchestra.run.smithi130.stdout:{"epoch":15,"fsid":"76d48f4f-9f5b-4bde-b048-7f763396fe3f","created":"2024-09-12T12:55:54.555731+0000","modified":"2024-09-12T12:56:13.288371+0000","last_up_change":"2024-09-12T12:56:06.254589+0000","last_in_change":"2024-09-12T12:55:55.652531+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T12:56:07.548658+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-12T12:56:10.277057+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":15,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"14760318-3ea5-426d-949a-87b78b1c5170","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6800","nonce":373021749},{"type":"v1","addr":"172.21.15.130:6801","nonce":373021749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6802","nonce":373021749},{"type":"v1","addr":"172.21.15.130:6803","nonce":373021749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6806","nonce":373021749},{"type":"v1","addr":"172.21.15.130:6807","nonce":373021749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6804","nonce":373021749},{"type":"v1","addr":"172.21.15.130:6805","nonce":373021749}]},"public_addr":"172.21.15.130:6801/373021749","cluster_addr":"172.21.15.130:6803/373021749","heartbeat_back_addr":"172.21.15.130:6807/373021749","heartbeat_front_addr":"172.21.15.130:6805/373021749","state":["exists","up"]},{"osd":1,"uuid":"5129799d-ce0e-4fbc-9bcc-2528285fa983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6808","nonce":1890016022},{"type":"v1","addr":"172.21.15.130:6809","nonce":1890016022}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6810","nonce":1890016022},{"type":"v1","addr":"172.21.15.130:6811","nonce":1890016022}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6817","nonce":1890016022},{"type":"v1","addr":"172.21.15.130:6819","nonce":1890016022}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6813","nonce":1890016022},{"type":"v1","addr":"172.21.15.130:6815","nonce":1890016022}]},"public_addr":"172.21.15.130:6809/1890016022","cluster_addr":"172.21.15.130:6811/1890016022","heartbeat_back_addr":"172.21.15.130:6819/1890016022","heartbeat_front_addr":"172.21.15.130:6815/1890016022","state":["exists","up"]},{"osd":2,"uuid":"3321b514-7dac-4085-9fe1-55714a27c720","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6812","nonce":2246267968},{"type":"v1","addr":"172.21.15.130:6814","nonce":2246267968}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6816","nonce":2246267968},{"type":"v1","addr":"172.21.15.130:6818","nonce":2246267968}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6822","nonce":2246267968},{"type":"v1","addr":"172.21.15.130:6823","nonce":2246267968}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6820","nonce":2246267968},{"type":"v1","addr":"172.21.15.130:6821","nonce":2246267968}]},"public_addr":"172.21.15.130:6814/2246267968","cluster_addr":"172.21.15.130:6818/2246267968","heartbeat_back_addr":"172.21.15.130:6823/2246267968","heartbeat_front_addr":"172.21.15.130:6821/2246267968","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:56:03.468853+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:56:03.255425+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:56:05.253102+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T12:56:14.446 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster2 tell osd.0 flush_pg_stats 2024-09-12T12:56:14.446 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster2 tell osd.1 flush_pg_stats 2024-09-12T12:56:14.446 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster2 tell osd.2 flush_pg_stats 2024-09-12T12:56:14.651 INFO:teuthology.orchestra.run.smithi130.stdout:34359738372 2024-09-12T12:56:14.651 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster2 osd last-stat-seq osd.1 2024-09-12T12:56:14.652 INFO:teuthology.orchestra.run.smithi130.stdout:34359738372 2024-09-12T12:56:14.653 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster2 osd last-stat-seq osd.0 2024-09-12T12:56:14.653 INFO:teuthology.orchestra.run.smithi130.stdout:34359738374 2024-09-12T12:56:14.654 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster2 osd last-stat-seq osd.2 2024-09-12T12:56:15.057 INFO:teuthology.orchestra.run.smithi130.stdout:34359738371 2024-09-12T12:56:15.059 INFO:teuthology.orchestra.run.smithi130.stdout:34359738373 2024-09-12T12:56:15.070 INFO:teuthology.orchestra.run.smithi130.stdout:34359738371 2024-09-12T12:56:15.078 INFO:tasks.ceph.ceph_manager.cluster2:need seq 34359738372 got 34359738371 for osd.0 2024-09-12T12:56:15.078 INFO:tasks.ceph.ceph_manager.cluster2:need seq 34359738374 got 34359738373 for osd.2 2024-09-12T12:56:15.089 INFO:tasks.ceph.ceph_manager.cluster2:need seq 34359738372 got 34359738371 for osd.1 2024-09-12T12:56:16.079 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster2 osd last-stat-seq osd.0 2024-09-12T12:56:16.079 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster2 osd last-stat-seq osd.2 2024-09-12T12:56:16.089 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster2 osd last-stat-seq osd.1 2024-09-12T12:56:16.451 INFO:teuthology.orchestra.run.smithi130.stdout:34359738374 2024-09-12T12:56:16.471 INFO:tasks.ceph.ceph_manager.cluster2:need seq 34359738374 got 34359738374 for osd.2 2024-09-12T12:56:16.471 INFO:teuthology.orchestra.run.smithi130.stdout:34359738372 2024-09-12T12:56:16.471 DEBUG:teuthology.parallel:result is None 2024-09-12T12:56:16.488 INFO:tasks.ceph.ceph_manager.cluster2:need seq 34359738372 got 34359738372 for osd.0 2024-09-12T12:56:16.488 DEBUG:teuthology.parallel:result is None 2024-09-12T12:56:16.505 INFO:teuthology.orchestra.run.smithi130.stdout:34359738372 2024-09-12T12:56:16.524 INFO:tasks.ceph.ceph_manager.cluster2:need seq 34359738372 got 34359738372 for osd.1 2024-09-12T12:56:16.524 DEBUG:teuthology.parallel:result is None 2024-09-12T12:56:16.524 INFO:tasks.ceph.ceph_manager.cluster2:waiting for clean 2024-09-12T12:56:16.525 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster2 pg dump --format=json 2024-09-12T12:56:16.900 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-09-12T12:56:16.900 INFO:teuthology.orchestra.run.smithi130.stderr:dumped all 2024-09-12T12:56:16.917 INFO:teuthology.orchestra.run.smithi130.stdout:{"pg_ready":true,"pg_map":{"version":18,"stamp":"2024-09-12T12:56:15.495464+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":139,"num_write_kb":1896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":100,"ondisk_log_size":100,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":17,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":82784,"kb_used_data":1552,"kb_used_omap":4,"kb_used_meta":81019,"kb_avail":283032736,"statfs":{"total":289910292480,"available":289825521664,"internally_reserved":0,"allocated":1589248,"data_stored":1294801,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4785,"internal_metadata":82963791},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.206249"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":19,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-12T12:56:13.297616+0000","last_change":"2024-09-12T12:56:13.297872+0000","last_active":"2024-09-12T12:56:13.297616+0000","last_peered":"2024-09-12T12:56:13.297616+0000","last_clean":"2024-09-12T12:56:13.297616+0000","last_became_active":"2024-09-12T12:56:11.291035+0000","last_became_peered":"2024-09-12T12:56:11.291035+0000","last_unstale":"2024-09-12T12:56:13.297616+0000","last_undegraded":"2024-09-12T12:56:13.297616+0000","last_fullsized":"2024-09-12T12:56:13.297616+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:56:10.277688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:56:10.277688+0000","last_clean_scrub_stamp":"2024-09-12T12:56:10.277688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:44:09.505949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001875938,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":19,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-12T12:56:13.297227+0000","last_change":"2024-09-12T12:56:13.297542+0000","last_active":"2024-09-12T12:56:13.297227+0000","last_peered":"2024-09-12T12:56:13.297227+0000","last_clean":"2024-09-12T12:56:13.297227+0000","last_became_active":"2024-09-12T12:56:11.294341+0000","last_became_peered":"2024-09-12T12:56:11.294341+0000","last_unstale":"2024-09-12T12:56:13.297227+0000","last_undegraded":"2024-09-12T12:56:13.297227+0000","last_fullsized":"2024-09-12T12:56:13.297227+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:56:10.277688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:56:10.277688+0000","last_clean_scrub_stamp":"2024-09-12T12:56:10.277688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:41:52.966407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016741390000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-12T12:56:13.297109+0000","last_change":"2024-09-12T12:56:13.297678+0000","last_active":"2024-09-12T12:56:13.297109+0000","last_peered":"2024-09-12T12:56:13.297109+0000","last_clean":"2024-09-12T12:56:13.297109+0000","last_became_active":"2024-09-12T12:56:11.294629+0000","last_became_peered":"2024-09-12T12:56:11.294629+0000","last_unstale":"2024-09-12T12:56:13.297109+0000","last_undegraded":"2024-09-12T12:56:13.297109+0000","last_fullsized":"2024-09-12T12:56:13.297109+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:56:10.277688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:56:10.277688+0000","last_clean_scrub_stamp":"2024-09-12T12:56:10.277688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:27:43.345132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018622739999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-12T12:56:13.296800+0000","last_change":"2024-09-12T12:56:13.297034+0000","last_active":"2024-09-12T12:56:13.296800+0000","last_peered":"2024-09-12T12:56:13.296800+0000","last_clean":"2024-09-12T12:56:13.296800+0000","last_became_active":"2024-09-12T12:56:11.291147+0000","last_became_peered":"2024-09-12T12:56:11.291147+0000","last_unstale":"2024-09-12T12:56:13.296800+0000","last_undegraded":"2024-09-12T12:56:13.296800+0000","last_fullsized":"2024-09-12T12:56:13.296800+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:56:10.277688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:56:10.277688+0000","last_clean_scrub_stamp":"2024-09-12T12:56:10.277688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:23:38.950871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001237943,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"15'2","reported_seq":21,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-12T12:56:13.292357+0000","last_change":"2024-09-12T12:56:13.292357+0000","last_active":"2024-09-12T12:56:13.292357+0000","last_peered":"2024-09-12T12:56:13.292357+0000","last_clean":"2024-09-12T12:56:13.292357+0000","last_became_active":"2024-09-12T12:56:11.295878+0000","last_became_peered":"2024-09-12T12:56:11.295878+0000","last_unstale":"2024-09-12T12:56:13.292357+0000","last_undegraded":"2024-09-12T12:56:13.292357+0000","last_fullsized":"2024-09-12T12:56:13.292357+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:56:10.277688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:56:10.277688+0000","last_clean_scrub_stamp":"2024-09-12T12:56:10.277688+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:53:37.770371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054639100000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":17,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-12T12:56:13.355124+0000","last_change":"2024-09-12T12:56:13.355367+0000","last_active":"2024-09-12T12:56:13.355124+0000","last_peered":"2024-09-12T12:56:13.355124+0000","last_clean":"2024-09-12T12:56:13.355124+0000","last_became_active":"2024-09-12T12:56:11.295181+0000","last_became_peered":"2024-09-12T12:56:11.295181+0000","last_unstale":"2024-09-12T12:56:13.355124+0000","last_undegraded":"2024-09-12T12:56:13.355124+0000","last_fullsized":"2024-09-12T12:56:13.355124+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:56:10.277688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:56:10.277688+0000","last_clean_scrub_stamp":"2024-09-12T12:56:10.277688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:20:30.198897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057238600000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":17,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-12T12:56:13.355079+0000","last_change":"2024-09-12T12:56:13.355375+0000","last_active":"2024-09-12T12:56:13.355079+0000","last_peered":"2024-09-12T12:56:13.355079+0000","last_clean":"2024-09-12T12:56:13.355079+0000","last_became_active":"2024-09-12T12:56:11.293181+0000","last_became_peered":"2024-09-12T12:56:11.293181+0000","last_unstale":"2024-09-12T12:56:13.355079+0000","last_undegraded":"2024-09-12T12:56:13.355079+0000","last_fullsized":"2024-09-12T12:56:13.355079+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:56:10.277688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:56:10.277688+0000","last_clean_scrub_stamp":"2024-09-12T12:56:10.277688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:41:47.856672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058228100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"13'1","reported_seq":20,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-12T12:56:13.296732+0000","last_change":"2024-09-12T12:56:13.297158+0000","last_active":"2024-09-12T12:56:13.296732+0000","last_peered":"2024-09-12T12:56:13.296732+0000","last_clean":"2024-09-12T12:56:13.296732+0000","last_became_active":"2024-09-12T12:56:11.290474+0000","last_became_peered":"2024-09-12T12:56:11.290474+0000","last_unstale":"2024-09-12T12:56:13.296732+0000","last_undegraded":"2024-09-12T12:56:13.296732+0000","last_fullsized":"2024-09-12T12:56:13.296732+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:56:10.277688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:56:10.277688+0000","last_clean_scrub_stamp":"2024-09-12T12:56:10.277688+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:29:42.607257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001374526,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"12'97","reported_seq":145,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-12T12:56:13.295304+0000","last_change":"2024-09-12T12:56:09.442875+0000","last_active":"2024-09-12T12:56:13.295304+0000","last_peered":"2024-09-12T12:56:13.295304+0000","last_clean":"2024-09-12T12:56:13.295304+0000","last_became_active":"2024-09-12T12:56:09.442486+0000","last_became_peered":"2024-09-12T12:56:09.442486+0000","last_unstale":"2024-09-12T12:56:13.295304+0000","last_undegraded":"2024-09-12T12:56:13.295304+0000","last_fullsized":"2024-09-12T12:56:13.295304+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:56:08.256849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:56:08.256849+0000","last_clean_scrub_stamp":"2024-09-12T12:56:08.256849+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T00:45:42.906851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738374,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27160,"kb_used_data":128,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344680,"statfs":{"total":96636764160,"available":96608952320,"internally_reserved":0,"allocated":131072,"data_stored":38009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":8,"seq":34359738372,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27744,"kb_used_data":712,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344096,"statfs":{"total":96636764160,"available":96608354304,"internally_reserved":0,"allocated":729088,"data_stored":628396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738372,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27880,"kb_used_data":712,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343960,"statfs":{"total":96636764160,"available":96608215040,"internally_reserved":0,"allocated":729088,"data_stored":628396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T12:56:16.918 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster2 pg dump --format=json 2024-09-12T12:56:17.257 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-09-12T12:56:17.257 INFO:teuthology.orchestra.run.smithi130.stderr:dumped all 2024-09-12T12:56:17.274 INFO:teuthology.orchestra.run.smithi130.stdout:{"pg_ready":true,"pg_map":{"version":18,"stamp":"2024-09-12T12:56:15.495464+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":139,"num_write_kb":1896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":100,"ondisk_log_size":100,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":17,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":82784,"kb_used_data":1552,"kb_used_omap":4,"kb_used_meta":81019,"kb_avail":283032736,"statfs":{"total":289910292480,"available":289825521664,"internally_reserved":0,"allocated":1589248,"data_stored":1294801,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4785,"internal_metadata":82963791},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.206249"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":19,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-12T12:56:13.297616+0000","last_change":"2024-09-12T12:56:13.297872+0000","last_active":"2024-09-12T12:56:13.297616+0000","last_peered":"2024-09-12T12:56:13.297616+0000","last_clean":"2024-09-12T12:56:13.297616+0000","last_became_active":"2024-09-12T12:56:11.291035+0000","last_became_peered":"2024-09-12T12:56:11.291035+0000","last_unstale":"2024-09-12T12:56:13.297616+0000","last_undegraded":"2024-09-12T12:56:13.297616+0000","last_fullsized":"2024-09-12T12:56:13.297616+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:56:10.277688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:56:10.277688+0000","last_clean_scrub_stamp":"2024-09-12T12:56:10.277688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:44:09.505949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001875938,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":19,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-12T12:56:13.297227+0000","last_change":"2024-09-12T12:56:13.297542+0000","last_active":"2024-09-12T12:56:13.297227+0000","last_peered":"2024-09-12T12:56:13.297227+0000","last_clean":"2024-09-12T12:56:13.297227+0000","last_became_active":"2024-09-12T12:56:11.294341+0000","last_became_peered":"2024-09-12T12:56:11.294341+0000","last_unstale":"2024-09-12T12:56:13.297227+0000","last_undegraded":"2024-09-12T12:56:13.297227+0000","last_fullsized":"2024-09-12T12:56:13.297227+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:56:10.277688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:56:10.277688+0000","last_clean_scrub_stamp":"2024-09-12T12:56:10.277688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:41:52.966407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016741390000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-12T12:56:13.297109+0000","last_change":"2024-09-12T12:56:13.297678+0000","last_active":"2024-09-12T12:56:13.297109+0000","last_peered":"2024-09-12T12:56:13.297109+0000","last_clean":"2024-09-12T12:56:13.297109+0000","last_became_active":"2024-09-12T12:56:11.294629+0000","last_became_peered":"2024-09-12T12:56:11.294629+0000","last_unstale":"2024-09-12T12:56:13.297109+0000","last_undegraded":"2024-09-12T12:56:13.297109+0000","last_fullsized":"2024-09-12T12:56:13.297109+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:56:10.277688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:56:10.277688+0000","last_clean_scrub_stamp":"2024-09-12T12:56:10.277688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:27:43.345132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018622739999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-12T12:56:13.296800+0000","last_change":"2024-09-12T12:56:13.297034+0000","last_active":"2024-09-12T12:56:13.296800+0000","last_peered":"2024-09-12T12:56:13.296800+0000","last_clean":"2024-09-12T12:56:13.296800+0000","last_became_active":"2024-09-12T12:56:11.291147+0000","last_became_peered":"2024-09-12T12:56:11.291147+0000","last_unstale":"2024-09-12T12:56:13.296800+0000","last_undegraded":"2024-09-12T12:56:13.296800+0000","last_fullsized":"2024-09-12T12:56:13.296800+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:56:10.277688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:56:10.277688+0000","last_clean_scrub_stamp":"2024-09-12T12:56:10.277688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:23:38.950871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001237943,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"15'2","reported_seq":21,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-12T12:56:13.292357+0000","last_change":"2024-09-12T12:56:13.292357+0000","last_active":"2024-09-12T12:56:13.292357+0000","last_peered":"2024-09-12T12:56:13.292357+0000","last_clean":"2024-09-12T12:56:13.292357+0000","last_became_active":"2024-09-12T12:56:11.295878+0000","last_became_peered":"2024-09-12T12:56:11.295878+0000","last_unstale":"2024-09-12T12:56:13.292357+0000","last_undegraded":"2024-09-12T12:56:13.292357+0000","last_fullsized":"2024-09-12T12:56:13.292357+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:56:10.277688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:56:10.277688+0000","last_clean_scrub_stamp":"2024-09-12T12:56:10.277688+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:53:37.770371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054639100000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":17,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-12T12:56:13.355124+0000","last_change":"2024-09-12T12:56:13.355367+0000","last_active":"2024-09-12T12:56:13.355124+0000","last_peered":"2024-09-12T12:56:13.355124+0000","last_clean":"2024-09-12T12:56:13.355124+0000","last_became_active":"2024-09-12T12:56:11.295181+0000","last_became_peered":"2024-09-12T12:56:11.295181+0000","last_unstale":"2024-09-12T12:56:13.355124+0000","last_undegraded":"2024-09-12T12:56:13.355124+0000","last_fullsized":"2024-09-12T12:56:13.355124+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:56:10.277688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:56:10.277688+0000","last_clean_scrub_stamp":"2024-09-12T12:56:10.277688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:20:30.198897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057238600000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":17,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-12T12:56:13.355079+0000","last_change":"2024-09-12T12:56:13.355375+0000","last_active":"2024-09-12T12:56:13.355079+0000","last_peered":"2024-09-12T12:56:13.355079+0000","last_clean":"2024-09-12T12:56:13.355079+0000","last_became_active":"2024-09-12T12:56:11.293181+0000","last_became_peered":"2024-09-12T12:56:11.293181+0000","last_unstale":"2024-09-12T12:56:13.355079+0000","last_undegraded":"2024-09-12T12:56:13.355079+0000","last_fullsized":"2024-09-12T12:56:13.355079+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:56:10.277688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:56:10.277688+0000","last_clean_scrub_stamp":"2024-09-12T12:56:10.277688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:41:47.856672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058228100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"13'1","reported_seq":20,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-12T12:56:13.296732+0000","last_change":"2024-09-12T12:56:13.297158+0000","last_active":"2024-09-12T12:56:13.296732+0000","last_peered":"2024-09-12T12:56:13.296732+0000","last_clean":"2024-09-12T12:56:13.296732+0000","last_became_active":"2024-09-12T12:56:11.290474+0000","last_became_peered":"2024-09-12T12:56:11.290474+0000","last_unstale":"2024-09-12T12:56:13.296732+0000","last_undegraded":"2024-09-12T12:56:13.296732+0000","last_fullsized":"2024-09-12T12:56:13.296732+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:56:10.277688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:56:10.277688+0000","last_clean_scrub_stamp":"2024-09-12T12:56:10.277688+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:29:42.607257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001374526,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"12'97","reported_seq":145,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-12T12:56:13.295304+0000","last_change":"2024-09-12T12:56:09.442875+0000","last_active":"2024-09-12T12:56:13.295304+0000","last_peered":"2024-09-12T12:56:13.295304+0000","last_clean":"2024-09-12T12:56:13.295304+0000","last_became_active":"2024-09-12T12:56:09.442486+0000","last_became_peered":"2024-09-12T12:56:09.442486+0000","last_unstale":"2024-09-12T12:56:13.295304+0000","last_undegraded":"2024-09-12T12:56:13.295304+0000","last_fullsized":"2024-09-12T12:56:13.295304+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:56:08.256849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:56:08.256849+0000","last_clean_scrub_stamp":"2024-09-12T12:56:08.256849+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T00:45:42.906851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738374,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27160,"kb_used_data":128,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344680,"statfs":{"total":96636764160,"available":96608952320,"internally_reserved":0,"allocated":131072,"data_stored":38009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":8,"seq":34359738372,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27744,"kb_used_data":712,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344096,"statfs":{"total":96636764160,"available":96608354304,"internally_reserved":0,"allocated":729088,"data_stored":628396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738372,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27880,"kb_used_data":712,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343960,"statfs":{"total":96636764160,"available":96608215040,"internally_reserved":0,"allocated":729088,"data_stored":628396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T12:56:17.275 INFO:tasks.ceph.ceph_manager.cluster2:clean! 2024-09-12T12:56:17.275 INFO:tasks.ceph:Waiting until ceph cluster cluster2 is healthy... 2024-09-12T12:56:17.275 INFO:tasks.ceph.ceph_manager.cluster2:wait_until_healthy 2024-09-12T12:56:17.276 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster2 health --format=json 2024-09-12T12:56:17.650 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-09-12T12:56:17.650 INFO:teuthology.orchestra.run.smithi130.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-12T12:56:17.669 INFO:tasks.ceph.ceph_manager.cluster2:wait_until_healthy done 2024-09-12T12:56:17.669 INFO:teuthology.run_tasks:Running task rbd-mirror... 2024-09-12T12:56:17.682 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2024-09-12T12:56:17.682 DEBUG:teuthology.orchestra.run.smithi130:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill rbd-mirror --foreground --cluster cluster1 --id mirror.0 2024-09-12T12:56:17.686 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2024-09-12T12:56:17.687 INFO:teuthology.run_tasks:Running task rbd-mirror... 2024-09-12T12:56:17.695 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2024-09-12T12:56:17.695 DEBUG:teuthology.orchestra.run.smithi130:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill rbd-mirror --foreground --cluster cluster2 --id mirror.0 2024-09-12T12:56:17.698 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2024-09-12T12:56:17.699 INFO:teuthology.run_tasks:Running task workunit... 2024-09-12T12:56:17.718 INFO:tasks.workunit:Pulling workunits from ref 086c737fa75c22d521a3932c59b31c5e1d1c9ef4 2024-09-12T12:56:17.719 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-09-12T12:56:17.719 DEBUG:teuthology.orchestra.run.smithi130:> stat -- /home/ubuntu/cephtest/mnt.cluster1.mirror 2024-09-12T12:56:17.754 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T12:56:17.754 INFO:teuthology.orchestra.run.smithi130.stderr:stat: cannot statx '/home/ubuntu/cephtest/mnt.cluster1.mirror': No such file or directory 2024-09-12T12:56:17.755 DEBUG:teuthology.orchestra.run.smithi130:> mkdir -- /home/ubuntu/cephtest/mnt.cluster1.mirror 2024-09-12T12:56:17.834 INFO:tasks.workunit:Created dir /home/ubuntu/cephtest/mnt.cluster1.mirror 2024-09-12T12:56:17.834 DEBUG:teuthology.orchestra.run.smithi130:> cd -- /home/ubuntu/cephtest/mnt.cluster1.mirror && mkdir -- client.mirror 2024-09-12T12:56:17.909 INFO:tasks.workunit:timeout=3h 2024-09-12T12:56:17.909 INFO:tasks.workunit:cleanup=True 2024-09-12T12:56:17.909 DEBUG:teuthology.orchestra.run.smithi130:> rm -rf /home/ubuntu/cephtest/clone.cluster1.client.mirror && git clone https://git.ceph.com/ceph.git /home/ubuntu/cephtest/clone.cluster1.client.mirror && cd /home/ubuntu/cephtest/clone.cluster1.client.mirror && git checkout 086c737fa75c22d521a3932c59b31c5e1d1c9ef4 2024-09-12T12:56:17.986 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:Cloning into '/home/ubuntu/cephtest/clone.cluster1.client.mirror'... 2024-09-12T12:57:20.516 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:Updating files: 66% (8550/12903) Updating files: 67% (8646/12903) Updating files: 68% (8775/12903) Updating files: 69% (8904/12903) Updating files: 70% (9033/12903) Updating files: 71% (9162/12903) Updating files: 72% (9291/12903) Updating files: 73% (9420/12903) Updating files: 74% (9549/12903) Updating files: 75% (9678/12903) Updating files: 76% (9807/12903) Updating files: 77% (9936/12903) Updating files: 78% (10065/12903) Updating files: 79% (10194/12903) Updating files: 80% (10323/12903) Updating files: 81% (10452/12903) Updating files: 82% (10581/12903) Updating files: 83% (10710/12903) Updating files: 84% (10839/12903) Updating files: 85% (10968/12903) Updating files: 86% (11097/12903) Updating files: 87% (11226/12903) Updating files: 88% (11355/12903) Updating files: 89% (11484/12903) Updating files: 90% (11613/12903) Updating files: 91% (11742/12903) Updating files: 92% (11871/12903) Updating files: 93% (12000/12903) Updating files: 94% (12129/12903) Updating files: 95% (12258/12903) Updating files: 96% (12387/12903) Updating files: 97% (12516/12903) Updating files: 98% (12645/12903) Updating files: 99% (12774/12903) Updating files: 100% (12903/12903) Updating files: 100% (12903/12903), done. 2024-09-12T12:57:21.379 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:Note: switching to '086c737fa75c22d521a3932c59b31c5e1d1c9ef4'. 2024-09-12T12:57:21.379 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 2024-09-12T12:57:21.379 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-09-12T12:57:21.379 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:changes and commit them, and you can discard any commits you make in this 2024-09-12T12:57:21.379 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:state without impacting any branches by switching back to a branch. 2024-09-12T12:57:21.379 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 2024-09-12T12:57:21.379 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:If you want to create a new branch to retain commits you create, you may 2024-09-12T12:57:21.380 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:do so (now or later) by using -c with the switch command. Example: 2024-09-12T12:57:21.380 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 2024-09-12T12:57:21.380 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: git switch -c 2024-09-12T12:57:21.380 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 2024-09-12T12:57:21.380 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:Or undo this operation with: 2024-09-12T12:57:21.380 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 2024-09-12T12:57:21.380 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: git switch - 2024-09-12T12:57:21.380 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 2024-09-12T12:57:21.380 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-09-12T12:57:21.380 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 2024-09-12T12:57:21.380 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:HEAD is now at 086c737fa75 Merge pull request #59733 from zdover23/wip-doc-2024-09-11-backport-59635-to-squid 2024-09-12T12:57:21.393 DEBUG:teuthology.orchestra.run.smithi130:> cd -- /home/ubuntu/cephtest/clone.cluster1.client.mirror/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.cluster1.client.mirror 2024-09-12T12:57:21.533 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-09-12T12:57:21.538 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.cluster1.client.mirror/qa/workunits/direct_io' 2024-09-12T12:57:21.538 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-09-12T12:57:22.228 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-09-12T12:57:22.335 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-09-12T12:57:22.400 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.cluster1.client.mirror/qa/workunits/direct_io' 2024-09-12T12:57:22.404 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.cluster1.client.mirror/qa/workunits/fs' 2024-09-12T12:57:22.404 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-09-12T12:57:22.468 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.cluster1.client.mirror/qa/workunits/fs' 2024-09-12T12:57:22.475 DEBUG:teuthology.orchestra.run.smithi130:> set -ex 2024-09-12T12:57:22.475 DEBUG:teuthology.orchestra.run.smithi130:> dd if=/home/ubuntu/cephtest/workunits.list.cluster1.client.mirror of=/dev/stdout 2024-09-12T12:57:22.511 INFO:tasks.workunit:Running workunits matching rbd/compare_mirror_images.sh on cluster1.client.mirror... 2024-09-12T12:57:22.512 INFO:tasks.workunit:Running workunit rbd/compare_mirror_images.sh... 2024-09-12T12:57:22.512 DEBUG:teuthology.orchestra.run.smithi130:workunit test rbd/compare_mirror_images.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.cluster1.mirror/client.mirror/tmp && cd -- /home/ubuntu/cephtest/mnt.cluster1.mirror/client.mirror/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=086c737fa75c22d521a3932c59b31c5e1d1c9ef4 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster cluster1" CEPH_ID="mirror" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.cluster1.client.mirror CEPH_ROOT=/home/ubuntu/cephtest/clone.cluster1.client.mirror CEPH_MNT=/home/ubuntu/cephtest/mnt.cluster1.mirror RBD_DEVICE_TYPE=krbd RBD_MIRROR_USE_RBD_MIRROR=1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.cluster1.client.mirror/qa/workunits/rbd/compare_mirror_images.sh 2024-09-12T12:57:22.597 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ IMG_PREFIX=image-primary 2024-09-12T12:57:22.597 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ MIRROR_IMAGE_MODE=snapshot 2024-09-12T12:57:22.597 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ MIRROR_POOL_MODE=image 2024-09-12T12:57:22.598 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ MNTPT_PREFIX=test-primary 2024-09-12T12:57:22.598 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ RBD_IMAGE_FEATURES=layering,exclusive-lock,object-map,fast-diff 2024-09-12T12:57:22.598 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ RBD_MIRROR_INSTANCES=1 2024-09-12T12:57:22.598 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ RBD_MIRROR_MODE=snapshot 2024-09-12T12:57:22.598 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ RBD_MIRROR_USE_EXISTING_CLUSTER=1 2024-09-12T12:57:22.598 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ dirname /home/ubuntu/cephtest/clone.cluster1.client.mirror/qa/workunits/rbd/compare_mirror_images.sh 2024-09-12T12:57:22.599 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ . /home/ubuntu/cephtest/clone.cluster1.client.mirror/qa/workunits/rbd/rbd_mirror_helpers.sh 2024-09-12T12:57:22.599 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ type xmlstarlet 2024-09-12T12:57:22.600 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ XMLSTARLET=xmlstarlet 2024-09-12T12:57:22.600 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ RBD_MIRROR_INSTANCES=1 2024-09-12T12:57:22.601 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CLUSTER1=cluster1 2024-09-12T12:57:22.601 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CLUSTER2=cluster2 2024-09-12T12:57:22.601 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ PEER_CLUSTER_SUFFIX= 2024-09-12T12:57:22.601 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ POOL=mirror 2024-09-12T12:57:22.601 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ PARENT_POOL=mirror_parent 2024-09-12T12:57:22.601 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ NS1=ns1 2024-09-12T12:57:22.601 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ NS2=ns2 2024-09-12T12:57:22.601 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ TEMPDIR= 2024-09-12T12:57:22.601 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ID=mirror 2024-09-12T12:57:22.601 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ RBD_IMAGE_FEATURES=layering,exclusive-lock,object-map,fast-diff 2024-09-12T12:57:22.601 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ MIRROR_USER_ID_PREFIX=mirror. 2024-09-12T12:57:22.601 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ RBD_MIRROR_MODE=snapshot 2024-09-12T12:57:22.602 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ MIRROR_POOL_MODE=image 2024-09-12T12:57:22.602 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ '[' snapshot = snapshot ']' 2024-09-12T12:57:22.602 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ MIRROR_POOL_MODE=image 2024-09-12T12:57:22.602 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ export 'CEPH_ARGS=--id mirror' 2024-09-12T12:57:22.602 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS='--id mirror' 2024-09-12T12:57:22.602 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ LAST_MIRROR_INSTANCE=0 2024-09-12T12:57:22.602 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++++ dirname /home/ubuntu/cephtest/clone.cluster1.client.mirror/qa/workunits/rbd/compare_mirror_images.sh 2024-09-12T12:57:22.603 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+++ readlink -f /home/ubuntu/cephtest/clone.cluster1.client.mirror/qa/workunits/rbd/../../../src 2024-09-12T12:57:22.604 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ROOT=/home/ubuntu/cephtest/clone.cluster1.client.mirror/src 2024-09-12T12:57:22.605 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_BIN=. 2024-09-12T12:57:22.605 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_SRC=. 2024-09-12T12:57:22.605 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ '[' -e CMakeCache.txt ']' 2024-09-12T12:57:22.607 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ '[' 0 -gt 0 ']' 2024-09-12T12:57:22.608 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ setup 2024-09-12T12:57:22.608 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local c 2024-09-12T12:57:22.608 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ trap 'cleanup $?' INT TERM EXIT 2024-09-12T12:57:22.608 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ setup_tempdir 2024-09-12T12:57:22.608 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T12:57:22.608 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ mktemp -d 2024-09-12T12:57:22.610 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ TEMPDIR=/tmp/tmp.PVdhy5vDz3 2024-09-12T12:57:22.610 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -z 1 ']' 2024-09-12T12:57:22.610 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ setup_pools cluster1 cluster2 2024-09-12T12:57:22.611 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:57:22.611 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local remote_cluster=cluster2 2024-09-12T12:57:22.611 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mon_map_file 2024-09-12T12:57:22.611 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mon_addr 2024-09-12T12:57:22.611 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local admin_key_file 2024-09-12T12:57:22.611 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local uuid 2024-09-12T12:57:22.611 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ CEPH_ARGS= 2024-09-12T12:57:22.611 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ ceph --cluster cluster1 osd pool create mirror 64 64 2024-09-12T12:57:23.911 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:pool 'mirror' already exists 2024-09-12T12:57:23.927 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ CEPH_ARGS= 2024-09-12T12:57:23.927 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ ceph --cluster cluster1 osd pool create mirror_parent 64 64 2024-09-12T12:57:24.916 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:pool 'mirror_parent' already exists 2024-09-12T12:57:24.930 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ CEPH_ARGS= 2024-09-12T12:57:24.930 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 pool init mirror 2024-09-12T12:57:27.838 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ CEPH_ARGS= 2024-09-12T12:57:27.839 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 pool init mirror_parent 2024-09-12T12:57:30.859 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T12:57:30.860 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ CEPH_ARGS= 2024-09-12T12:57:30.860 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror pool enable --site-name cluster1 mirror image 2024-09-12T12:57:30.923 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror pool enable mirror_parent image 2024-09-12T12:57:30.980 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 namespace create mirror/ns1 2024-09-12T12:57:31.036 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 namespace create mirror/ns2 2024-09-12T12:57:31.089 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror pool enable mirror/ns1 image 2024-09-12T12:57:31.147 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror pool enable mirror/ns2 image 2024-09-12T12:57:31.204 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -z ']' 2024-09-12T12:57:31.204 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -z ']' 2024-09-12T12:57:31.204 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ peer_add cluster1 mirror cluster2 2024-09-12T12:57:31.204 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:57:31.204 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T12:57:31.204 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:57:31.205 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T12:57:31.205 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local client_cluster=cluster2 2024-09-12T12:57:31.205 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T12:57:31.205 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local remote_cluster=cluster2 2024-09-12T12:57:31.205 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local uuid_var_name 2024-09-12T12:57:31.205 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T12:57:31.205 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local error_code 2024-09-12T12:57:31.205 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local peer_uuid 2024-09-12T12:57:31.205 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 16 32 2024-09-12T12:57:31.206 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ set +e 2024-09-12T12:57:31.206 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror pool peer add mirror cluster2 2024-09-12T12:57:31.258 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ peer_uuid=c27bc53f-c0e2-4dbe-abb3-51479180ef0b 2024-09-12T12:57:31.259 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ error_code=0 2024-09-12T12:57:31.259 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ set -e 2024-09-12T12:57:31.259 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' 0 -eq 17 ']' 2024-09-12T12:57:31.259 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test 0 -eq 0 2024-09-12T12:57:31.259 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T12:57:31.259 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T12:57:31.259 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ peer_add cluster1 mirror_parent cluster2 2024-09-12T12:57:31.259 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:57:31.259 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T12:57:31.259 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror_parent 2024-09-12T12:57:31.259 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T12:57:31.259 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local client_cluster=cluster2 2024-09-12T12:57:31.259 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T12:57:31.260 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local remote_cluster=cluster2 2024-09-12T12:57:31.260 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local uuid_var_name 2024-09-12T12:57:31.260 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T12:57:31.260 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local error_code 2024-09-12T12:57:31.260 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local peer_uuid 2024-09-12T12:57:31.260 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 16 32 2024-09-12T12:57:31.260 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ set +e 2024-09-12T12:57:31.260 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror pool peer add mirror_parent cluster2 2024-09-12T12:57:31.313 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ peer_uuid=bd139086-ae14-4b35-a082-d021fc7cb86e 2024-09-12T12:57:31.313 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ error_code=0 2024-09-12T12:57:31.313 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ set -e 2024-09-12T12:57:31.314 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' 0 -eq 17 ']' 2024-09-12T12:57:31.314 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test 0 -eq 0 2024-09-12T12:57:31.314 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T12:57:31.315 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T12:57:31.315 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ setup_pools cluster2 cluster1 2024-09-12T12:57:31.315 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T12:57:31.315 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local remote_cluster=cluster1 2024-09-12T12:57:31.315 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mon_map_file 2024-09-12T12:57:31.315 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mon_addr 2024-09-12T12:57:31.315 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local admin_key_file 2024-09-12T12:57:31.315 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local uuid 2024-09-12T12:57:31.315 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ CEPH_ARGS= 2024-09-12T12:57:31.315 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ ceph --cluster cluster2 osd pool create mirror 64 64 2024-09-12T12:57:31.802 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:pool 'mirror' already exists 2024-09-12T12:57:31.817 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ CEPH_ARGS= 2024-09-12T12:57:31.817 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ ceph --cluster cluster2 osd pool create mirror_parent 64 64 2024-09-12T12:57:32.828 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:pool 'mirror_parent' already exists 2024-09-12T12:57:32.843 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ CEPH_ARGS= 2024-09-12T12:57:32.843 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster2 pool init mirror 2024-09-12T12:57:35.740 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ CEPH_ARGS= 2024-09-12T12:57:35.740 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster2 pool init mirror_parent 2024-09-12T12:57:38.754 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T12:57:38.754 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ CEPH_ARGS= 2024-09-12T12:57:38.755 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster2 mirror pool enable --site-name cluster2 mirror image 2024-09-12T12:57:38.815 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster2 mirror pool enable mirror_parent image 2024-09-12T12:57:38.870 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster2 namespace create mirror/ns1 2024-09-12T12:57:38.923 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster2 namespace create mirror/ns2 2024-09-12T12:57:38.975 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster2 mirror pool enable mirror/ns1 image 2024-09-12T12:57:39.029 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster2 mirror pool enable mirror/ns2 image 2024-09-12T12:57:39.083 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -z ']' 2024-09-12T12:57:39.083 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -z ']' 2024-09-12T12:57:39.083 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ peer_add cluster2 mirror cluster1 2024-09-12T12:57:39.083 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T12:57:39.084 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T12:57:39.084 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:57:39.084 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T12:57:39.084 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local client_cluster=cluster1 2024-09-12T12:57:39.084 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T12:57:39.084 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local remote_cluster=cluster1 2024-09-12T12:57:39.084 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local uuid_var_name 2024-09-12T12:57:39.084 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T12:57:39.084 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local error_code 2024-09-12T12:57:39.084 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local peer_uuid 2024-09-12T12:57:39.084 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 16 32 2024-09-12T12:57:39.084 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ set +e 2024-09-12T12:57:39.084 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster2 mirror pool peer add mirror cluster1 2024-09-12T12:57:39.137 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ peer_uuid=bd1f1a8d-bab5-4d8f-b271-a799adf9494b 2024-09-12T12:57:39.137 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ error_code=0 2024-09-12T12:57:39.137 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ set -e 2024-09-12T12:57:39.137 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' 0 -eq 17 ']' 2024-09-12T12:57:39.137 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test 0 -eq 0 2024-09-12T12:57:39.137 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T12:57:39.137 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T12:57:39.137 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ peer_add cluster2 mirror_parent cluster1 2024-09-12T12:57:39.137 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T12:57:39.137 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T12:57:39.138 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror_parent 2024-09-12T12:57:39.138 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T12:57:39.138 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local client_cluster=cluster1 2024-09-12T12:57:39.138 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T12:57:39.138 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local remote_cluster=cluster1 2024-09-12T12:57:39.138 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local uuid_var_name 2024-09-12T12:57:39.138 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T12:57:39.138 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local error_code 2024-09-12T12:57:39.138 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local peer_uuid 2024-09-12T12:57:39.138 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 16 32 2024-09-12T12:57:39.138 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ set +e 2024-09-12T12:57:39.138 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster2 mirror pool peer add mirror_parent cluster1 2024-09-12T12:57:39.190 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ peer_uuid=74c0e84b-7a01-4376-a2c4-077ff8f719a1 2024-09-12T12:57:39.190 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ error_code=0 2024-09-12T12:57:39.190 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ set -e 2024-09-12T12:57:39.190 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' 0 -eq 17 ']' 2024-09-12T12:57:39.190 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test 0 -eq 0 2024-09-12T12:57:39.190 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T12:57:39.190 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T12:57:39.190 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T12:57:39.190 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ start_mirrors cluster1 2024-09-12T12:57:39.190 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:57:39.191 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ seq 0 0 2024-09-12T12:57:39.201 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-09-12T12:57:39.201 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ start_mirror cluster1:0 2024-09-12T12:57:39.201 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1:0 2024-09-12T12:57:39.201 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local instance 2024-09-12T12:57:39.201 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-09-12T12:57:39.201 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local val=cluster1:0 2024-09-12T12:57:39.201 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster_var_name=cluster 2024-09-12T12:57:39.201 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local instance_var_name=instance 2024-09-12T12:57:39.201 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ cluster=cluster1 2024-09-12T12:57:39.202 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ instance=0 2024-09-12T12:57:39.202 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' 0 = cluster1:0 ']' 2024-09-12T12:57:39.202 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ eval cluster=cluster1 2024-09-12T12:57:39.202 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ cluster=cluster1 2024-09-12T12:57:39.202 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ eval instance=0 2024-09-12T12:57:39.202 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ instance=0 2024-09-12T12:57:39.202 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test -n 1 2024-09-12T12:57:39.202 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 2024-09-12T12:57:39.202 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ start_mirrors cluster2 2024-09-12T12:57:39.202 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T12:57:39.202 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ seq 0 0 2024-09-12T12:57:39.203 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-09-12T12:57:39.203 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ start_mirror cluster2:0 2024-09-12T12:57:39.204 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2:0 2024-09-12T12:57:39.204 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local instance 2024-09-12T12:57:39.204 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ set_cluster_instance cluster2:0 cluster instance 2024-09-12T12:57:39.204 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local val=cluster2:0 2024-09-12T12:57:39.204 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster_var_name=cluster 2024-09-12T12:57:39.204 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local instance_var_name=instance 2024-09-12T12:57:39.204 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ cluster=cluster2 2024-09-12T12:57:39.204 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ instance=0 2024-09-12T12:57:39.204 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' 0 = cluster2:0 ']' 2024-09-12T12:57:39.204 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ eval cluster=cluster2 2024-09-12T12:57:39.204 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ cluster=cluster2 2024-09-12T12:57:39.205 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ eval instance=0 2024-09-12T12:57:39.205 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ instance=0 2024-09-12T12:57:39.205 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test -n 1 2024-09-12T12:57:39.205 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 2024-09-12T12:57:39.205 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wget https://download.ceph.com/qa/linux-5.4.tar.gz 2024-09-12T12:57:39.228 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:--2024-09-12 12:57:39-- https://download.ceph.com/qa/linux-5.4.tar.gz 2024-09-12T12:57:39.418 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:Resolving download.ceph.com (download.ceph.com)... 158.69.68.124, 2607:5300:201:2000::3:58a1 2024-09-12T12:57:39.439 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:Connecting to download.ceph.com (download.ceph.com)|158.69.68.124|:443... connected. 2024-09-12T12:57:39.554 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:HTTP request sent, awaiting response... 200 OK 2024-09-12T12:57:39.555 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:Length: 172616875 (165M) [application/octet-stream] 2024-09-12T12:57:39.555 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:Saving to: ‘linux-5.4.tar.gz’ 2024-09-12T12:57:39.555 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 2024-09-12T12:57:39.574 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 0K .......... .......... .......... .......... .......... 0% 2.39M 69s 2024-09-12T12:57:39.596 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 50K .......... .......... .......... .......... .......... 0% 2.39M 69s 2024-09-12T12:57:39.596 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 100K .......... .......... .......... .......... .......... 0% 199M 46s 2024-09-12T12:57:39.618 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 150K .......... .......... .......... .......... .......... 0% 2.46M 51s 2024-09-12T12:57:39.618 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 200K .......... .......... .......... .......... .......... 0% 113M 41s 2024-09-12T12:57:39.618 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 250K .......... .......... .......... .......... .......... 0% 217M 35s 2024-09-12T12:57:39.641 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 300K .......... .......... .......... .......... .......... 0% 2.49M 39s 2024-09-12T12:57:39.641 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 350K .......... .......... .......... .......... .......... 0% 91.2M 34s 2024-09-12T12:57:39.641 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 400K .......... .......... .......... .......... .......... 0% 202M 31s 2024-09-12T12:57:39.641 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 450K .......... .......... .......... .......... .......... 0% 194M 28s 2024-09-12T12:57:39.641 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 500K .......... .......... .......... .......... .......... 0% 164M 25s 2024-09-12T12:57:39.641 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 550K .......... .......... .......... .......... .......... 0% 183M 23s 2024-09-12T12:57:39.641 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 600K .......... .......... .......... .......... .......... 0% 178M 21s 2024-09-12T12:57:39.665 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 650K .......... .......... .......... .......... .......... 0% 2.57M 24s 2024-09-12T12:57:39.665 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 700K .......... .......... .......... .......... .......... 0% 146M 23s 2024-09-12T12:57:39.665 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 750K .......... .......... .......... .......... .......... 0% 178M 22s 2024-09-12T12:57:39.665 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 800K .......... .......... .......... .......... .......... 0% 214M 20s 2024-09-12T12:57:39.665 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 850K .......... .......... .......... .......... .......... 0% 216M 19s 2024-09-12T12:57:39.665 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 900K .......... .......... .......... .......... .......... 0% 191M 18s 2024-09-12T12:57:39.665 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 950K .......... .......... .......... .......... .......... 0% 68.9M 17s 2024-09-12T12:57:39.666 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 1000K .......... .......... .......... .......... .......... 0% 140M 17s 2024-09-12T12:57:39.666 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 1050K .......... .......... .......... .......... .......... 0% 23.8M 16s 2024-09-12T12:57:39.666 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 1100K .......... .......... .......... .......... .......... 0% 23.2M 16s 2024-09-12T12:57:39.666 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 1150K .......... .......... .......... .......... .......... 0% 23.7M 15s 2024-09-12T12:57:39.667 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 1200K .......... .......... .......... .......... .......... 0% 26.0M 15s 2024-09-12T12:57:39.667 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 1250K .......... .......... .......... .......... .......... 0% 126M 15s 2024-09-12T12:57:39.669 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 1300K .......... .......... .......... .......... .......... 0% 21.2M 14s 2024-09-12T12:57:39.680 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 1350K .......... .......... .......... .......... .......... 0% 7.07M 15s 2024-09-12T12:57:39.680 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 1400K .......... .......... .......... .......... .......... 0% 188M 14s 2024-09-12T12:57:39.680 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 1450K .......... .......... .......... .......... .......... 0% 201M 14s 2024-09-12T12:57:39.680 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 1500K .......... .......... .......... .......... .......... 0% 188M 13s 2024-09-12T12:57:39.681 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 1550K .......... .......... .......... .......... .......... 0% 34.7M 13s 2024-09-12T12:57:39.681 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 1600K .......... .......... .......... .......... .......... 0% 28.4M 13s 2024-09-12T12:57:39.682 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 1650K .......... .......... .......... .......... .......... 1% 23.2M 13s 2024-09-12T12:57:39.684 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 1700K .......... .......... .......... .......... .......... 1% 22.4M 12s 2024-09-12T12:57:39.685 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 1750K .......... .......... .......... .......... .......... 1% 134M 12s 2024-09-12T12:57:39.687 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 1800K .......... .......... .......... .......... .......... 1% 24.1M 12s 2024-09-12T12:57:39.689 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 1850K .......... .......... .......... .......... .......... 1% 23.2M 12s 2024-09-12T12:57:39.691 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 1900K .......... .......... .......... .......... .......... 1% 24.4M 12s 2024-09-12T12:57:39.693 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 1950K .......... .......... .......... .......... .......... 1% 20.0M 12s 2024-09-12T12:57:39.694 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 2000K .......... .......... .......... .......... .......... 1% 148M 11s 2024-09-12T12:57:39.695 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 2050K .......... .......... .......... .......... .......... 1% 25.3M 11s 2024-09-12T12:57:39.697 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 2100K .......... .......... .......... .......... .......... 1% 26.6M 11s 2024-09-12T12:57:39.700 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 2150K .......... .......... .......... .......... .......... 1% 24.7M 11s 2024-09-12T12:57:39.700 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 2200K .......... .......... .......... .......... .......... 1% 201M 11s 2024-09-12T12:57:39.701 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 2250K .......... .......... .......... .......... .......... 1% 22.1M 11s 2024-09-12T12:57:39.704 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 2300K .......... .......... .......... .......... .......... 1% 24.3M 11s 2024-09-12T12:57:39.706 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 2350K .......... .......... .......... .......... .......... 1% 23.6M 11s 2024-09-12T12:57:39.708 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 2400K .......... .......... .......... .......... .......... 1% 23.2M 10s 2024-09-12T12:57:39.709 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 2450K .......... .......... .......... .......... .......... 1% 36.2M 10s 2024-09-12T12:57:39.710 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 2500K .......... .......... .......... .......... .......... 1% 114M 10s 2024-09-12T12:57:39.712 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 2550K .......... .......... .......... .......... .......... 1% 20.9M 10s 2024-09-12T12:57:39.713 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 2600K .......... .......... .......... .......... .......... 1% 40.7M 10s 2024-09-12T12:57:39.716 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 2650K .......... .......... .......... .......... .......... 1% 16.3M 10s 2024-09-12T12:57:39.717 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 2700K .......... .......... .......... .......... .......... 1% 182M 10s 2024-09-12T12:57:39.719 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 2750K .......... .......... .......... .......... .......... 1% 21.6M 10s 2024-09-12T12:57:39.721 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 2800K .......... .......... .......... .......... .......... 1% 22.8M 10s 2024-09-12T12:57:39.721 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 2850K .......... .......... .......... .......... .......... 1% 50.6M 10s 2024-09-12T12:57:39.725 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 2900K .......... .......... .......... .......... .......... 1% 17.1M 10s 2024-09-12T12:57:39.728 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 2950K .......... .......... .......... .......... .......... 1% 13.0M 10s 2024-09-12T12:57:39.729 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 3000K .......... .......... .......... .......... .......... 1% 184M 9s 2024-09-12T12:57:39.730 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 3050K .......... .......... .......... .......... .......... 1% 26.0M 9s 2024-09-12T12:57:39.733 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 3100K .......... .......... .......... .......... .......... 1% 25.7M 9s 2024-09-12T12:57:39.735 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 3150K .......... .......... .......... .......... .......... 1% 23.2M 9s 2024-09-12T12:57:39.737 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 3200K .......... .......... .......... .......... .......... 1% 25.2M 9s 2024-09-12T12:57:39.737 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 3250K .......... .......... .......... .......... .......... 1% 116M 9s 2024-09-12T12:57:39.739 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 3300K .......... .......... .......... .......... .......... 1% 27.1M 9s 2024-09-12T12:57:39.741 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 3350K .......... .......... .......... .......... .......... 2% 24.0M 9s 2024-09-12T12:57:39.743 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 3400K .......... .......... .......... .......... .......... 2% 23.9M 9s 2024-09-12T12:57:39.745 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 3450K .......... .......... .......... .......... .......... 2% 23.0M 9s 2024-09-12T12:57:39.745 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 3500K .......... .......... .......... .......... .......... 2% 181M 9s 2024-09-12T12:57:39.747 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 3550K .......... .......... .......... .......... .......... 2% 23.6M 9s 2024-09-12T12:57:39.750 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 3600K .......... .......... .......... .......... .......... 2% 18.5M 9s 2024-09-12T12:57:39.755 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 3650K .......... .......... .......... .......... .......... 2% 10.6M 9s 2024-09-12T12:57:39.756 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 3700K .......... .......... .......... .......... .......... 2% 29.9M 9s 2024-09-12T12:57:39.757 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 3750K .......... .......... .......... .......... .......... 2% 41.6M 9s 2024-09-12T12:57:39.763 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 3800K .......... .......... .......... .......... .......... 2% 8.40M 9s 2024-09-12T12:57:39.764 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 3850K .......... .......... .......... .......... .......... 2% 44.1M 9s 2024-09-12T12:57:39.766 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 3900K .......... .......... .......... .......... .......... 2% 24.7M 9s 2024-09-12T12:57:39.769 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 3950K .......... .......... .......... .......... .......... 2% 18.4M 9s 2024-09-12T12:57:39.772 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 4000K .......... .......... .......... .......... .......... 2% 19.9M 9s 2024-09-12T12:57:39.772 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 4050K .......... .......... .......... .......... .......... 2% 77.8M 9s 2024-09-12T12:57:39.774 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 4100K .......... .......... .......... .......... .......... 2% 32.0M 9s 2024-09-12T12:57:39.776 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 4150K .......... .......... .......... .......... .......... 2% 21.0M 9s 2024-09-12T12:57:39.777 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 4200K .......... .......... .......... .......... .......... 2% 28.6M 9s 2024-09-12T12:57:39.778 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 4250K .......... .......... .......... .......... .......... 2% 50.0M 9s 2024-09-12T12:57:39.780 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 4300K .......... .......... .......... .......... .......... 2% 29.9M 9s 2024-09-12T12:57:39.783 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 4350K .......... .......... .......... .......... .......... 2% 22.9M 9s 2024-09-12T12:57:39.785 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 4400K .......... .......... .......... .......... .......... 2% 22.5M 9s 2024-09-12T12:57:39.787 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 4450K .......... .......... .......... .......... .......... 2% 27.3M 8s 2024-09-12T12:57:39.787 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 4500K .......... .......... .......... .......... .......... 2% 63.1M 8s 2024-09-12T12:57:39.789 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 4550K .......... .......... .......... .......... .......... 2% 20.0M 8s 2024-09-12T12:57:39.791 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 4600K .......... .......... .......... .......... .......... 2% 35.7M 8s 2024-09-12T12:57:39.793 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 4650K .......... .......... .......... .......... .......... 2% 25.6M 8s 2024-09-12T12:57:39.794 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 4700K .......... .......... .......... .......... .......... 2% 27.9M 8s 2024-09-12T12:57:39.796 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 4750K .......... .......... .......... .......... .......... 2% 31.5M 8s 2024-09-12T12:57:39.797 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 4800K .......... .......... .......... .......... .......... 2% 27.5M 8s 2024-09-12T12:57:39.799 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 4850K .......... .......... .......... .......... .......... 2% 30.4M 8s 2024-09-12T12:57:39.802 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 4900K .......... .......... .......... .......... .......... 2% 24.8M 8s 2024-09-12T12:57:39.803 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 4950K .......... .......... .......... .......... .......... 2% 27.2M 8s 2024-09-12T12:57:39.813 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 5000K .......... .......... .......... .......... .......... 2% 7.89M 8s 2024-09-12T12:57:39.813 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 5050K .......... .......... .......... .......... .......... 3% 94.1M 8s 2024-09-12T12:57:39.814 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 5100K .......... .......... .......... .......... .......... 3% 134M 8s 2024-09-12T12:57:39.814 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 5150K .......... .......... .......... .......... .......... 3% 71.2M 8s 2024-09-12T12:57:39.814 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 5200K .......... .......... .......... .......... .......... 3% 39.6M 8s 2024-09-12T12:57:39.815 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 5250K .......... .......... .......... .......... .......... 3% 20.5M 8s 2024-09-12T12:57:39.815 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 5300K .......... .......... .......... .......... .......... 3% 36.9M 8s 2024-09-12T12:57:39.817 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 5350K .......... .......... .......... .......... .......... 3% 26.5M 8s 2024-09-12T12:57:39.819 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 5400K .......... .......... .......... .......... .......... 3% 41.9M 8s 2024-09-12T12:57:39.821 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 5450K .......... .......... .......... .......... .......... 3% 23.1M 8s 2024-09-12T12:57:39.821 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 5500K .......... .......... .......... .......... .......... 3% 57.7M 8s 2024-09-12T12:57:39.824 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 5550K .......... .......... .......... .......... .......... 3% 20.8M 8s 2024-09-12T12:57:39.826 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 5600K .......... .......... .......... .......... .......... 3% 24.5M 8s 2024-09-12T12:57:39.828 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 5650K .......... .......... .......... .......... .......... 3% 25.7M 8s 2024-09-12T12:57:39.829 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 5700K .......... .......... .......... .......... .......... 3% 33.2M 8s 2024-09-12T12:57:39.831 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 5750K .......... .......... .......... .......... .......... 3% 24.2M 8s 2024-09-12T12:57:39.833 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 5800K .......... .......... .......... .......... .......... 3% 27.2M 8s 2024-09-12T12:57:39.835 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 5850K .......... .......... .......... .......... .......... 3% 23.5M 8s 2024-09-12T12:57:39.836 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 5900K .......... .......... .......... .......... .......... 3% 49.2M 8s 2024-09-12T12:57:39.838 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 5950K .......... .......... .......... .......... .......... 3% 23.7M 8s 2024-09-12T12:57:39.840 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 6000K .......... .......... .......... .......... .......... 3% 24.0M 8s 2024-09-12T12:57:39.841 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 6050K .......... .......... .......... .......... .......... 3% 42.6M 8s 2024-09-12T12:57:39.844 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 6100K .......... .......... .......... .......... .......... 3% 24.7M 8s 2024-09-12T12:57:39.845 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 6150K .......... .......... .......... .......... .......... 3% 41.2M 8s 2024-09-12T12:57:39.848 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 6200K .......... .......... .......... .......... .......... 3% 16.4M 8s 2024-09-12T12:57:39.849 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 6250K .......... .......... .......... .......... .......... 3% 23.8M 8s 2024-09-12T12:57:39.851 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 6300K .......... .......... .......... .......... .......... 3% 43.8M 8s 2024-09-12T12:57:39.853 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 6350K .......... .......... .......... .......... .......... 3% 23.3M 8s 2024-09-12T12:57:39.855 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 6400K .......... .......... .......... .......... .......... 3% 22.6M 8s 2024-09-12T12:57:39.856 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 6450K .......... .......... .......... .......... .......... 3% 51.0M 8s 2024-09-12T12:57:39.858 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 6500K .......... .......... .......... .......... .......... 3% 23.4M 8s 2024-09-12T12:57:39.859 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 6550K .......... .......... .......... .......... .......... 3% 48.9M 7s 2024-09-12T12:57:39.861 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 6600K .......... .......... .......... .......... .......... 3% 22.2M 7s 2024-09-12T12:57:39.862 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 6650K .......... .......... .......... .......... .......... 3% 36.6M 7s 2024-09-12T12:57:39.864 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 6700K .......... .......... .......... .......... .......... 4% 25.6M 7s 2024-09-12T12:57:39.867 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 6750K .......... .......... .......... .......... .......... 4% 20.8M 7s 2024-09-12T12:57:39.869 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 6800K .......... .......... .......... .......... .......... 4% 26.1M 7s 2024-09-12T12:57:39.869 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 6850K .......... .......... .......... .......... .......... 4% 53.5M 7s 2024-09-12T12:57:39.872 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 6900K .......... .......... .......... .......... .......... 4% 22.5M 7s 2024-09-12T12:57:39.873 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 6950K .......... .......... .......... .......... .......... 4% 36.7M 7s 2024-09-12T12:57:39.875 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 7000K .......... .......... .......... .......... .......... 4% 24.0M 7s 2024-09-12T12:57:39.876 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 7050K .......... .......... .......... .......... .......... 4% 50.8M 7s 2024-09-12T12:57:39.881 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 7100K .......... .......... .......... .......... .......... 4% 9.18M 7s 2024-09-12T12:57:39.888 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 7150K .......... .......... .......... .......... .......... 4% 7.94M 7s 2024-09-12T12:57:39.889 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 7200K .......... .......... .......... .......... .......... 4% 43.0M 7s 2024-09-12T12:57:39.891 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 7250K .......... .......... .......... .......... .......... 4% 23.7M 7s 2024-09-12T12:57:39.892 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 7300K .......... .......... .......... .......... .......... 4% 49.0M 7s 2024-09-12T12:57:39.893 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 7350K .......... .......... .......... .......... .......... 4% 24.6M 7s 2024-09-12T12:57:39.896 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 7400K .......... .......... .......... .......... .......... 4% 34.8M 7s 2024-09-12T12:57:39.897 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 7450K .......... .......... .......... .......... .......... 4% 21.8M 7s 2024-09-12T12:57:39.898 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 7500K .......... .......... .......... .......... .......... 4% 42.4M 7s 2024-09-12T12:57:39.902 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 7550K .......... .......... .......... .......... .......... 4% 17.5M 7s 2024-09-12T12:57:39.902 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 7600K .......... .......... .......... .......... .......... 4% 55.2M 7s 2024-09-12T12:57:39.904 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 7650K .......... .......... .......... .......... .......... 4% 23.4M 7s 2024-09-12T12:57:39.906 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 7700K .......... .......... .......... .......... .......... 4% 36.4M 7s 2024-09-12T12:57:39.908 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 7750K .......... .......... .......... .......... .......... 4% 24.8M 7s 2024-09-12T12:57:39.909 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 7800K .......... .......... .......... .......... .......... 4% 38.2M 7s 2024-09-12T12:57:39.911 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 7850K .......... .......... .......... .......... .......... 4% 25.1M 7s 2024-09-12T12:57:39.912 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 7900K .......... .......... .......... .......... .......... 4% 45.5M 7s 2024-09-12T12:57:39.915 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 7950K .......... .......... .......... .......... .......... 4% 20.0M 7s 2024-09-12T12:57:39.917 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 8000K .......... .......... .......... .......... .......... 4% 21.7M 7s 2024-09-12T12:57:39.918 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 8050K .......... .......... .......... .......... .......... 4% 32.7M 7s 2024-09-12T12:57:39.919 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 8100K .......... .......... .......... .......... .......... 4% 50.4M 7s 2024-09-12T12:57:39.921 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 8150K .......... .......... .......... .......... .......... 4% 25.5M 7s 2024-09-12T12:57:39.922 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 8200K .......... .......... .......... .......... .......... 4% 29.9M 7s 2024-09-12T12:57:39.924 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 8250K .......... .......... .......... .......... .......... 4% 27.9M 7s 2024-09-12T12:57:39.926 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 8300K .......... .......... .......... .......... .......... 4% 48.0M 7s 2024-09-12T12:57:39.927 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 8350K .......... .......... .......... .......... .......... 4% 21.2M 7s 2024-09-12T12:57:39.929 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 8400K .......... .......... .......... .......... .......... 5% 27.6M 7s 2024-09-12T12:57:39.931 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 8450K .......... .......... .......... .......... .......... 5% 35.7M 7s 2024-09-12T12:57:39.933 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 8500K .......... .......... .......... .......... .......... 5% 22.3M 7s 2024-09-12T12:57:39.935 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 8550K .......... .......... .......... .......... .......... 5% 21.2M 7s 2024-09-12T12:57:39.936 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 8600K .......... .......... .......... .......... .......... 5% 48.5M 7s 2024-09-12T12:57:39.938 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 8650K .......... .......... .......... .......... .......... 5% 27.2M 7s 2024-09-12T12:57:39.940 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 8700K .......... .......... .......... .......... .......... 5% 22.7M 7s 2024-09-12T12:57:39.942 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 8750K .......... .......... .......... .......... .......... 5% 21.0M 7s 2024-09-12T12:57:39.944 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 8800K .......... .......... .......... .......... .......... 5% 27.5M 7s 2024-09-12T12:57:39.945 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 8850K .......... .......... .......... .......... .......... 5% 45.5M 7s 2024-09-12T12:57:39.947 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 8900K .......... .......... .......... .......... .......... 5% 23.2M 7s 2024-09-12T12:57:39.949 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 8950K .......... .......... .......... .......... .......... 5% 35.4M 7s 2024-09-12T12:57:39.950 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 9000K .......... .......... .......... .......... .......... 5% 27.3M 7s 2024-09-12T12:57:39.952 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 9050K .......... .......... .......... .......... .......... 5% 41.0M 7s 2024-09-12T12:57:39.955 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 9100K .......... .......... .......... .......... .......... 5% 19.4M 7s 2024-09-12T12:57:39.956 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 9150K .......... .......... .......... .......... .......... 5% 25.1M 7s 2024-09-12T12:57:39.958 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 9200K .......... .......... .......... .......... .......... 5% 23.7M 7s 2024-09-12T12:57:39.960 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 9250K .......... .......... .......... .......... .......... 5% 47.0M 7s 2024-09-12T12:57:39.963 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 9300K .......... .......... .......... .......... .......... 5% 22.2M 7s 2024-09-12T12:57:39.963 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 9350K .......... .......... .......... .......... .......... 5% 46.0M 7s 2024-09-12T12:57:39.965 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 9400K .......... .......... .......... .......... .......... 5% 24.4M 7s 2024-09-12T12:57:39.966 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 9450K .......... .......... .......... .......... .......... 5% 46.3M 7s 2024-09-12T12:57:39.968 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 9500K .......... .......... .......... .......... .......... 5% 23.9M 7s 2024-09-12T12:57:39.973 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 9550K .......... .......... .......... .......... .......... 5% 18.2M 7s 2024-09-12T12:57:39.974 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 9600K .......... .......... .......... .......... .......... 5% 47.6M 7s 2024-09-12T12:57:39.974 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 9650K .......... .......... .......... .......... .......... 5% 21.6M 7s 2024-09-12T12:57:39.976 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 9700K .......... .......... .......... .......... .......... 5% 24.6M 7s 2024-09-12T12:57:39.977 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 9750K .......... .......... .......... .......... .......... 5% 40.9M 7s 2024-09-12T12:57:39.979 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 9800K .......... .......... .......... .......... .......... 5% 23.8M 7s 2024-09-12T12:57:39.980 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 9850K .......... .......... .......... .......... .......... 5% 32.9M 7s 2024-09-12T12:57:39.982 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 9900K .......... .......... .......... .......... .......... 5% 22.1M 7s 2024-09-12T12:57:39.985 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 9950K .......... .......... .......... .......... .......... 5% 27.0M 7s 2024-09-12T12:57:39.986 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 10000K .......... .......... .......... .......... .......... 5% 38.8M 7s 2024-09-12T12:57:39.988 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 10050K .......... .......... .......... .......... .......... 5% 25.3M 7s 2024-09-12T12:57:39.989 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 10100K .......... .......... .......... .......... .......... 6% 50.0M 7s 2024-09-12T12:57:39.991 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 10150K .......... .......... .......... .......... .......... 6% 22.3M 7s 2024-09-12T12:57:39.993 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 10200K .......... .......... .......... .......... .......... 6% 23.7M 7s 2024-09-12T12:57:39.994 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 10250K .......... .......... .......... .......... .......... 6% 39.6M 7s 2024-09-12T12:57:39.996 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 10300K .......... .......... .......... .......... .......... 6% 22.3M 7s 2024-09-12T12:57:39.999 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 10350K .......... .......... .......... .......... .......... 6% 18.6M 7s 2024-09-12T12:57:40.000 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 10400K .......... .......... .......... .......... .......... 6% 37.5M 7s 2024-09-12T12:57:40.004 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 10450K .......... .......... .......... .......... .......... 6% 14.2M 7s 2024-09-12T12:57:40.006 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 10500K .......... .......... .......... .......... .......... 6% 20.3M 7s 2024-09-12T12:57:40.011 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 10550K .......... .......... .......... .......... .......... 6% 9.28M 7s 2024-09-12T12:57:40.018 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 10600K .......... .......... .......... .......... .......... 6% 61.5M 7s 2024-09-12T12:57:40.019 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 10650K .......... .......... .......... .......... .......... 6% 11.3M 7s 2024-09-12T12:57:40.021 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 10700K .......... .......... .......... .......... .......... 6% 11.3M 7s 2024-09-12T12:57:40.024 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 10750K .......... .......... .......... .......... .......... 6% 13.4M 7s 2024-09-12T12:57:40.029 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 10800K .......... .......... .......... .......... .......... 6% 11.8M 7s 2024-09-12T12:57:40.030 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 10850K .......... .......... .......... .......... .......... 6% 45.3M 7s 2024-09-12T12:57:40.031 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 10900K .......... .......... .......... .......... .......... 6% 31.8M 7s 2024-09-12T12:57:40.033 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 10950K .......... .......... .......... .......... .......... 6% 23.5M 7s 2024-09-12T12:57:40.035 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 11000K .......... .......... .......... .......... .......... 6% 33.0M 7s 2024-09-12T12:57:40.037 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 11050K .......... .......... .......... .......... .......... 6% 24.7M 7s 2024-09-12T12:57:40.038 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 11100K .......... .......... .......... .......... .......... 6% 49.0M 7s 2024-09-12T12:57:40.040 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 11150K .......... .......... .......... .......... .......... 6% 23.8M 7s 2024-09-12T12:57:40.042 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 11200K .......... .......... .......... .......... .......... 6% 24.1M 7s 2024-09-12T12:57:40.043 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 11250K .......... .......... .......... .......... .......... 6% 39.1M 7s 2024-09-12T12:57:40.045 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 11300K .......... .......... .......... .......... .......... 6% 24.2M 7s 2024-09-12T12:57:40.048 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 11350K .......... .......... .......... .......... .......... 6% 21.8M 7s 2024-09-12T12:57:40.048 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 11400K .......... .......... .......... .......... .......... 6% 67.0M 7s 2024-09-12T12:57:40.050 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 11450K .......... .......... .......... .......... .......... 6% 29.0M 7s 2024-09-12T12:57:40.051 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 11500K .......... .......... .......... .......... .......... 6% 27.9M 7s 2024-09-12T12:57:40.056 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 11550K .......... .......... .......... .......... .......... 6% 17.8M 7s 2024-09-12T12:57:40.056 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 11600K .......... .......... .......... .......... .......... 6% 33.6M 7s 2024-09-12T12:57:40.057 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 11650K .......... .......... .......... .......... .......... 6% 49.3M 7s 2024-09-12T12:57:40.059 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 11700K .......... .......... .......... .......... .......... 6% 23.9M 7s 2024-09-12T12:57:40.060 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 11750K .......... .......... .......... .......... .......... 7% 32.6M 7s 2024-09-12T12:57:40.062 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 11800K .......... .......... .......... .......... .......... 7% 25.7M 7s 2024-09-12T12:57:40.063 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 11850K .......... .......... .......... .......... .......... 7% 52.5M 7s 2024-09-12T12:57:40.065 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 11900K .......... .......... .......... .......... .......... 7% 22.8M 7s 2024-09-12T12:57:40.067 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 11950K .......... .......... .......... .......... .......... 7% 26.9M 7s 2024-09-12T12:57:40.068 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 12000K .......... .......... .......... .......... .......... 7% 35.0M 7s 2024-09-12T12:57:40.071 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 12050K .......... .......... .......... .......... .......... 7% 22.0M 7s 2024-09-12T12:57:40.073 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 12100K .......... .......... .......... .......... .......... 7% 23.3M 7s 2024-09-12T12:57:40.074 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 12150K .......... .......... .......... .......... .......... 7% 36.9M 7s 2024-09-12T12:57:40.076 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 12200K .......... .......... .......... .......... .......... 7% 27.4M 7s 2024-09-12T12:57:40.077 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 12250K .......... .......... .......... .......... .......... 7% 37.7M 7s 2024-09-12T12:57:40.079 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 12300K .......... .......... .......... .......... .......... 7% 23.8M 7s 2024-09-12T12:57:40.081 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 12350K .......... .......... .......... .......... .......... 7% 25.2M 7s 2024-09-12T12:57:40.082 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 12400K .......... .......... .......... .......... .......... 7% 38.8M 7s 2024-09-12T12:57:40.084 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 12450K .......... .......... .......... .......... .......... 7% 26.9M 7s 2024-09-12T12:57:40.086 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 12500K .......... .......... .......... .......... .......... 7% 35.8M 7s 2024-09-12T12:57:40.088 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 12550K .......... .......... .......... .......... .......... 7% 23.7M 7s 2024-09-12T12:57:40.089 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 12600K .......... .......... .......... .......... .......... 7% 27.5M 7s 2024-09-12T12:57:40.090 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 12650K .......... .......... .......... .......... .......... 7% 41.8M 7s 2024-09-12T12:57:40.092 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 12700K .......... .......... .......... .......... .......... 7% 26.4M 7s 2024-09-12T12:57:40.095 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 12750K .......... .......... .......... .......... .......... 7% 21.1M 7s 2024-09-12T12:57:40.096 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 12800K .......... .......... .......... .......... .......... 7% 34.6M 7s 2024-09-12T12:57:40.098 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 12850K .......... .......... .......... .......... .......... 7% 26.9M 7s 2024-09-12T12:57:40.099 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 12900K .......... .......... .......... .......... .......... 7% 40.3M 7s 2024-09-12T12:57:40.102 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 12950K .......... .......... .......... .......... .......... 7% 21.8M 7s 2024-09-12T12:57:40.103 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 13000K .......... .......... .......... .......... .......... 7% 27.1M 7s 2024-09-12T12:57:40.104 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 13050K .......... .......... .......... .......... .......... 7% 40.8M 7s 2024-09-12T12:57:40.106 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 13100K .......... .......... .......... .......... .......... 7% 24.3M 7s 2024-09-12T12:57:40.109 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 13150K .......... .......... .......... .......... .......... 7% 18.1M 7s 2024-09-12T12:57:40.110 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 13200K .......... .......... .......... .......... .......... 7% 36.4M 7s 2024-09-12T12:57:40.112 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 13250K .......... .......... .......... .......... .......... 7% 24.3M 7s 2024-09-12T12:57:40.115 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 13300K .......... .......... .......... .......... .......... 7% 29.5M 7s 2024-09-12T12:57:40.116 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 13350K .......... .......... .......... .......... .......... 7% 28.8M 7s 2024-09-12T12:57:40.117 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 13400K .......... .......... .......... .......... .......... 7% 38.4M 6s 2024-09-12T12:57:40.118 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 13450K .......... .......... .......... .......... .......... 8% 31.3M 6s 2024-09-12T12:57:40.121 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 13500K .......... .......... .......... .......... .......... 8% 20.9M 6s 2024-09-12T12:57:40.123 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 13550K .......... .......... .......... .......... .......... 8% 23.0M 6s 2024-09-12T12:57:40.125 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 13600K .......... .......... .......... .......... .......... 8% 30.7M 6s 2024-09-12T12:57:40.127 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 13650K .......... .......... .......... .......... .......... 8% 25.7M 6s 2024-09-12T12:57:40.127 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 13700K .......... .......... .......... .......... .......... 8% 47.0M 6s 2024-09-12T12:57:40.130 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 13750K .......... .......... .......... .......... .......... 8% 23.3M 6s 2024-09-12T12:57:40.131 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 13800K .......... .......... .......... .......... .......... 8% 40.1M 6s 2024-09-12T12:57:40.134 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 13850K .......... .......... .......... .......... .......... 8% 14.6M 6s 2024-09-12T12:57:40.137 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 13900K .......... .......... .......... .......... .......... 8% 17.5M 6s 2024-09-12T12:57:40.142 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 13950K .......... .......... .......... .......... .......... 8% 10.6M 6s 2024-09-12T12:57:40.144 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 14000K .......... .......... .......... .......... .......... 8% 19.0M 6s 2024-09-12T12:57:40.148 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 14050K .......... .......... .......... .......... .......... 8% 12.2M 7s 2024-09-12T12:57:40.153 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 14100K .......... .......... .......... .......... .......... 8% 10.3M 7s 2024-09-12T12:57:40.154 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 14150K .......... .......... .......... .......... .......... 8% 31.7M 7s 2024-09-12T12:57:40.158 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 14200K .......... .......... .......... .......... .......... 8% 14.8M 7s 2024-09-12T12:57:40.163 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 14250K .......... .......... .......... .......... .......... 8% 9.33M 7s 2024-09-12T12:57:40.165 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 14300K .......... .......... .......... .......... .......... 8% 24.8M 7s 2024-09-12T12:57:40.233 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 14350K .......... .......... .......... .......... .......... 8% 736K 7s 2024-09-12T12:57:40.235 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 14400K .......... .......... .......... .......... .......... 8% 61.3M 7s 2024-09-12T12:57:40.235 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 14450K .......... .......... .......... .......... .......... 8% 43.9M 7s 2024-09-12T12:57:40.237 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 14500K .......... .......... .......... .......... .......... 8% 39.6M 7s 2024-09-12T12:57:40.237 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 14550K .......... .......... .......... .......... .......... 8% 46.6M 7s 2024-09-12T12:57:40.274 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 14600K .......... .......... .......... .......... .......... 8% 25.9M 7s 2024-09-12T12:57:40.274 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 14650K .......... .......... .......... .......... .......... 8% 199M 7s 2024-09-12T12:57:40.274 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 14700K .......... .......... .......... .......... .......... 8% 193M 7s 2024-09-12T12:57:40.274 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 14750K .......... .......... .......... .......... .......... 8% 178M 7s 2024-09-12T12:57:40.274 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 14800K .......... .......... .......... .......... .......... 8% 178M 7s 2024-09-12T12:57:40.275 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 14850K .......... .......... .......... .......... .......... 8% 198M 7s 2024-09-12T12:57:40.275 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 14900K .......... .......... .......... .......... .......... 8% 207M 7s 2024-09-12T12:57:40.275 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 14950K .......... .......... .......... .......... .......... 8% 183M 7s 2024-09-12T12:57:40.275 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 15000K .......... .......... .......... .......... .......... 8% 185M 7s 2024-09-12T12:57:40.275 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 15050K .......... .......... .......... .......... .......... 8% 209M 7s 2024-09-12T12:57:40.275 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 15100K .......... .......... .......... .......... .......... 8% 198M 7s 2024-09-12T12:57:40.275 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 15150K .......... .......... .......... .......... .......... 9% 8.90M 7s 2024-09-12T12:57:40.275 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 15200K .......... .......... .......... .......... .......... 9% 177M 7s 2024-09-12T12:57:40.275 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 15250K .......... .......... .......... .......... .......... 9% 194M 7s 2024-09-12T12:57:40.275 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 15300K .......... .......... .......... .......... .......... 9% 182M 7s 2024-09-12T12:57:40.275 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 15350K .......... .......... .......... .......... .......... 9% 155M 7s 2024-09-12T12:57:40.275 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 15400K .......... .......... .......... .......... .......... 9% 203M 7s 2024-09-12T12:57:40.276 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 15450K .......... .......... .......... .......... .......... 9% 210M 7s 2024-09-12T12:57:40.276 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 15500K .......... .......... .......... .......... .......... 9% 181M 7s 2024-09-12T12:57:40.276 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 15550K .......... .......... .......... .......... .......... 9% 173M 7s 2024-09-12T12:57:40.276 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 15600K .......... .......... .......... .......... .......... 9% 198M 7s 2024-09-12T12:57:40.276 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 15650K .......... .......... .......... .......... .......... 9% 205M 7s 2024-09-12T12:57:40.276 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 15700K .......... .......... .......... .......... .......... 9% 193M 7s 2024-09-12T12:57:40.276 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 15750K .......... .......... .......... .......... .......... 9% 169M 7s 2024-09-12T12:57:40.276 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 15800K .......... .......... .......... .......... .......... 9% 217M 7s 2024-09-12T12:57:40.276 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 15850K .......... .......... .......... .......... .......... 9% 212M 7s 2024-09-12T12:57:40.276 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 15900K .......... .......... .......... .......... .......... 9% 187M 7s 2024-09-12T12:57:40.276 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 15950K .......... .......... .......... .......... .......... 9% 158M 7s 2024-09-12T12:57:40.276 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 16000K .......... .......... .......... .......... .......... 9% 219M 7s 2024-09-12T12:57:40.276 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 16050K .......... .......... .......... .......... .......... 9% 197M 7s 2024-09-12T12:57:40.277 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 16100K .......... .......... .......... .......... .......... 9% 182M 7s 2024-09-12T12:57:40.277 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 16150K .......... .......... .......... .......... .......... 9% 175M 7s 2024-09-12T12:57:40.277 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 16200K .......... .......... .......... .......... .......... 9% 214M 7s 2024-09-12T12:57:40.277 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 16250K .......... .......... .......... .......... .......... 9% 222M 7s 2024-09-12T12:57:40.277 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 16300K .......... .......... .......... .......... .......... 9% 179M 7s 2024-09-12T12:57:40.277 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 16350K .......... .......... .......... .......... .......... 9% 163M 6s 2024-09-12T12:57:40.277 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 16400K .......... .......... .......... .......... .......... 9% 196M 6s 2024-09-12T12:57:40.277 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 16450K .......... .......... .......... .......... .......... 9% 210M 6s 2024-09-12T12:57:40.277 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 16500K .......... .......... .......... .......... .......... 9% 185M 6s 2024-09-12T12:57:40.277 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 16550K .......... .......... .......... .......... .......... 9% 186M 6s 2024-09-12T12:57:40.277 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 16600K .......... .......... .......... .......... .......... 9% 9.95M 6s 2024-09-12T12:57:40.277 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 16650K .......... .......... .......... .......... .......... 9% 24.7M 6s 2024-09-12T12:57:40.277 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 16700K .......... .......... .......... .......... .......... 9% 17.1M 6s 2024-09-12T12:57:40.278 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 16750K .......... .......... .......... .......... .......... 9% 12.3M 6s 2024-09-12T12:57:40.278 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 16800K .......... .......... .......... .......... .......... 9% 15.7M 6s 2024-09-12T12:57:40.279 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 16850K .......... .......... .......... .......... .......... 10% 8.42M 6s 2024-09-12T12:57:40.282 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 16900K .......... .......... .......... .......... .......... 10% 10.5M 7s 2024-09-12T12:57:40.282 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 16950K .......... .......... .......... .......... .......... 10% 86.2M 6s 2024-09-12T12:57:40.288 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 17000K .......... .......... .......... .......... .......... 10% 8.26M 7s 2024-09-12T12:57:40.292 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 17050K .......... .......... .......... .......... .......... 10% 14.4M 7s 2024-09-12T12:57:40.294 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 17100K .......... .......... .......... .......... .......... 10% 19.5M 7s 2024-09-12T12:57:40.297 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 17150K .......... .......... .......... .......... .......... 10% 17.9M 7s 2024-09-12T12:57:40.300 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 17200K .......... .......... .......... .......... .......... 10% 15.6M 7s 2024-09-12T12:57:40.305 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 17250K .......... .......... .......... .......... .......... 10% 9.10M 7s 2024-09-12T12:57:40.307 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 17300K .......... .......... .......... .......... .......... 10% 38.0M 7s 2024-09-12T12:57:40.307 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 17350K .......... .......... .......... .......... .......... 10% 52.4M 7s 2024-09-12T12:57:40.310 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 17400K .......... .......... .......... .......... .......... 10% 20.7M 7s 2024-09-12T12:57:40.368 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 17450K .......... .......... .......... .......... .......... 10% 873K 7s 2024-09-12T12:57:40.369 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 17500K .......... .......... .......... .......... .......... 10% 34.6M 7s 2024-09-12T12:57:40.370 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 17550K .......... .......... .......... .......... .......... 10% 115M 7s 2024-09-12T12:57:40.412 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 17600K .......... .......... .......... .......... .......... 10% 20.8M 7s 2024-09-12T12:57:40.412 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 17650K .......... .......... .......... .......... .......... 10% 203M 7s 2024-09-12T12:57:40.412 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 17700K .......... .......... .......... .......... .......... 10% 42.4M 7s 2024-09-12T12:57:40.412 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 17750K .......... .......... .......... .......... .......... 10% 88.4M 7s 2024-09-12T12:57:40.412 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 17800K .......... .......... .......... .......... .......... 10% 40.0M 7s 2024-09-12T12:57:40.413 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 17850K .......... .......... .......... .......... .......... 10% 190M 7s 2024-09-12T12:57:40.413 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 17900K .......... .......... .......... .......... .......... 10% 215M 7s 2024-09-12T12:57:40.413 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 17950K .......... .......... .......... .......... .......... 10% 132M 7s 2024-09-12T12:57:40.413 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 18000K .......... .......... .......... .......... .......... 10% 197M 7s 2024-09-12T12:57:40.413 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 18050K .......... .......... .......... .......... .......... 10% 214M 7s 2024-09-12T12:57:40.413 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 18100K .......... .......... .......... .......... .......... 10% 179M 7s 2024-09-12T12:57:40.413 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 18150K .......... .......... .......... .......... .......... 10% 165M 7s 2024-09-12T12:57:40.413 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 18200K .......... .......... .......... .......... .......... 10% 207M 7s 2024-09-12T12:57:40.413 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 18250K .......... .......... .......... .......... .......... 10% 169M 7s 2024-09-12T12:57:40.413 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 18300K .......... .......... .......... .......... .......... 10% 169M 7s 2024-09-12T12:57:40.413 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 18350K .......... .......... .......... .......... .......... 10% 164M 7s 2024-09-12T12:57:40.413 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 18400K .......... .......... .......... .......... .......... 10% 169M 7s 2024-09-12T12:57:40.414 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 18450K .......... .......... .......... .......... .......... 10% 168M 7s 2024-09-12T12:57:40.414 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 18500K .......... .......... .......... .......... .......... 11% 168M 7s 2024-09-12T12:57:40.414 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 18550K .......... .......... .......... .......... .......... 11% 177M 7s 2024-09-12T12:57:40.414 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 18600K .......... .......... .......... .......... .......... 11% 208M 7s 2024-09-12T12:57:40.414 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 18650K .......... .......... .......... .......... .......... 11% 187M 7s 2024-09-12T12:57:40.414 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 18700K .......... .......... .......... .......... .......... 11% 177M 7s 2024-09-12T12:57:40.414 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 18750K .......... .......... .......... .......... .......... 11% 155M 7s 2024-09-12T12:57:40.414 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 18800K .......... .......... .......... .......... .......... 11% 193M 7s 2024-09-12T12:57:40.414 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 18850K .......... .......... .......... .......... .......... 11% 180M 7s 2024-09-12T12:57:40.414 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 18900K .......... .......... .......... .......... .......... 11% 181M 7s 2024-09-12T12:57:40.414 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 18950K .......... .......... .......... .......... .......... 11% 169M 7s 2024-09-12T12:57:40.414 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 19000K .......... .......... .......... .......... .......... 11% 176M 6s 2024-09-12T12:57:40.414 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 19050K .......... .......... .......... .......... .......... 11% 207M 6s 2024-09-12T12:57:40.415 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 19100K .......... .......... .......... .......... .......... 11% 194M 6s 2024-09-12T12:57:40.415 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 19150K .......... .......... .......... .......... .......... 11% 56.8M 6s 2024-09-12T12:57:40.415 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 19200K .......... .......... .......... .......... .......... 11% 21.4M 6s 2024-09-12T12:57:40.415 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 19250K .......... .......... .......... .......... .......... 11% 22.2M 6s 2024-09-12T12:57:40.415 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 19300K .......... .......... .......... .......... .......... 11% 136M 6s 2024-09-12T12:57:40.415 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 19350K .......... .......... .......... .......... .......... 11% 22.0M 6s 2024-09-12T12:57:40.415 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 19400K .......... .......... .......... .......... .......... 11% 33.2M 6s 2024-09-12T12:57:40.415 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 19450K .......... .......... .......... .......... .......... 11% 31.4M 6s 2024-09-12T12:57:40.415 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 19500K .......... .......... .......... .......... .......... 11% 17.6M 6s 2024-09-12T12:57:40.415 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 19550K .......... .......... .......... .......... .......... 11% 34.9M 6s 2024-09-12T12:57:40.415 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 19600K .......... .......... .......... .......... .......... 11% 44.1M 6s 2024-09-12T12:57:40.415 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 19650K .......... .......... .......... .......... .......... 11% 29.1M 6s 2024-09-12T12:57:40.415 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 19700K .......... .......... .......... .......... .......... 11% 33.3M 6s 2024-09-12T12:57:40.415 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 19750K .......... .......... .......... .......... .......... 11% 18.0M 6s 2024-09-12T12:57:40.416 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 19800K .......... .......... .......... .......... .......... 11% 97.0M 6s 2024-09-12T12:57:40.416 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 19850K .......... .......... .......... .......... .......... 11% 30.9M 6s 2024-09-12T12:57:40.416 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 19900K .......... .......... .......... .......... .......... 11% 22.8M 6s 2024-09-12T12:57:40.416 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 19950K .......... .......... .......... .......... .......... 11% 25.4M 6s 2024-09-12T12:57:40.416 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 20000K .......... .......... .......... .......... .......... 11% 24.1M 6s 2024-09-12T12:57:40.419 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 20050K .......... .......... .......... .......... .......... 11% 37.2M 6s 2024-09-12T12:57:40.419 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 20100K .......... .......... .......... .......... .......... 11% 17.3M 6s 2024-09-12T12:57:40.419 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 20150K .......... .......... .......... .......... .......... 11% 42.0M 6s 2024-09-12T12:57:40.420 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 20200K .......... .......... .......... .......... .......... 12% 20.1M 6s 2024-09-12T12:57:40.421 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 20250K .......... .......... .......... .......... .......... 12% 38.5M 6s 2024-09-12T12:57:40.423 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 20300K .......... .......... .......... .......... .......... 12% 18.3M 6s 2024-09-12T12:57:40.425 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 20350K .......... .......... .......... .......... .......... 12% 27.3M 6s 2024-09-12T12:57:40.426 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 20400K .......... .......... .......... .......... .......... 12% 35.0M 6s 2024-09-12T12:57:40.429 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 20450K .......... .......... .......... .......... .......... 12% 22.3M 6s 2024-09-12T12:57:40.431 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 20500K .......... .......... .......... .......... .......... 12% 23.2M 6s 2024-09-12T12:57:40.432 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 20550K .......... .......... .......... .......... .......... 12% 39.3M 6s 2024-09-12T12:57:40.434 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 20600K .......... .......... .......... .......... .......... 12% 25.4M 6s 2024-09-12T12:57:40.435 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 20650K .......... .......... .......... .......... .......... 12% 36.5M 6s 2024-09-12T12:57:40.437 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 20700K .......... .......... .......... .......... .......... 12% 24.7M 6s 2024-09-12T12:57:40.439 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 20750K .......... .......... .......... .......... .......... 12% 23.7M 6s 2024-09-12T12:57:40.442 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 20800K .......... .......... .......... .......... .......... 12% 27.3M 6s 2024-09-12T12:57:40.442 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 20850K .......... .......... .......... .......... .......... 12% 35.7M 6s 2024-09-12T12:57:40.443 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 20900K .......... .......... .......... .......... .......... 12% 44.3M 6s 2024-09-12T12:57:40.446 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 20950K .......... .......... .......... .......... .......... 12% 24.4M 6s 2024-09-12T12:57:40.447 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 21000K .......... .......... .......... .......... .......... 12% 37.4M 6s 2024-09-12T12:57:40.449 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 21050K .......... .......... .......... .......... .......... 12% 22.6M 6s 2024-09-12T12:57:40.451 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 21100K .......... .......... .......... .......... .......... 12% 23.4M 6s 2024-09-12T12:57:40.453 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 21150K .......... .......... .......... .......... .......... 12% 22.2M 6s 2024-09-12T12:57:40.454 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 21200K .......... .......... .......... .......... .......... 12% 46.6M 6s 2024-09-12T12:57:40.456 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 21250K .......... .......... .......... .......... .......... 12% 24.0M 6s 2024-09-12T12:57:40.458 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 21300K .......... .......... .......... .......... .......... 12% 32.8M 6s 2024-09-12T12:57:40.459 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 21350K .......... .......... .......... .......... .......... 12% 41.4M 6s 2024-09-12T12:57:40.460 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 21400K .......... .......... .......... .......... .......... 12% 30.7M 6s 2024-09-12T12:57:40.463 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 21450K .......... .......... .......... .......... .......... 12% 23.0M 6s 2024-09-12T12:57:40.464 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 21500K .......... .......... .......... .......... .......... 12% 40.4M 6s 2024-09-12T12:57:40.466 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 21550K .......... .......... .......... .......... .......... 12% 24.2M 6s 2024-09-12T12:57:40.469 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 21600K .......... .......... .......... .......... .......... 12% 21.7M 6s 2024-09-12T12:57:40.470 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 21650K .......... .......... .......... .......... .......... 12% 28.3M 6s 2024-09-12T12:57:40.471 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 21700K .......... .......... .......... .......... .......... 12% 41.1M 6s 2024-09-12T12:57:40.473 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 21750K .......... .......... .......... .......... .......... 12% 24.1M 6s 2024-09-12T12:57:40.474 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 21800K .......... .......... .......... .......... .......... 12% 38.8M 6s 2024-09-12T12:57:40.478 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 21850K .......... .......... .......... .......... .......... 12% 21.6M 6s 2024-09-12T12:57:40.478 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 21900K .......... .......... .......... .......... .......... 13% 46.6M 6s 2024-09-12T12:57:40.480 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 21950K .......... .......... .......... .......... .......... 13% 20.6M 6s 2024-09-12T12:57:40.482 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 22000K .......... .......... .......... .......... .......... 13% 22.6M 6s 2024-09-12T12:57:40.485 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 22050K .......... .......... .......... .......... .......... 13% 25.2M 6s 2024-09-12T12:57:40.486 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 22100K .......... .......... .......... .......... .......... 13% 33.7M 6s 2024-09-12T12:57:40.488 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 22150K .......... .......... .......... .......... .......... 13% 22.0M 6s 2024-09-12T12:57:40.489 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 22200K .......... .......... .......... .......... .......... 13% 48.6M 6s 2024-09-12T12:57:40.491 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 22250K .......... .......... .......... .......... .......... 13% 22.5M 6s 2024-09-12T12:57:40.492 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 22300K .......... .......... .......... .......... .......... 13% 35.8M 6s 2024-09-12T12:57:40.495 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 22350K .......... .......... .......... .......... .......... 13% 22.1M 6s 2024-09-12T12:57:40.497 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 22400K .......... .......... .......... .......... .......... 13% 20.4M 6s 2024-09-12T12:57:40.498 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 22450K .......... .......... .......... .......... .......... 13% 37.4M 6s 2024-09-12T12:57:40.500 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 22500K .......... .......... .......... .......... .......... 13% 28.2M 6s 2024-09-12T12:57:40.501 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 22550K .......... .......... .......... .......... .......... 13% 37.0M 6s 2024-09-12T12:57:40.503 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 22600K .......... .......... .......... .......... .......... 13% 30.0M 6s 2024-09-12T12:57:40.504 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 22650K .......... .......... .......... .......... .......... 13% 35.5M 6s 2024-09-12T12:57:40.506 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 22700K .......... .......... .......... .......... .......... 13% 22.7M 6s 2024-09-12T12:57:40.508 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 22750K .......... .......... .......... .......... .......... 13% 25.4M 6s 2024-09-12T12:57:40.511 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 22800K .......... .......... .......... .......... .......... 13% 25.1M 6s 2024-09-12T12:57:40.513 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 22850K .......... .......... .......... .......... .......... 13% 25.3M 6s 2024-09-12T12:57:40.514 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 22900K .......... .......... .......... .......... .......... 13% 32.6M 6s 2024-09-12T12:57:40.515 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 22950K .......... .......... .......... .......... .......... 13% 34.2M 6s 2024-09-12T12:57:40.517 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 23000K .......... .......... .......... .......... .......... 13% 24.2M 6s 2024-09-12T12:57:40.518 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 23050K .......... .......... .......... .......... .......... 13% 40.0M 6s 2024-09-12T12:57:40.551 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 23100K .......... .......... .......... .......... .......... 13% 1.50M 6s 2024-09-12T12:57:40.553 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 23150K .......... .......... .......... .......... .......... 13% 22.7M 6s 2024-09-12T12:57:40.554 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 23200K .......... .......... .......... .......... .......... 13% 45.1M 6s 2024-09-12T12:57:40.556 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 23250K .......... .......... .......... .......... .......... 13% 25.1M 6s 2024-09-12T12:57:40.558 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 23300K .......... .......... .......... .......... .......... 13% 35.3M 6s 2024-09-12T12:57:40.559 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 23350K .......... .......... .......... .......... .......... 13% 27.3M 6s 2024-09-12T12:57:40.562 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 23400K .......... .......... .......... .......... .......... 13% 19.7M 6s 2024-09-12T12:57:40.563 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 23450K .......... .......... .......... .......... .......... 13% 60.7M 6s 2024-09-12T12:57:40.565 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 23500K .......... .......... .......... .......... .......... 13% 22.5M 6s 2024-09-12T12:57:40.567 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 23550K .......... .......... .......... .......... .......... 14% 26.7M 6s 2024-09-12T12:57:40.568 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 23600K .......... .......... .......... .......... .......... 14% 34.4M 6s 2024-09-12T12:57:40.570 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 23650K .......... .......... .......... .......... .......... 14% 27.2M 6s 2024-09-12T12:57:40.571 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 23700K .......... .......... .......... .......... .......... 14% 42.0M 6s 2024-09-12T12:57:40.573 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 23750K .......... .......... .......... .......... .......... 14% 22.9M 6s 2024-09-12T12:57:40.574 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 23800K .......... .......... .......... .......... .......... 14% 46.7M 6s 2024-09-12T12:57:40.576 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 23850K .......... .......... .......... .......... .......... 14% 22.5M 6s 2024-09-12T12:57:40.579 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 23900K .......... .......... .......... .......... .......... 14% 22.6M 6s 2024-09-12T12:57:40.581 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 23950K .......... .......... .......... .......... .......... 14% 26.8M 6s 2024-09-12T12:57:40.582 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 24000K .......... .......... .......... .......... .......... 14% 22.5M 6s 2024-09-12T12:57:40.585 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 24050K .......... .......... .......... .......... .......... 14% 28.5M 6s 2024-09-12T12:57:40.586 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 24100K .......... .......... .......... .......... .......... 14% 35.4M 6s 2024-09-12T12:57:40.587 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 24150K .......... .......... .......... .......... .......... 14% 26.2M 6s 2024-09-12T12:57:40.588 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 24200K .......... .......... .......... .......... .......... 14% 37.6M 6s 2024-09-12T12:57:40.591 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 24250K .......... .......... .......... .......... .......... 14% 25.9M 6s 2024-09-12T12:57:40.592 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 24300K .......... .......... .......... .......... .......... 14% 33.8M 6s 2024-09-12T12:57:40.594 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 24350K .......... .......... .......... .......... .......... 14% 19.3M 6s 2024-09-12T12:57:40.596 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 24400K .......... .......... .......... .......... .......... 14% 35.4M 6s 2024-09-12T12:57:40.598 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 24450K .......... .......... .......... .......... .......... 14% 24.4M 6s 2024-09-12T12:57:40.599 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 24500K .......... .......... .......... .......... .......... 14% 40.9M 6s 2024-09-12T12:57:40.601 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 24550K .......... .......... .......... .......... .......... 14% 21.7M 6s 2024-09-12T12:57:40.603 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 24600K .......... .......... .......... .......... .......... 14% 35.6M 6s 2024-09-12T12:57:40.604 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 24650K .......... .......... .......... .......... .......... 14% 29.4M 6s 2024-09-12T12:57:40.606 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 24700K .......... .......... .......... .......... .......... 14% 34.9M 6s 2024-09-12T12:57:40.608 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 24750K .......... .......... .......... .......... .......... 14% 24.1M 6s 2024-09-12T12:57:40.610 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 24800K .......... .......... .......... .......... .......... 14% 24.7M 6s 2024-09-12T12:57:40.611 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 24850K .......... .......... .......... .......... .......... 14% 35.9M 6s 2024-09-12T12:57:40.613 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 24900K .......... .......... .......... .......... .......... 14% 33.1M 6s 2024-09-12T12:57:40.614 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 24950K .......... .......... .......... .......... .......... 14% 32.3M 6s 2024-09-12T12:57:40.616 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 25000K .......... .......... .......... .......... .......... 14% 21.9M 6s 2024-09-12T12:57:40.618 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 25050K .......... .......... .......... .......... .......... 14% 25.2M 6s 2024-09-12T12:57:40.621 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 25100K .......... .......... .......... .......... .......... 14% 18.3M 6s 2024-09-12T12:57:40.627 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 25150K .......... .......... .......... .......... .......... 14% 8.18M 6s 2024-09-12T12:57:40.631 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 25200K .......... .......... .......... .......... .......... 14% 12.0M 6s 2024-09-12T12:57:40.633 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 25250K .......... .......... .......... .......... .......... 15% 23.9M 6s 2024-09-12T12:57:40.638 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 25300K .......... .......... .......... .......... .......... 15% 9.81M 6s 2024-09-12T12:57:40.641 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 25350K .......... .......... .......... .......... .......... 15% 19.3M 6s 2024-09-12T12:57:40.645 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 25400K .......... .......... .......... .......... .......... 15% 12.4M 6s 2024-09-12T12:57:40.646 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 25450K .......... .......... .......... .......... .......... 15% 26.6M 6s 2024-09-12T12:57:40.651 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 25500K .......... .......... .......... .......... .......... 15% 11.3M 6s 2024-09-12T12:57:40.655 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 25550K .......... .......... .......... .......... .......... 15% 11.9M 6s 2024-09-12T12:57:40.658 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 25600K .......... .......... .......... .......... .......... 15% 14.3M 6s 2024-09-12T12:57:40.663 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 25650K .......... .......... .......... .......... .......... 15% 11.3M 6s 2024-09-12T12:57:40.667 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 25700K .......... .......... .......... .......... .......... 15% 11.2M 6s 2024-09-12T12:57:40.669 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 25750K .......... .......... .......... .......... .......... 15% 27.3M 6s 2024-09-12T12:57:40.671 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 25800K .......... .......... .......... .......... .......... 15% 20.6M 6s 2024-09-12T12:57:40.673 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 25850K .......... .......... .......... .......... .......... 15% 26.5M 6s 2024-09-12T12:57:40.675 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 25900K .......... .......... .......... .......... .......... 15% 24.0M 6s 2024-09-12T12:57:40.677 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 25950K .......... .......... .......... .......... .......... 15% 24.7M 6s 2024-09-12T12:57:40.680 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 26000K .......... .......... .......... .......... .......... 15% 20.2M 6s 2024-09-12T12:57:40.680 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 26050K .......... .......... .......... .......... .......... 15% 83.0M 6s 2024-09-12T12:57:40.681 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 26100K .......... .......... .......... .......... .......... 15% 32.3M 6s 2024-09-12T12:57:40.684 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 26150K .......... .......... .......... .......... .......... 15% 23.3M 6s 2024-09-12T12:57:40.686 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 26200K .......... .......... .......... .......... .......... 15% 26.7M 6s 2024-09-12T12:57:40.687 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 26250K .......... .......... .......... .......... .......... 15% 44.8M 6s 2024-09-12T12:57:40.689 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 26300K .......... .......... .......... .......... .......... 15% 20.8M 6s 2024-09-12T12:57:40.691 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 26350K .......... .......... .......... .......... .......... 15% 31.6M 6s 2024-09-12T12:57:40.692 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 26400K .......... .......... .......... .......... .......... 15% 39.6M 6s 2024-09-12T12:57:40.694 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 26450K .......... .......... .......... .......... .......... 15% 22.4M 6s 2024-09-12T12:57:40.695 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 26500K .......... .......... .......... .......... .......... 15% 51.1M 6s 2024-09-12T12:57:40.696 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 26550K .......... .......... .......... .......... .......... 15% 28.3M 6s 2024-09-12T12:57:40.699 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 26600K .......... .......... .......... .......... .......... 15% 20.9M 6s 2024-09-12T12:57:40.700 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 26650K .......... .......... .......... .......... .......... 15% 43.2M 6s 2024-09-12T12:57:40.702 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 26700K .......... .......... .......... .......... .......... 15% 24.0M 6s 2024-09-12T12:57:40.704 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 26750K .......... .......... .......... .......... .......... 15% 24.1M 6s 2024-09-12T12:57:40.705 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 26800K .......... .......... .......... .......... .......... 15% 38.4M 6s 2024-09-12T12:57:40.707 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 26850K .......... .......... .......... .......... .......... 15% 26.8M 6s 2024-09-12T12:57:40.709 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 26900K .......... .......... .......... .......... .......... 15% 28.8M 6s 2024-09-12T12:57:40.711 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 26950K .......... .......... .......... .......... .......... 16% 28.0M 6s 2024-09-12T12:57:40.712 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 27000K .......... .......... .......... .......... .......... 16% 34.6M 6s 2024-09-12T12:57:40.714 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 27050K .......... .......... .......... .......... .......... 16% 26.6M 6s 2024-09-12T12:57:40.715 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 27100K .......... .......... .......... .......... .......... 16% 30.9M 6s 2024-09-12T12:57:40.717 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 27150K .......... .......... .......... .......... .......... 16% 24.3M 6s 2024-09-12T12:57:40.719 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 27200K .......... .......... .......... .......... .......... 16% 36.4M 6s 2024-09-12T12:57:40.721 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 27250K .......... .......... .......... .......... .......... 16% 22.1M 6s 2024-09-12T12:57:40.722 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 27300K .......... .......... .......... .......... .......... 16% 34.9M 6s 2024-09-12T12:57:40.724 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 27350K .......... .......... .......... .......... .......... 16% 22.8M 6s 2024-09-12T12:57:40.726 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 27400K .......... .......... .......... .......... .......... 16% 25.9M 6s 2024-09-12T12:57:40.728 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 27450K .......... .......... .......... .......... .......... 16% 24.6M 6s 2024-09-12T12:57:40.729 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 27500K .......... .......... .......... .......... .......... 16% 39.9M 6s 2024-09-12T12:57:40.731 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 27550K .......... .......... .......... .......... .......... 16% 23.6M 6s 2024-09-12T12:57:40.734 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 27600K .......... .......... .......... .......... .......... 16% 23.4M 6s 2024-09-12T12:57:40.735 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 27650K .......... .......... .......... .......... .......... 16% 36.1M 6s 2024-09-12T12:57:40.737 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 27700K .......... .......... .......... .......... .......... 16% 24.1M 6s 2024-09-12T12:57:40.739 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 27750K .......... .......... .......... .......... .......... 16% 32.4M 6s 2024-09-12T12:57:40.740 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 27800K .......... .......... .......... .......... .......... 16% 23.1M 6s 2024-09-12T12:57:40.742 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 27850K .......... .......... .......... .......... .......... 16% 38.1M 6s 2024-09-12T12:57:40.744 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 27900K .......... .......... .......... .......... .......... 16% 23.7M 6s 2024-09-12T12:57:40.746 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 27950K .......... .......... .......... .......... .......... 16% 25.2M 6s 2024-09-12T12:57:40.748 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 28000K .......... .......... .......... .......... .......... 16% 23.5M 6s 2024-09-12T12:57:40.749 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 28050K .......... .......... .......... .......... .......... 16% 48.1M 6s 2024-09-12T12:57:40.751 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 28100K .......... .......... .......... .......... .......... 16% 24.5M 6s 2024-09-12T12:57:40.752 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 28150K .......... .......... .......... .......... .......... 16% 35.0M 6s 2024-09-12T12:57:40.755 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 28200K .......... .......... .......... .......... .......... 16% 23.1M 6s 2024-09-12T12:57:40.756 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 28250K .......... .......... .......... .......... .......... 16% 43.4M 6s 2024-09-12T12:57:40.757 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 28300K .......... .......... .......... .......... .......... 16% 26.8M 6s 2024-09-12T12:57:40.760 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 28350K .......... .......... .......... .......... .......... 16% 24.5M 6s 2024-09-12T12:57:40.762 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 28400K .......... .......... .......... .......... .......... 16% 30.5M 6s 2024-09-12T12:57:40.764 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 28450K .......... .......... .......... .......... .......... 16% 20.7M 6s 2024-09-12T12:57:40.766 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 28500K .......... .......... .......... .......... .......... 16% 25.5M 6s 2024-09-12T12:57:40.766 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 28550K .......... .......... .......... .......... .......... 16% 57.6M 6s 2024-09-12T12:57:40.769 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 28600K .......... .......... .......... .......... .......... 16% 27.9M 6s 2024-09-12T12:57:40.770 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 28650K .......... .......... .......... .......... .......... 17% 24.2M 6s 2024-09-12T12:57:40.772 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 28700K .......... .......... .......... .......... .......... 17% 27.3M 6s 2024-09-12T12:57:40.774 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 28750K .......... .......... .......... .......... .......... 17% 23.8M 6s 2024-09-12T12:57:40.775 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 28800K .......... .......... .......... .......... .......... 17% 45.1M 6s 2024-09-12T12:57:40.776 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 28850K .......... .......... .......... .......... .......... 17% 31.1M 6s 2024-09-12T12:57:40.778 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 28900K .......... .......... .......... .......... .......... 17% 24.9M 6s 2024-09-12T12:57:40.781 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 28950K .......... .......... .......... .......... .......... 17% 22.3M 6s 2024-09-12T12:57:40.783 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 29000K .......... .......... .......... .......... .......... 17% 22.8M 6s 2024-09-12T12:57:40.784 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 29050K .......... .......... .......... .......... .......... 17% 54.3M 6s 2024-09-12T12:57:40.786 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 29100K .......... .......... .......... .......... .......... 17% 32.8M 6s 2024-09-12T12:57:40.789 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 29150K .......... .......... .......... .......... .......... 17% 22.3M 6s 2024-09-12T12:57:40.789 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 29200K .......... .......... .......... .......... .......... 17% 27.4M 6s 2024-09-12T12:57:40.791 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 29250K .......... .......... .......... .......... .......... 17% 27.2M 6s 2024-09-12T12:57:40.792 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 29300K .......... .......... .......... .......... .......... 17% 36.0M 6s 2024-09-12T12:57:40.795 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 29350K .......... .......... .......... .......... .......... 17% 23.1M 6s 2024-09-12T12:57:40.796 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 29400K .......... .......... .......... .......... .......... 17% 40.8M 6s 2024-09-12T12:57:40.798 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 29450K .......... .......... .......... .......... .......... 17% 24.4M 6s 2024-09-12T12:57:40.801 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 29500K .......... .......... .......... .......... .......... 17% 20.4M 6s 2024-09-12T12:57:40.802 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 29550K .......... .......... .......... .......... .......... 17% 24.1M 6s 2024-09-12T12:57:40.803 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 29600K .......... .......... .......... .......... .......... 17% 38.9M 6s 2024-09-12T12:57:40.810 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 29650K .......... .......... .......... .......... .......... 17% 12.4M 6s 2024-09-12T12:57:40.810 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 29700K .......... .......... .......... .......... .......... 17% 150M 6s 2024-09-12T12:57:40.810 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 29750K .......... .......... .......... .......... .......... 17% 42.8M 6s 2024-09-12T12:57:40.810 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 29800K .......... .......... .......... .......... .......... 17% 49.0M 6s 2024-09-12T12:57:40.812 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 29850K .......... .......... .......... .......... .......... 17% 24.7M 6s 2024-09-12T12:57:40.813 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 29900K .......... .......... .......... .......... .......... 17% 38.1M 6s 2024-09-12T12:57:40.815 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 29950K .......... .......... .......... .......... .......... 17% 20.6M 6s 2024-09-12T12:57:40.817 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 30000K .......... .......... .......... .......... .......... 17% 25.8M 6s 2024-09-12T12:57:40.820 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 30050K .......... .......... .......... .......... .......... 17% 23.3M 6s 2024-09-12T12:57:40.820 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 30100K .......... .......... .......... .......... .......... 17% 58.4M 6s 2024-09-12T12:57:40.822 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 30150K .......... .......... .......... .......... .......... 17% 22.3M 6s 2024-09-12T12:57:40.824 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 30200K .......... .......... .......... .......... .......... 17% 32.4M 6s 2024-09-12T12:57:40.826 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 30250K .......... .......... .......... .......... .......... 17% 22.9M 6s 2024-09-12T12:57:40.827 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 30300K .......... .......... .......... .......... .......... 18% 56.9M 6s 2024-09-12T12:57:40.828 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 30350K .......... .......... .......... .......... .......... 18% 26.9M 6s 2024-09-12T12:57:40.831 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 30400K .......... .......... .......... .......... .......... 18% 22.8M 6s 2024-09-12T12:57:40.832 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 30450K .......... .......... .......... .......... .......... 18% 36.3M 6s 2024-09-12T12:57:40.835 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 30500K .......... .......... .......... .......... .......... 18% 24.7M 6s 2024-09-12T12:57:40.836 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 30550K .......... .......... .......... .......... .......... 18% 36.0M 6s 2024-09-12T12:57:40.838 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 30600K .......... .......... .......... .......... .......... 18% 23.5M 6s 2024-09-12T12:57:40.839 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 30650K .......... .......... .......... .......... .......... 18% 39.3M 6s 2024-09-12T12:57:40.842 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 30700K .......... .......... .......... .......... .......... 18% 22.9M 6s 2024-09-12T12:57:40.843 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 30750K .......... .......... .......... .......... .......... 18% 27.3M 6s 2024-09-12T12:57:40.846 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 30800K .......... .......... .......... .......... .......... 18% 21.5M 6s 2024-09-12T12:57:40.846 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 30850K .......... .......... .......... .......... .......... 18% 54.1M 6s 2024-09-12T12:57:40.848 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 30900K .......... .......... .......... .......... .......... 18% 26.0M 6s 2024-09-12T12:57:40.849 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 30950K .......... .......... .......... .......... .......... 18% 34.1M 6s 2024-09-12T12:57:40.851 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 31000K .......... .......... .......... .......... .......... 18% 24.1M 6s 2024-09-12T12:57:40.852 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 31050K .......... .......... .......... .......... .......... 18% 52.8M 6s 2024-09-12T12:57:40.854 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 31100K .......... .......... .......... .......... .......... 18% 24.5M 6s 2024-09-12T12:57:40.856 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 31150K .......... .......... .......... .......... .......... 18% 22.5M 6s 2024-09-12T12:57:40.858 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 31200K .......... .......... .......... .......... .......... 18% 34.9M 6s 2024-09-12T12:57:40.860 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 31250K .......... .......... .......... .......... .......... 18% 26.9M 6s 2024-09-12T12:57:40.862 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 31300K .......... .......... .......... .......... .......... 18% 22.6M 6s 2024-09-12T12:57:40.863 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 31350K .......... .......... .......... .......... .......... 18% 52.4M 6s 2024-09-12T12:57:40.865 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 31400K .......... .......... .......... .......... .......... 18% 24.5M 6s 2024-09-12T12:57:40.866 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 31450K .......... .......... .......... .......... .......... 18% 36.8M 6s 2024-09-12T12:57:40.870 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 31500K .......... .......... .......... .......... .......... 18% 16.3M 6s 2024-09-12T12:57:40.871 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 31550K .......... .......... .......... .......... .......... 18% 24.6M 6s 2024-09-12T12:57:40.872 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 31600K .......... .......... .......... .......... .......... 18% 45.6M 6s 2024-09-12T12:57:40.874 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 31650K .......... .......... .......... .......... .......... 18% 24.9M 6s 2024-09-12T12:57:40.875 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 31700K .......... .......... .......... .......... .......... 18% 35.9M 6s 2024-09-12T12:57:40.877 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 31750K .......... .......... .......... .......... .......... 18% 23.8M 6s 2024-09-12T12:57:40.880 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 31800K .......... .......... .......... .......... .......... 18% 23.5M 6s 2024-09-12T12:57:40.880 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 31850K .......... .......... .......... .......... .......... 18% 67.2M 6s 2024-09-12T12:57:40.883 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 31900K .......... .......... .......... .......... .......... 18% 18.2M 6s 2024-09-12T12:57:40.885 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 31950K .......... .......... .......... .......... .......... 18% 25.3M 6s 2024-09-12T12:57:40.886 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 32000K .......... .......... .......... .......... .......... 19% 36.3M 6s 2024-09-12T12:57:40.888 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 32050K .......... .......... .......... .......... .......... 19% 25.0M 6s 2024-09-12T12:57:40.889 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 32100K .......... .......... .......... .......... .......... 19% 51.6M 6s 2024-09-12T12:57:40.891 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 32150K .......... .......... .......... .......... .......... 19% 22.8M 6s 2024-09-12T12:57:40.893 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 32200K .......... .......... .......... .......... .......... 19% 33.8M 6s 2024-09-12T12:57:40.895 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 32250K .......... .......... .......... .......... .......... 19% 26.8M 6s 2024-09-12T12:57:40.896 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 32300K .......... .......... .......... .......... .......... 19% 40.0M 6s 2024-09-12T12:57:40.900 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 32350K .......... .......... .......... .......... .......... 19% 15.8M 6s 2024-09-12T12:57:40.900 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 32400K .......... .......... .......... .......... .......... 19% 87.6M 6s 2024-09-12T12:57:40.902 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 32450K .......... .......... .......... .......... .......... 19% 19.8M 6s 2024-09-12T12:57:40.903 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 32500K .......... .......... .......... .......... .......... 19% 42.8M 6s 2024-09-12T12:57:40.905 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 32550K .......... .......... .......... .......... .......... 19% 22.9M 6s 2024-09-12T12:57:40.906 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 32600K .......... .......... .......... .......... .......... 19% 36.8M 6s 2024-09-12T12:57:40.908 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 32650K .......... .......... .......... .......... .......... 19% 23.0M 6s 2024-09-12T12:57:40.910 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 32700K .......... .......... .......... .......... .......... 19% 35.4M 6s 2024-09-12T12:57:40.912 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 32750K .......... .......... .......... .......... .......... 19% 25.5M 6s 2024-09-12T12:57:40.914 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 32800K .......... .......... .......... .......... .......... 19% 24.9M 6s 2024-09-12T12:57:40.915 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 32850K .......... .......... .......... .......... .......... 19% 40.1M 6s 2024-09-12T12:57:40.917 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 32900K .......... .......... .......... .......... .......... 19% 25.8M 6s 2024-09-12T12:57:40.919 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 32950K .......... .......... .......... .......... .......... 19% 27.2M 6s 2024-09-12T12:57:40.920 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 33000K .......... .......... .......... .......... .......... 19% 27.3M 6s 2024-09-12T12:57:40.923 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 33050K .......... .......... .......... .......... .......... 19% 20.9M 6s 2024-09-12T12:57:40.924 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 33100K .......... .......... .......... .......... .......... 19% 58.3M 6s 2024-09-12T12:57:40.926 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 33150K .......... .......... .......... .......... .......... 19% 23.5M 6s 2024-09-12T12:57:40.928 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 33200K .......... .......... .......... .......... .......... 19% 25.8M 6s 2024-09-12T12:57:40.929 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 33250K .......... .......... .......... .......... .......... 19% 38.6M 6s 2024-09-12T12:57:40.931 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 33300K .......... .......... .......... .......... .......... 19% 24.8M 6s 2024-09-12T12:57:40.932 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 33350K .......... .......... .......... .......... .......... 19% 43.6M 6s 2024-09-12T12:57:40.934 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 33400K .......... .......... .......... .......... .......... 19% 24.9M 6s 2024-09-12T12:57:40.936 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 33450K .......... .......... .......... .......... .......... 19% 29.0M 6s 2024-09-12T12:57:40.938 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 33500K .......... .......... .......... .......... .......... 19% 24.4M 6s 2024-09-12T12:57:40.940 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 33550K .......... .......... .......... .......... .......... 19% 23.7M 6s 2024-09-12T12:57:40.942 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 33600K .......... .......... .......... .......... .......... 19% 24.7M 6s 2024-09-12T12:57:40.943 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 33650K .......... .......... .......... .......... .......... 19% 37.0M 6s 2024-09-12T12:57:40.945 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 33700K .......... .......... .......... .......... .......... 20% 25.2M 6s 2024-09-12T12:57:40.947 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 33750K .......... .......... .......... .......... .......... 20% 27.6M 6s 2024-09-12T12:57:40.949 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 33800K .......... .......... .......... .......... .......... 20% 25.5M 6s 2024-09-12T12:57:40.949 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 33850K .......... .......... .......... .......... .......... 20% 46.9M 6s 2024-09-12T12:57:40.952 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 33900K .......... .......... .......... .......... .......... 20% 22.4M 6s 2024-09-12T12:57:40.954 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 33950K .......... .......... .......... .......... .......... 20% 26.5M 6s 2024-09-12T12:57:40.955 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 34000K .......... .......... .......... .......... .......... 20% 37.7M 6s 2024-09-12T12:57:40.961 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 34050K .......... .......... .......... .......... .......... 20% 22.0M 6s 2024-09-12T12:57:40.961 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 34100K .......... .......... .......... .......... .......... 20% 26.7M 6s 2024-09-12T12:57:40.961 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 34150K .......... .......... .......... .......... .......... 20% 41.8M 6s 2024-09-12T12:57:40.962 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 34200K .......... .......... .......... .......... .......... 20% 24.5M 6s 2024-09-12T12:57:40.963 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 34250K .......... .......... .......... .......... .......... 20% 41.6M 6s 2024-09-12T12:57:40.966 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 34300K .......... .......... .......... .......... .......... 20% 21.8M 6s 2024-09-12T12:57:40.968 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 34350K .......... .......... .......... .......... .......... 20% 24.4M 6s 2024-09-12T12:57:40.969 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 34400K .......... .......... .......... .......... .......... 20% 53.2M 6s 2024-09-12T12:57:40.971 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 34450K .......... .......... .......... .......... .......... 20% 20.9M 6s 2024-09-12T12:57:40.972 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 34500K .......... .......... .......... .......... .......... 20% 37.7M 6s 2024-09-12T12:57:40.974 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 34550K .......... .......... .......... .......... .......... 20% 27.1M 5s 2024-09-12T12:57:40.977 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 34600K .......... .......... .......... .......... .......... 20% 22.7M 5s 2024-09-12T12:57:40.977 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 34650K .......... .......... .......... .......... .......... 20% 53.4M 5s 2024-09-12T12:57:40.979 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 34700K .......... .......... .......... .......... .......... 20% 21.5M 5s 2024-09-12T12:57:40.982 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 34750K .......... .......... .......... .......... .......... 20% 25.2M 5s 2024-09-12T12:57:40.982 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 34800K .......... .......... .......... .......... .......... 20% 36.2M 5s 2024-09-12T12:57:40.986 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 34850K .......... .......... .......... .......... .......... 20% 23.4M 5s 2024-09-12T12:57:40.986 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 34900K .......... .......... .......... .......... .......... 20% 52.1M 5s 2024-09-12T12:57:40.988 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 34950K .......... .......... .......... .......... .......... 20% 24.1M 5s 2024-09-12T12:57:40.990 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 35000K .......... .......... .......... .......... .......... 20% 36.9M 5s 2024-09-12T12:57:40.994 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 35050K .......... .......... .......... .......... .......... 20% 24.0M 5s 2024-09-12T12:57:40.994 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 35100K .......... .......... .......... .......... .......... 20% 50.7M 5s 2024-09-12T12:57:40.996 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 35150K .......... .......... .......... .......... .......... 20% 15.6M 5s 2024-09-12T12:57:40.996 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 35200K .......... .......... .......... .......... .......... 20% 58.6M 5s 2024-09-12T12:57:40.998 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 35250K .......... .......... .......... .......... .......... 20% 23.0M 5s 2024-09-12T12:57:41.000 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 35300K .......... .......... .......... .......... .......... 20% 32.5M 5s 2024-09-12T12:57:41.001 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 35350K .......... .......... .......... .......... .......... 21% 25.5M 5s 2024-09-12T12:57:41.002 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 35400K .......... .......... .......... .......... .......... 21% 54.8M 5s 2024-09-12T12:57:41.005 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 35450K .......... .......... .......... .......... .......... 21% 23.9M 5s 2024-09-12T12:57:41.009 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 35500K .......... .......... .......... .......... .......... 21% 33.4M 5s 2024-09-12T12:57:41.010 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 35550K .......... .......... .......... .......... .......... 21% 24.1M 5s 2024-09-12T12:57:41.011 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 35600K .......... .......... .......... .......... .......... 21% 20.7M 5s 2024-09-12T12:57:41.014 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 35650K .......... .......... .......... .......... .......... 21% 30.9M 5s 2024-09-12T12:57:41.014 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 35700K .......... .......... .......... .......... .......... 21% 24.4M 5s 2024-09-12T12:57:41.016 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 35750K .......... .......... .......... .......... .......... 21% 33.4M 5s 2024-09-12T12:57:41.018 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 35800K .......... .......... .......... .......... .......... 21% 27.1M 5s 2024-09-12T12:57:41.019 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 35850K .......... .......... .......... .......... .......... 21% 23.0M 5s 2024-09-12T12:57:41.020 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 35900K .......... .......... .......... .......... .......... 21% 43.4M 5s 2024-09-12T12:57:41.022 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 35950K .......... .......... .......... .......... .......... 21% 22.2M 5s 2024-09-12T12:57:41.025 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 36000K .......... .......... .......... .......... .......... 21% 24.9M 5s 2024-09-12T12:57:41.026 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 36050K .......... .......... .......... .......... .......... 21% 36.4M 5s 2024-09-12T12:57:41.028 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 36100K .......... .......... .......... .......... .......... 21% 23.1M 5s 2024-09-12T12:57:41.029 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 36150K .......... .......... .......... .......... .......... 21% 54.6M 5s 2024-09-12T12:57:41.031 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 36200K .......... .......... .......... .......... .......... 21% 22.0M 5s 2024-09-12T12:57:41.032 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 36250K .......... .......... .......... .......... .......... 21% 43.1M 5s 2024-09-12T12:57:41.035 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 36300K .......... .......... .......... .......... .......... 21% 22.3M 5s 2024-09-12T12:57:41.036 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 36350K .......... .......... .......... .......... .......... 21% 25.8M 5s 2024-09-12T12:57:41.039 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 36400K .......... .......... .......... .......... .......... 21% 22.8M 5s 2024-09-12T12:57:41.039 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 36450K .......... .......... .......... .......... .......... 21% 56.4M 5s 2024-09-12T12:57:41.041 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 36500K .......... .......... .......... .......... .......... 21% 24.6M 5s 2024-09-12T12:57:41.044 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 36550K .......... .......... .......... .......... .......... 21% 29.8M 5s 2024-09-12T12:57:41.046 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 36600K .......... .......... .......... .......... .......... 21% 26.8M 5s 2024-09-12T12:57:41.046 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 36650K .......... .......... .......... .......... .......... 21% 46.4M 5s 2024-09-12T12:57:41.048 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 36700K .......... .......... .......... .......... .......... 21% 22.6M 5s 2024-09-12T12:57:41.052 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 36750K .......... .......... .......... .......... .......... 21% 21.0M 5s 2024-09-12T12:57:41.053 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 36800K .......... .......... .......... .......... .......... 21% 43.0M 5s 2024-09-12T12:57:41.054 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 36850K .......... .......... .......... .......... .......... 21% 23.1M 5s 2024-09-12T12:57:41.056 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 36900K .......... .......... .......... .......... .......... 21% 22.8M 5s 2024-09-12T12:57:41.056 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 36950K .......... .......... .......... .......... .......... 21% 55.5M 5s 2024-09-12T12:57:41.060 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 37000K .......... .......... .......... .......... .......... 21% 21.7M 5s 2024-09-12T12:57:41.060 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 37050K .......... .......... .......... .......... .......... 22% 41.4M 5s 2024-09-12T12:57:41.062 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 37100K .......... .......... .......... .......... .......... 22% 22.8M 5s 2024-09-12T12:57:41.064 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 37150K .......... .......... .......... .......... .......... 22% 21.7M 5s 2024-09-12T12:57:41.065 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 37200K .......... .......... .......... .......... .......... 22% 53.3M 5s 2024-09-12T12:57:41.067 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 37250K .......... .......... .......... .......... .......... 22% 23.6M 5s 2024-09-12T12:57:41.069 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 37300K .......... .......... .......... .......... .......... 22% 32.0M 5s 2024-09-12T12:57:41.071 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 37350K .......... .......... .......... .......... .......... 22% 23.0M 5s 2024-09-12T12:57:41.072 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 37400K .......... .......... .......... .......... .......... 22% 58.8M 5s 2024-09-12T12:57:41.074 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 37450K .......... .......... .......... .......... .......... 22% 22.9M 5s 2024-09-12T12:57:41.076 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 37500K .......... .......... .......... .......... .......... 22% 37.8M 5s 2024-09-12T12:57:41.079 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 37550K .......... .......... .......... .......... .......... 22% 16.0M 5s 2024-09-12T12:57:41.080 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 37600K .......... .......... .......... .......... .......... 22% 36.3M 5s 2024-09-12T12:57:41.082 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 37650K .......... .......... .......... .......... .......... 22% 24.5M 5s 2024-09-12T12:57:41.083 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 37700K .......... .......... .......... .......... .......... 22% 39.3M 5s 2024-09-12T12:57:41.085 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 37750K .......... .......... .......... .......... .......... 22% 26.2M 5s 2024-09-12T12:57:41.087 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 37800K .......... .......... .......... .......... .......... 22% 26.7M 5s 2024-09-12T12:57:41.088 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 37850K .......... .......... .......... .......... .......... 22% 58.8M 5s 2024-09-12T12:57:41.089 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 37900K .......... .......... .......... .......... .......... 22% 29.6M 5s 2024-09-12T12:57:41.092 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 37950K .......... .......... .......... .......... .......... 22% 16.8M 5s 2024-09-12T12:57:41.094 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 38000K .......... .......... .......... .......... .......... 22% 28.8M 5s 2024-09-12T12:57:41.095 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 38050K .......... .......... .......... .......... .......... 22% 30.6M 5s 2024-09-12T12:57:41.097 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 38100K .......... .......... .......... .......... .......... 22% 25.8M 5s 2024-09-12T12:57:41.099 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 38150K .......... .......... .......... .......... .......... 22% 24.6M 5s 2024-09-12T12:57:41.100 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 38200K .......... .......... .......... .......... .......... 22% 40.3M 5s 2024-09-12T12:57:41.103 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 38250K .......... .......... .......... .......... .......... 22% 26.5M 5s 2024-09-12T12:57:41.103 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 38300K .......... .......... .......... .......... .......... 22% 33.0M 5s 2024-09-12T12:57:41.105 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 38350K .......... .......... .......... .......... .......... 22% 26.5M 5s 2024-09-12T12:57:41.108 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 38400K .......... .......... .......... .......... .......... 22% 21.0M 5s 2024-09-12T12:57:41.108 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 38450K .......... .......... .......... .......... .......... 22% 72.5M 5s 2024-09-12T12:57:41.111 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 38500K .......... .......... .......... .......... .......... 22% 23.8M 5s 2024-09-12T12:57:41.112 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 38550K .......... .......... .......... .......... .......... 22% 40.8M 5s 2024-09-12T12:57:41.114 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 38600K .......... .......... .......... .......... .......... 22% 18.3M 5s 2024-09-12T12:57:41.116 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 38650K .......... .......... .......... .......... .......... 22% 31.0M 5s 2024-09-12T12:57:41.117 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 38700K .......... .......... .......... .......... .......... 22% 40.3M 5s 2024-09-12T12:57:41.119 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 38750K .......... .......... .......... .......... .......... 23% 21.8M 5s 2024-09-12T12:57:41.121 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 38800K .......... .......... .......... .......... .......... 23% 27.2M 5s 2024-09-12T12:57:41.123 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 38850K .......... .......... .......... .......... .......... 23% 36.6M 5s 2024-09-12T12:57:41.125 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 38900K .......... .......... .......... .......... .......... 23% 24.4M 5s 2024-09-12T12:57:41.126 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 38950K .......... .......... .......... .......... .......... 23% 40.5M 5s 2024-09-12T12:57:41.128 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 39000K .......... .......... .......... .......... .......... 23% 23.3M 5s 2024-09-12T12:57:41.129 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 39050K .......... .......... .......... .......... .......... 23% 39.2M 5s 2024-09-12T12:57:41.131 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 39100K .......... .......... .......... .......... .......... 23% 26.9M 5s 2024-09-12T12:57:41.133 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 39150K .......... .......... .......... .......... .......... 23% 20.8M 5s 2024-09-12T12:57:41.135 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 39200K .......... .......... .......... .......... .......... 23% 25.9M 5s 2024-09-12T12:57:41.136 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 39250K .......... .......... .......... .......... .......... 23% 42.4M 5s 2024-09-12T12:57:41.138 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 39300K .......... .......... .......... .......... .......... 23% 26.4M 5s 2024-09-12T12:57:41.139 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 39350K .......... .......... .......... .......... .......... 23% 36.7M 5s 2024-09-12T12:57:41.141 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 39400K .......... .......... .......... .......... .......... 23% 24.1M 5s 2024-09-12T12:57:41.143 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 39450K .......... .......... .......... .......... .......... 23% 43.0M 5s 2024-09-12T12:57:41.145 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 39500K .......... .......... .......... .......... .......... 23% 24.4M 5s 2024-09-12T12:57:41.148 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 39550K .......... .......... .......... .......... .......... 23% 26.0M 5s 2024-09-12T12:57:41.150 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 39600K .......... .......... .......... .......... .......... 23% 23.4M 5s 2024-09-12T12:57:41.150 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 39650K .......... .......... .......... .......... .......... 23% 42.7M 5s 2024-09-12T12:57:41.153 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 39700K .......... .......... .......... .......... .......... 23% 22.4M 5s 2024-09-12T12:57:41.153 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 39750K .......... .......... .......... .......... .......... 23% 49.4M 5s 2024-09-12T12:57:41.156 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 39800K .......... .......... .......... .......... .......... 23% 15.3M 5s 2024-09-12T12:57:41.157 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 39850K .......... .......... .......... .......... .......... 23% 47.6M 5s 2024-09-12T12:57:41.160 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 39900K .......... .......... .......... .......... .......... 23% 23.4M 5s 2024-09-12T12:57:41.160 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 39950K .......... .......... .......... .......... .......... 23% 45.3M 5s 2024-09-12T12:57:41.162 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 40000K .......... .......... .......... .......... .......... 23% 25.9M 5s 2024-09-12T12:57:41.164 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 40050K .......... .......... .......... .......... .......... 23% 22.8M 5s 2024-09-12T12:57:41.165 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 40100K .......... .......... .......... .......... .......... 23% 40.5M 5s 2024-09-12T12:57:41.168 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 40150K .......... .......... .......... .......... .......... 23% 23.2M 5s 2024-09-12T12:57:41.169 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 40200K .......... .......... .......... .......... .......... 23% 44.7M 5s 2024-09-12T12:57:41.170 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 40250K .......... .......... .......... .......... .......... 23% 25.2M 5s 2024-09-12T12:57:41.172 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 40300K .......... .......... .......... .......... .......... 23% 29.6M 5s 2024-09-12T12:57:41.175 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 40350K .......... .......... .......... .......... .......... 23% 17.5M 5s 2024-09-12T12:57:41.176 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 40400K .......... .......... .......... .......... .......... 23% 39.2M 5s 2024-09-12T12:57:41.178 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 40450K .......... .......... .......... .......... .......... 24% 24.9M 5s 2024-09-12T12:57:41.179 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 40500K .......... .......... .......... .......... .......... 24% 45.3M 5s 2024-09-12T12:57:41.182 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 40550K .......... .......... .......... .......... .......... 24% 21.8M 5s 2024-09-12T12:57:41.182 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 40600K .......... .......... .......... .......... .......... 24% 49.9M 5s 2024-09-12T12:57:41.185 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 40650K .......... .......... .......... .......... .......... 24% 22.4M 5s 2024-09-12T12:57:41.186 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 40700K .......... .......... .......... .......... .......... 24% 42.5M 5s 2024-09-12T12:57:41.188 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 40750K .......... .......... .......... .......... .......... 24% 27.5M 5s 2024-09-12T12:57:41.189 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 40800K .......... .......... .......... .......... .......... 24% 27.3M 5s 2024-09-12T12:57:41.192 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 40850K .......... .......... .......... .......... .......... 24% 23.7M 5s 2024-09-12T12:57:41.194 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 40900K .......... .......... .......... .......... .......... 24% 22.2M 5s 2024-09-12T12:57:41.197 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 40950K .......... .......... .......... .......... .......... 24% 27.7M 5s 2024-09-12T12:57:41.197 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 41000K .......... .......... .......... .......... .......... 24% 37.7M 5s 2024-09-12T12:57:41.198 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 41050K .......... .......... .......... .......... .......... 24% 27.7M 5s 2024-09-12T12:57:41.199 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 41100K .......... .......... .......... .......... .......... 24% 52.2M 5s 2024-09-12T12:57:41.202 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 41150K .......... .......... .......... .......... .......... 24% 19.2M 5s 2024-09-12T12:57:41.204 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 41200K .......... .......... .......... .......... .......... 24% 23.7M 5s 2024-09-12T12:57:41.205 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 41250K .......... .......... .......... .......... .......... 24% 41.2M 5s 2024-09-12T12:57:41.207 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 41300K .......... .......... .......... .......... .......... 24% 25.6M 5s 2024-09-12T12:57:41.209 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 41350K .......... .......... .......... .......... .......... 24% 29.3M 5s 2024-09-12T12:57:41.211 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 41400K .......... .......... .......... .......... .......... 24% 26.2M 5s 2024-09-12T12:57:41.213 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 41450K .......... .......... .......... .......... .......... 24% 25.7M 5s 2024-09-12T12:57:41.214 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 41500K .......... .......... .......... .......... .......... 24% 39.9M 5s 2024-09-12T12:57:41.216 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 41550K .......... .......... .......... .......... .......... 24% 23.1M 5s 2024-09-12T12:57:41.218 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 41600K .......... .......... .......... .......... .......... 24% 25.7M 5s 2024-09-12T12:57:41.220 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 41650K .......... .......... .......... .......... .......... 24% 33.5M 5s 2024-09-12T12:57:41.221 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 41700K .......... .......... .......... .......... .......... 24% 28.0M 5s 2024-09-12T12:57:41.222 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 41750K .......... .......... .......... .......... .......... 24% 34.5M 5s 2024-09-12T12:57:41.224 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 41800K .......... .......... .......... .......... .......... 24% 26.1M 5s 2024-09-12T12:57:41.226 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 41850K .......... .......... .......... .......... .......... 24% 38.3M 5s 2024-09-12T12:57:41.228 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 41900K .......... .......... .......... .......... .......... 24% 24.7M 5s 2024-09-12T12:57:41.230 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 41950K .......... .......... .......... .......... .......... 24% 21.0M 5s 2024-09-12T12:57:41.232 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 42000K .......... .......... .......... .......... .......... 24% 23.9M 5s 2024-09-12T12:57:41.233 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 42050K .......... .......... .......... .......... .......... 24% 52.3M 5s 2024-09-12T12:57:41.236 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 42100K .......... .......... .......... .......... .......... 25% 19.5M 5s 2024-09-12T12:57:41.237 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 42150K .......... .......... .......... .......... .......... 25% 34.2M 5s 2024-09-12T12:57:41.239 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 42200K .......... .......... .......... .......... .......... 25% 27.9M 5s 2024-09-12T12:57:41.240 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 42250K .......... .......... .......... .......... .......... 25% 46.1M 5s 2024-09-12T12:57:41.242 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 42300K .......... .......... .......... .......... .......... 25% 23.8M 5s 2024-09-12T12:57:41.244 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 42350K .......... .......... .......... .......... .......... 25% 24.2M 5s 2024-09-12T12:57:41.245 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 42400K .......... .......... .......... .......... .......... 25% 39.8M 5s 2024-09-12T12:57:41.247 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 42450K .......... .......... .......... .......... .......... 25% 24.1M 5s 2024-09-12T12:57:41.248 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 42500K .......... .......... .......... .......... .......... 25% 48.1M 5s 2024-09-12T12:57:41.250 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 42550K .......... .......... .......... .......... .......... 25% 25.6M 5s 2024-09-12T12:57:41.252 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 42600K .......... .......... .......... .......... .......... 25% 28.3M 5s 2024-09-12T12:57:41.253 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 42650K .......... .......... .......... .......... .......... 25% 29.8M 5s 2024-09-12T12:57:41.256 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 42700K .......... .......... .......... .......... .......... 25% 20.8M 5s 2024-09-12T12:57:41.258 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 42750K .......... .......... .......... .......... .......... 25% 23.3M 5s 2024-09-12T12:57:41.259 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 42800K .......... .......... .......... .......... .......... 25% 44.4M 5s 2024-09-12T12:57:41.261 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 42850K .......... .......... .......... .......... .......... 25% 23.0M 5s 2024-09-12T12:57:41.263 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 42900K .......... .......... .......... .......... .......... 25% 38.0M 5s 2024-09-12T12:57:41.264 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 42950K .......... .......... .......... .......... .......... 25% 26.9M 5s 2024-09-12T12:57:41.265 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 43000K .......... .......... .......... .......... .......... 25% 45.3M 5s 2024-09-12T12:57:41.267 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 43050K .......... .......... .......... .......... .......... 25% 23.0M 5s 2024-09-12T12:57:41.269 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 43100K .......... .......... .......... .......... .......... 25% 29.8M 5s 2024-09-12T12:57:41.271 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 43150K .......... .......... .......... .......... .......... 25% 23.3M 5s 2024-09-12T12:57:41.273 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 43200K .......... .......... .......... .......... .......... 25% 27.4M 5s 2024-09-12T12:57:41.275 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 43250K .......... .......... .......... .......... .......... 25% 25.1M 5s 2024-09-12T12:57:41.276 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 43300K .......... .......... .......... .......... .......... 25% 48.7M 5s 2024-09-12T12:57:41.278 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 43350K .......... .......... .......... .......... .......... 25% 21.5M 5s 2024-09-12T12:57:41.279 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 43400K .......... .......... .......... .......... .......... 25% 41.2M 5s 2024-09-12T12:57:41.281 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 43450K .......... .......... .......... .......... .......... 25% 24.4M 5s 2024-09-12T12:57:41.282 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 43500K .......... .......... .......... .......... .......... 25% 44.1M 5s 2024-09-12T12:57:41.284 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 43550K .......... .......... .......... .......... .......... 25% 25.5M 5s 2024-09-12T12:57:41.286 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 43600K .......... .......... .......... .......... .......... 25% 26.9M 5s 2024-09-12T12:57:41.287 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 43650K .......... .......... .......... .......... .......... 25% 36.9M 5s 2024-09-12T12:57:41.290 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 43700K .......... .......... .......... .......... .......... 25% 20.2M 5s 2024-09-12T12:57:41.292 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 43750K .......... .......... .......... .......... .......... 25% 26.2M 5s 2024-09-12T12:57:41.293 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 43800K .......... .......... .......... .......... .......... 26% 43.2M 5s 2024-09-12T12:57:41.294 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 43850K .......... .......... .......... .......... .......... 26% 27.1M 5s 2024-09-12T12:57:41.297 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 43900K .......... .......... .......... .......... .......... 26% 31.4M 5s 2024-09-12T12:57:41.298 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 43950K .......... .......... .......... .......... .......... 26% 23.1M 5s 2024-09-12T12:57:41.301 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 44000K .......... .......... .......... .......... .......... 26% 21.3M 5s 2024-09-12T12:57:41.302 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 44050K .......... .......... .......... .......... .......... 26% 38.9M 5s 2024-09-12T12:57:41.304 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 44100K .......... .......... .......... .......... .......... 26% 25.2M 5s 2024-09-12T12:57:41.305 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 44150K .......... .......... .......... .......... .......... 26% 35.8M 5s 2024-09-12T12:57:41.306 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 44200K .......... .......... .......... .......... .......... 26% 32.0M 5s 2024-09-12T12:57:41.310 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 44250K .......... .......... .......... .......... .......... 26% 20.4M 5s 2024-09-12T12:57:41.310 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 44300K .......... .......... .......... .......... .......... 26% 51.3M 5s 2024-09-12T12:57:41.312 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 44350K .......... .......... .......... .......... .......... 26% 23.7M 5s 2024-09-12T12:57:41.314 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 44400K .......... .......... .......... .......... .......... 26% 22.6M 5s 2024-09-12T12:57:41.315 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 44450K .......... .......... .......... .......... .......... 26% 37.8M 5s 2024-09-12T12:57:41.317 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 44500K .......... .......... .......... .......... .......... 26% 32.2M 5s 2024-09-12T12:57:41.319 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 44550K .......... .......... .......... .......... .......... 26% 28.7M 5s 2024-09-12T12:57:41.321 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 44600K .......... .......... .......... .......... .......... 26% 21.5M 5s 2024-09-12T12:57:41.322 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 44650K .......... .......... .......... .......... .......... 26% 41.5M 5s 2024-09-12T12:57:41.325 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 44700K .......... .......... .......... .......... .......... 26% 24.7M 5s 2024-09-12T12:57:41.326 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 44750K .......... .......... .......... .......... .......... 26% 25.9M 5s 2024-09-12T12:57:41.328 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 44800K .......... .......... .......... .......... .......... 26% 23.3M 5s 2024-09-12T12:57:41.329 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 44850K .......... .......... .......... .......... .......... 26% 47.8M 5s 2024-09-12T12:57:41.331 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 44900K .......... .......... .......... .......... .......... 26% 23.8M 5s 2024-09-12T12:57:41.332 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 44950K .......... .......... .......... .......... .......... 26% 37.0M 5s 2024-09-12T12:57:41.334 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 45000K .......... .......... .......... .......... .......... 26% 24.6M 5s 2024-09-12T12:57:41.335 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 45050K .......... .......... .......... .......... .......... 26% 43.5M 5s 2024-09-12T12:57:41.338 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 45100K .......... .......... .......... .......... .......... 26% 22.1M 5s 2024-09-12T12:57:41.340 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 45150K .......... .......... .......... .......... .......... 26% 23.0M 5s 2024-09-12T12:57:41.341 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 45200K .......... .......... .......... .......... .......... 26% 42.6M 5s 2024-09-12T12:57:41.343 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 45250K .......... .......... .......... .......... .......... 26% 23.7M 5s 2024-09-12T12:57:41.344 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 45300K .......... .......... .......... .......... .......... 26% 50.1M 5s 2024-09-12T12:57:41.346 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 45350K .......... .......... .......... .......... .......... 26% 22.1M 5s 2024-09-12T12:57:41.348 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 45400K .......... .......... .......... .......... .......... 26% 37.0M 5s 2024-09-12T12:57:41.350 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 45450K .......... .......... .......... .......... .......... 26% 23.0M 5s 2024-09-12T12:57:41.352 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 45500K .......... .......... .......... .......... .......... 27% 24.4M 5s 2024-09-12T12:57:41.355 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 45550K .......... .......... .......... .......... .......... 27% 22.1M 5s 2024-09-12T12:57:41.355 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 45600K .......... .......... .......... .......... .......... 27% 50.7M 5s 2024-09-12T12:57:41.357 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 45650K .......... .......... .......... .......... .......... 27% 24.7M 5s 2024-09-12T12:57:41.358 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 45700K .......... .......... .......... .......... .......... 27% 33.7M 5s 2024-09-12T12:57:41.360 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 45750K .......... .......... .......... .......... .......... 27% 26.0M 5s 2024-09-12T12:57:41.361 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 45800K .......... .......... .......... .......... .......... 27% 42.3M 5s 2024-09-12T12:57:41.363 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 45850K .......... .......... .......... .......... .......... 27% 26.1M 5s 2024-09-12T12:57:41.364 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 45900K .......... .......... .......... .......... .......... 27% 41.0M 5s 2024-09-12T12:57:41.367 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 45950K .......... .......... .......... .......... .......... 27% 17.4M 5s 2024-09-12T12:57:41.370 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 46000K .......... .......... .......... .......... .......... 27% 28.0M 5s 2024-09-12T12:57:41.371 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 46050K .......... .......... .......... .......... .......... 27% 25.8M 5s 2024-09-12T12:57:41.372 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 46100K .......... .......... .......... .......... .......... 27% 49.8M 5s 2024-09-12T12:57:41.374 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 46150K .......... .......... .......... .......... .......... 27% 24.0M 5s 2024-09-12T12:57:41.375 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 46200K .......... .......... .......... .......... .......... 27% 35.6M 5s 2024-09-12T12:57:41.377 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 46250K .......... .......... .......... .......... .......... 27% 22.8M 5s 2024-09-12T12:57:41.378 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 46300K .......... .......... .......... .......... .......... 27% 56.4M 5s 2024-09-12T12:57:41.380 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 46350K .......... .......... .......... .......... .......... 27% 22.3M 5s 2024-09-12T12:57:41.383 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 46400K .......... .......... .......... .......... .......... 27% 23.2M 5s 2024-09-12T12:57:41.384 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 46450K .......... .......... .......... .......... .......... 27% 35.5M 5s 2024-09-12T12:57:41.386 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 46500K .......... .......... .......... .......... .......... 27% 24.7M 5s 2024-09-12T12:57:41.388 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 46550K .......... .......... .......... .......... .......... 27% 23.5M 5s 2024-09-12T12:57:41.389 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 46600K .......... .......... .......... .......... .......... 27% 51.2M 5s 2024-09-12T12:57:41.391 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 46650K .......... .......... .......... .......... .......... 27% 23.2M 5s 2024-09-12T12:57:41.392 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 46700K .......... .......... .......... .......... .......... 27% 42.5M 5s 2024-09-12T12:57:41.394 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 46750K .......... .......... .......... .......... .......... 27% 23.2M 5s 2024-09-12T12:57:41.397 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 46800K .......... .......... .......... .......... .......... 27% 23.8M 5s 2024-09-12T12:57:41.397 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 46850K .......... .......... .......... .......... .......... 27% 49.9M 5s 2024-09-12T12:57:41.401 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 46900K .......... .......... .......... .......... .......... 27% 26.5M 5s 2024-09-12T12:57:41.401 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 46950K .......... .......... .......... .......... .......... 27% 34.4M 5s 2024-09-12T12:57:41.403 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 47000K .......... .......... .......... .......... .......... 27% 20.2M 5s 2024-09-12T12:57:41.405 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 47050K .......... .......... .......... .......... .......... 27% 23.4M 5s 2024-09-12T12:57:41.406 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 47100K .......... .......... .......... .......... .......... 27% 52.6M 5s 2024-09-12T12:57:41.408 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 47150K .......... .......... .......... .......... .......... 28% 26.1M 5s 2024-09-12T12:57:41.410 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 47200K .......... .......... .......... .......... .......... 28% 23.2M 5s 2024-09-12T12:57:41.411 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 47250K .......... .......... .......... .......... .......... 28% 46.8M 5s 2024-09-12T12:57:41.413 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 47300K .......... .......... .......... .......... .......... 28% 21.9M 5s 2024-09-12T12:57:41.414 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 47350K .......... .......... .......... .......... .......... 28% 43.8M 5s 2024-09-12T12:57:41.417 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 47400K .......... .......... .......... .......... .......... 28% 21.8M 5s 2024-09-12T12:57:41.418 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 47450K .......... .......... .......... .......... .......... 28% 39.6M 5s 2024-09-12T12:57:41.420 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 47500K .......... .......... .......... .......... .......... 28% 25.6M 5s 2024-09-12T12:57:41.422 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 47550K .......... .......... .......... .......... .......... 28% 22.5M 5s 2024-09-12T12:57:41.424 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 47600K .......... .......... .......... .......... .......... 28% 23.7M 5s 2024-09-12T12:57:41.425 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 47650K .......... .......... .......... .......... .......... 28% 48.5M 5s 2024-09-12T12:57:41.426 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 47700K .......... .......... .......... .......... .......... 28% 38.1M 5s 2024-09-12T12:57:41.429 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 47750K .......... .......... .......... .......... .......... 28% 21.9M 5s 2024-09-12T12:57:41.431 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 47800K .......... .......... .......... .......... .......... 28% 24.7M 5s 2024-09-12T12:57:41.431 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 47850K .......... .......... .......... .......... .......... 28% 55.1M 5s 2024-09-12T12:57:41.433 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 47900K .......... .......... .......... .......... .......... 28% 24.1M 5s 2024-09-12T12:57:41.435 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 47950K .......... .......... .......... .......... .......... 28% 24.3M 5s 2024-09-12T12:57:41.437 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 48000K .......... .......... .......... .......... .......... 28% 33.4M 5s 2024-09-12T12:57:41.439 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 48050K .......... .......... .......... .......... .......... 28% 25.0M 5s 2024-09-12T12:57:41.440 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 48100K .......... .......... .......... .......... .......... 28% 54.6M 5s 2024-09-12T12:57:41.442 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 48150K .......... .......... .......... .......... .......... 28% 23.8M 5s 2024-09-12T12:57:41.443 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 48200K .......... .......... .......... .......... .......... 28% 33.4M 5s 2024-09-12T12:57:41.446 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 48250K .......... .......... .......... .......... .......... 28% 18.0M 5s 2024-09-12T12:57:41.448 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 48300K .......... .......... .......... .......... .......... 28% 24.8M 5s 2024-09-12T12:57:41.450 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 48350K .......... .......... .......... .......... .......... 28% 24.5M 5s 2024-09-12T12:57:41.451 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 48400K .......... .......... .......... .......... .......... 28% 48.8M 5s 2024-09-12T12:57:41.453 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 48450K .......... .......... .......... .......... .......... 28% 23.8M 5s 2024-09-12T12:57:41.454 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 48500K .......... .......... .......... .......... .......... 28% 34.1M 5s 2024-09-12T12:57:41.456 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 48550K .......... .......... .......... .......... .......... 28% 24.5M 5s 2024-09-12T12:57:41.457 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 48600K .......... .......... .......... .......... .......... 28% 50.9M 5s 2024-09-12T12:57:41.459 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 48650K .......... .......... .......... .......... .......... 28% 24.1M 5s 2024-09-12T12:57:41.461 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 48700K .......... .......... .......... .......... .......... 28% 33.5M 5s 2024-09-12T12:57:41.463 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 48750K .......... .......... .......... .......... .......... 28% 22.7M 5s 2024-09-12T12:57:41.465 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 48800K .......... .......... .......... .......... .......... 28% 30.1M 5s 2024-09-12T12:57:41.467 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 48850K .......... .......... .......... .......... .......... 29% 20.0M 5s 2024-09-12T12:57:41.468 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 48900K .......... .......... .......... .......... .......... 29% 52.4M 5s 2024-09-12T12:57:41.470 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 48950K .......... .......... .......... .......... .......... 29% 24.6M 5s 2024-09-12T12:57:41.471 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 49000K .......... .......... .......... .......... .......... 29% 34.0M 5s 2024-09-12T12:57:41.473 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 49050K .......... .......... .......... .......... .......... 29% 23.8M 5s 2024-09-12T12:57:41.474 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 49100K .......... .......... .......... .......... .......... 29% 57.0M 5s 2024-09-12T12:57:41.476 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 49150K .......... .......... .......... .......... .......... 29% 24.0M 5s 2024-09-12T12:57:41.478 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 49200K .......... .......... .......... .......... .......... 29% 24.2M 5s 2024-09-12T12:57:41.480 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 49250K .......... .......... .......... .......... .......... 29% 31.9M 5s 2024-09-12T12:57:41.482 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 49300K .......... .......... .......... .......... .......... 29% 24.6M 5s 2024-09-12T12:57:41.485 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 49350K .......... .......... .......... .......... .......... 29% 23.0M 5s 2024-09-12T12:57:41.485 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 49400K .......... .......... .......... .......... .......... 29% 50.3M 5s 2024-09-12T12:57:41.487 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 49450K .......... .......... .......... .......... .......... 29% 23.6M 5s 2024-09-12T12:57:41.488 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 49500K .......... .......... .......... .......... .......... 29% 36.6M 5s 2024-09-12T12:57:41.490 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 49550K .......... .......... .......... .......... .......... 29% 24.7M 5s 2024-09-12T12:57:41.493 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 49600K .......... .......... .......... .......... .......... 29% 22.6M 5s 2024-09-12T12:57:41.494 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 49650K .......... .......... .......... .......... .......... 29% 36.8M 5s 2024-09-12T12:57:41.496 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 49700K .......... .......... .......... .......... .......... 29% 28.4M 5s 2024-09-12T12:57:41.497 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 49750K .......... .......... .......... .......... .......... 29% 38.6M 5s 2024-09-12T12:57:41.507 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 49800K .......... .......... .......... .......... .......... 29% 4.95M 5s 2024-09-12T12:57:41.508 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 49850K .......... .......... .......... .......... .......... 29% 34.8M 5s 2024-09-12T12:57:41.510 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 49900K .......... .......... .......... .......... .......... 29% 25.8M 5s 2024-09-12T12:57:41.512 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 49950K .......... .......... .......... .......... .......... 29% 23.4M 5s 2024-09-12T12:57:41.514 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 50000K .......... .......... .......... .......... .......... 29% 22.1M 5s 2024-09-12T12:57:41.515 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 50050K .......... .......... .......... .......... .......... 29% 44.7M 5s 2024-09-12T12:57:41.517 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 50100K .......... .......... .......... .......... .......... 29% 24.6M 5s 2024-09-12T12:57:41.518 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 50150K .......... .......... .......... .......... .......... 29% 45.7M 5s 2024-09-12T12:57:41.521 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 50200K .......... .......... .......... .......... .......... 29% 20.9M 5s 2024-09-12T12:57:41.522 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 50250K .......... .......... .......... .......... .......... 29% 37.5M 5s 2024-09-12T12:57:41.524 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 50300K .......... .......... .......... .......... .......... 29% 26.0M 5s 2024-09-12T12:57:41.526 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 50350K .......... .......... .......... .......... .......... 29% 22.7M 5s 2024-09-12T12:57:41.527 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 50400K .......... .......... .......... .......... .......... 29% 43.8M 5s 2024-09-12T12:57:41.529 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 50450K .......... .......... .......... .......... .......... 29% 23.9M 5s 2024-09-12T12:57:41.531 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 50500K .......... .......... .......... .......... .......... 29% 29.6M 5s 2024-09-12T12:57:41.533 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 50550K .......... .......... .......... .......... .......... 30% 27.7M 5s 2024-09-12T12:57:41.535 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 50600K .......... .......... .......... .......... .......... 30% 25.2M 5s 2024-09-12T12:57:41.535 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 50650K .......... .......... .......... .......... .......... 30% 49.2M 5s 2024-09-12T12:57:41.538 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 50700K .......... .......... .......... .......... .......... 30% 21.6M 5s 2024-09-12T12:57:41.540 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 50750K .......... .......... .......... .......... .......... 30% 24.1M 5s 2024-09-12T12:57:41.541 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 50800K .......... .......... .......... .......... .......... 30% 37.0M 5s 2024-09-12T12:57:41.543 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 50850K .......... .......... .......... .......... .......... 30% 28.4M 5s 2024-09-12T12:57:41.544 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 50900K .......... .......... .......... .......... .......... 30% 32.3M 5s 2024-09-12T12:57:41.546 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 50950K .......... .......... .......... .......... .......... 30% 23.6M 5s 2024-09-12T12:57:41.549 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 51000K .......... .......... .......... .......... .......... 30% 24.6M 5s 2024-09-12T12:57:41.549 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 51050K .......... .......... .......... .......... .......... 30% 46.3M 5s 2024-09-12T12:57:41.552 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 51100K .......... .......... .......... .......... .......... 30% 22.3M 5s 2024-09-12T12:57:41.554 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 51150K .......... .......... .......... .......... .......... 30% 26.4M 5s 2024-09-12T12:57:41.555 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 51200K .......... .......... .......... .......... .......... 30% 45.8M 5s 2024-09-12T12:57:41.557 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 51250K .......... .......... .......... .......... .......... 30% 26.7M 5s 2024-09-12T12:57:41.558 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 51300K .......... .......... .......... .......... .......... 30% 30.0M 5s 2024-09-12T12:57:41.561 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 51350K .......... .......... .......... .......... .......... 30% 23.2M 5s 2024-09-12T12:57:41.561 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 51400K .......... .......... .......... .......... .......... 30% 44.6M 5s 2024-09-12T12:57:41.563 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 51450K .......... .......... .......... .......... .......... 30% 25.9M 5s 2024-09-12T12:57:41.566 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 51500K .......... .......... .......... .......... .......... 30% 22.5M 5s 2024-09-12T12:57:41.569 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 51550K .......... .......... .......... .......... .......... 30% 24.7M 5s 2024-09-12T12:57:41.569 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 51600K .......... .......... .......... .......... .......... 30% 39.7M 5s 2024-09-12T12:57:41.571 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 51650K .......... .......... .......... .......... .......... 30% 23.4M 5s 2024-09-12T12:57:41.572 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 51700K .......... .......... .......... .......... .......... 30% 38.7M 5s 2024-09-12T12:57:41.573 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 51750K .......... .......... .......... .......... .......... 30% 32.7M 5s 2024-09-12T12:57:41.575 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 51800K .......... .......... .......... .......... .......... 30% 33.4M 5s 2024-09-12T12:57:41.577 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 51850K .......... .......... .......... .......... .......... 30% 23.1M 5s 2024-09-12T12:57:41.579 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 51900K .......... .......... .......... .......... .......... 30% 38.3M 5s 2024-09-12T12:57:41.580 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 51950K .......... .......... .......... .......... .......... 30% 26.0M 5s 2024-09-12T12:57:41.583 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 52000K .......... .......... .......... .......... .......... 30% 21.2M 5s 2024-09-12T12:57:41.584 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 52050K .......... .......... .......... .......... .......... 30% 36.9M 5s 2024-09-12T12:57:41.585 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 52100K .......... .......... .......... .......... .......... 30% 28.9M 5s 2024-09-12T12:57:41.588 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 52150K .......... .......... .......... .......... .......... 30% 21.3M 5s 2024-09-12T12:57:41.589 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 52200K .......... .......... .......... .......... .......... 30% 26.1M 5s 2024-09-12T12:57:41.591 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 52250K .......... .......... .......... .......... .......... 31% 25.7M 5s 2024-09-12T12:57:41.593 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 52300K .......... .......... .......... .......... .......... 31% 40.6M 5s 2024-09-12T12:57:41.595 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 52350K .......... .......... .......... .......... .......... 31% 23.1M 5s 2024-09-12T12:57:41.598 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 52400K .......... .......... .......... .......... .......... 31% 19.2M 5s 2024-09-12T12:57:41.598 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 52450K .......... .......... .......... .......... .......... 31% 111M 5s 2024-09-12T12:57:41.600 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 52500K .......... .......... .......... .......... .......... 31% 24.0M 5s 2024-09-12T12:57:41.603 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 52550K .......... .......... .......... .......... .......... 31% 32.9M 5s 2024-09-12T12:57:41.605 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 52600K .......... .......... .......... .......... .......... 31% 11.7M 5s 2024-09-12T12:57:41.611 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 52650K .......... .......... .......... .......... .......... 31% 9.62M 5s 2024-09-12T12:57:41.615 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 52700K .......... .......... .......... .......... .......... 31% 9.80M 5s 2024-09-12T12:57:41.623 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 52750K .......... .......... .......... .......... .......... 31% 6.66M 5s 2024-09-12T12:57:41.628 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 52800K .......... .......... .......... .......... .......... 31% 9.99M 5s 2024-09-12T12:57:41.629 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 52850K .......... .......... .......... .......... .......... 31% 37.7M 5s 2024-09-12T12:57:41.630 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 52900K .......... .......... .......... .......... .......... 31% 42.9M 5s 2024-09-12T12:57:41.632 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 52950K .......... .......... .......... .......... .......... 31% 27.7M 5s 2024-09-12T12:57:41.634 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 53000K .......... .......... .......... .......... .......... 31% 22.9M 5s 2024-09-12T12:57:41.635 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 53050K .......... .......... .......... .......... .......... 31% 38.5M 5s 2024-09-12T12:57:41.638 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 53100K .......... .......... .......... .......... .......... 31% 22.9M 5s 2024-09-12T12:57:41.639 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 53150K .......... .......... .......... .......... .......... 31% 26.6M 5s 2024-09-12T12:57:41.641 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 53200K .......... .......... .......... .......... .......... 31% 44.4M 5s 2024-09-12T12:57:41.642 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 53250K .......... .......... .......... .......... .......... 31% 22.9M 5s 2024-09-12T12:57:41.644 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 53300K .......... .......... .......... .......... .......... 31% 33.8M 5s 2024-09-12T12:57:41.646 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 53350K .......... .......... .......... .......... .......... 31% 23.4M 5s 2024-09-12T12:57:41.648 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 53400K .......... .......... .......... .......... .......... 31% 25.3M 5s 2024-09-12T12:57:41.649 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 53450K .......... .......... .......... .......... .......... 31% 43.9M 5s 2024-09-12T12:57:41.651 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 53500K .......... .......... .......... .......... .......... 31% 25.3M 5s 2024-09-12T12:57:41.653 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 53550K .......... .......... .......... .......... .......... 31% 21.7M 5s 2024-09-12T12:57:41.654 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 53600K .......... .......... .......... .......... .......... 31% 39.8M 5s 2024-09-12T12:57:41.657 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 53650K .......... .......... .......... .......... .......... 31% 21.6M 4s 2024-09-12T12:57:41.658 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 53700K .......... .......... .......... .......... .......... 31% 40.1M 4s 2024-09-12T12:57:41.660 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 53750K .......... .......... .......... .......... .......... 31% 25.6M 4s 2024-09-12T12:57:41.662 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 53800K .......... .......... .......... .......... .......... 31% 24.8M 4s 2024-09-12T12:57:41.663 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 53850K .......... .......... .......... .......... .......... 31% 41.9M 4s 2024-09-12T12:57:41.665 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 53900K .......... .......... .......... .......... .......... 32% 23.5M 4s 2024-09-12T12:57:41.667 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 53950K .......... .......... .......... .......... .......... 32% 25.6M 4s 2024-09-12T12:57:41.668 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 54000K .......... .......... .......... .......... .......... 32% 36.2M 4s 2024-09-12T12:57:41.670 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 54050K .......... .......... .......... .......... .......... 32% 31.9M 4s 2024-09-12T12:57:41.672 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 54100K .......... .......... .......... .......... .......... 32% 28.7M 4s 2024-09-12T12:57:41.673 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 54150K .......... .......... .......... .......... .......... 32% 25.2M 4s 2024-09-12T12:57:41.675 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 54200K .......... .......... .......... .......... .......... 32% 36.0M 4s 2024-09-12T12:57:41.677 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 54250K .......... .......... .......... .......... .......... 32% 28.4M 4s 2024-09-12T12:57:41.679 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 54300K .......... .......... .......... .......... .......... 32% 22.1M 4s 2024-09-12T12:57:41.681 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 54350K .......... .......... .......... .......... .......... 32% 23.3M 4s 2024-09-12T12:57:41.682 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 54400K .......... .......... .......... .......... .......... 32% 38.7M 4s 2024-09-12T12:57:41.685 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 54450K .......... .......... .......... .......... .......... 32% 23.8M 4s 2024-09-12T12:57:41.685 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 54500K .......... .......... .......... .......... .......... 32% 36.8M 4s 2024-09-12T12:57:41.687 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 54550K .......... .......... .......... .......... .......... 32% 38.8M 4s 2024-09-12T12:57:41.689 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 54600K .......... .......... .......... .......... .......... 32% 28.0M 4s 2024-09-12T12:57:41.691 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 54650K .......... .......... .......... .......... .......... 32% 22.2M 4s 2024-09-12T12:57:41.692 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 54700K .......... .......... .......... .......... .......... 32% 38.1M 4s 2024-09-12T12:57:41.694 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 54750K .......... .......... .......... .......... .......... 32% 24.3M 4s 2024-09-12T12:57:41.696 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 54800K .......... .......... .......... .......... .......... 32% 27.4M 4s 2024-09-12T12:57:41.698 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 54850K .......... .......... .......... .......... .......... 32% 23.2M 4s 2024-09-12T12:57:41.699 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 54900K .......... .......... .......... .......... .......... 32% 39.1M 4s 2024-09-12T12:57:41.700 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 54950K .......... .......... .......... .......... .......... 32% 41.9M 4s 2024-09-12T12:57:41.703 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 55000K .......... .......... .......... .......... .......... 32% 22.2M 4s 2024-09-12T12:57:41.705 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 55050K .......... .......... .......... .......... .......... 32% 21.6M 4s 2024-09-12T12:57:41.706 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 55100K .......... .......... .......... .......... .......... 32% 32.4M 4s 2024-09-12T12:57:41.713 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 55150K .......... .......... .......... .......... .......... 32% 7.78M 4s 2024-09-12T12:57:41.717 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 55200K .......... .......... .......... .......... .......... 32% 11.5M 4s 2024-09-12T12:57:41.719 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 55250K .......... .......... .......... .......... .......... 32% 21.3M 4s 2024-09-12T12:57:41.726 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 55300K .......... .......... .......... .......... .......... 32% 7.64M 4s 2024-09-12T12:57:41.727 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 55350K .......... .......... .......... .......... .......... 32% 24.3M 4s 2024-09-12T12:57:41.731 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 55400K .......... .......... .......... .......... .......... 32% 13.0M 4s 2024-09-12T12:57:41.734 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 55450K .......... .......... .......... .......... .......... 32% 19.7M 4s 2024-09-12T12:57:41.738 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 55500K .......... .......... .......... .......... .......... 32% 11.2M 4s 2024-09-12T12:57:41.743 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 55550K .......... .......... .......... .......... .......... 32% 10.0M 4s 2024-09-12T12:57:41.746 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 55600K .......... .......... .......... .......... .......... 33% 16.1M 4s 2024-09-12T12:57:41.753 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 55650K .......... .......... .......... .......... .......... 33% 7.09M 4s 2024-09-12T12:57:41.757 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 55700K .......... .......... .......... .......... .......... 33% 12.0M 4s 2024-09-12T12:57:41.759 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 55750K .......... .......... .......... .......... .......... 33% 24.8M 4s 2024-09-12T12:57:41.763 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 55800K .......... .......... .......... .......... .......... 33% 12.5M 4s 2024-09-12T12:57:41.765 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 55850K .......... .......... .......... .......... .......... 33% 26.2M 4s 2024-09-12T12:57:41.767 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 55900K .......... .......... .......... .......... .......... 33% 18.9M 4s 2024-09-12T12:57:41.769 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 55950K .......... .......... .......... .......... .......... 33% 25.9M 4s 2024-09-12T12:57:41.771 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 56000K .......... .......... .......... .......... .......... 33% 34.8M 4s 2024-09-12T12:57:41.773 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 56050K .......... .......... .......... .......... .......... 33% 26.7M 4s 2024-09-12T12:57:41.774 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 56100K .......... .......... .......... .......... .......... 33% 40.9M 4s 2024-09-12T12:57:41.776 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 56150K .......... .......... .......... .......... .......... 33% 22.0M 4s 2024-09-12T12:57:41.777 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 56200K .......... .......... .......... .......... .......... 33% 32.1M 4s 2024-09-12T12:57:41.779 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 56250K .......... .......... .......... .......... .......... 33% 34.9M 4s 2024-09-12T12:57:41.781 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 56300K .......... .......... .......... .......... .......... 33% 20.4M 4s 2024-09-12T12:57:41.783 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 56350K .......... .......... .......... .......... .......... 33% 25.1M 4s 2024-09-12T12:57:41.784 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 56400K .......... .......... .......... .......... .......... 33% 47.1M 4s 2024-09-12T12:57:41.786 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 56450K .......... .......... .......... .......... .......... 33% 24.6M 4s 2024-09-12T12:57:41.787 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 56500K .......... .......... .......... .......... .......... 33% 37.1M 4s 2024-09-12T12:57:41.789 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 56550K .......... .......... .......... .......... .......... 33% 29.0M 4s 2024-09-12T12:57:41.792 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 56600K .......... .......... .......... .......... .......... 33% 23.1M 4s 2024-09-12T12:57:41.792 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 56650K .......... .......... .......... .......... .......... 33% 43.3M 4s 2024-09-12T12:57:41.794 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 56700K .......... .......... .......... .......... .......... 33% 21.9M 4s 2024-09-12T12:57:41.797 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 56750K .......... .......... .......... .......... .......... 33% 21.4M 4s 2024-09-12T12:57:41.798 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 56800K .......... .......... .......... .......... .......... 33% 42.0M 4s 2024-09-12T12:57:41.800 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 56850K .......... .......... .......... .......... .......... 33% 26.2M 4s 2024-09-12T12:57:41.802 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 56900K .......... .......... .......... .......... .......... 33% 25.5M 4s 2024-09-12T12:57:41.813 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 56950K .......... .......... .......... .......... .......... 33% 7.65M 4s 2024-09-12T12:57:41.814 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 57000K .......... .......... .......... .......... .......... 33% 64.4M 4s 2024-09-12T12:57:41.814 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 57050K .......... .......... .......... .......... .......... 33% 129M 4s 2024-09-12T12:57:41.814 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 57100K .......... .......... .......... .......... .......... 33% 146M 4s 2024-09-12T12:57:41.814 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 57150K .......... .......... .......... .......... .......... 33% 34.0M 4s 2024-09-12T12:57:41.814 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 57200K .......... .......... .......... .......... .......... 33% 16.1M 4s 2024-09-12T12:57:41.817 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 57250K .......... .......... .......... .......... .......... 33% 15.8M 4s 2024-09-12T12:57:41.820 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 57300K .......... .......... .......... .......... .......... 34% 18.4M 4s 2024-09-12T12:57:41.822 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 57350K .......... .......... .......... .......... .......... 34% 26.6M 4s 2024-09-12T12:57:41.825 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 57400K .......... .......... .......... .......... .......... 34% 15.4M 4s 2024-09-12T12:57:41.827 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 57450K .......... .......... .......... .......... .......... 34% 20.2M 4s 2024-09-12T12:57:41.830 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 57500K .......... .......... .......... .......... .......... 34% 17.2M 4s 2024-09-12T12:57:41.833 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 57550K .......... .......... .......... .......... .......... 34% 16.2M 4s 2024-09-12T12:57:41.836 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 57600K .......... .......... .......... .......... .......... 34% 16.0M 4s 2024-09-12T12:57:41.841 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 57650K .......... .......... .......... .......... .......... 34% 11.9M 4s 2024-09-12T12:57:41.842 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 57700K .......... .......... .......... .......... .......... 34% 26.8M 4s 2024-09-12T12:57:41.844 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 57750K .......... .......... .......... .......... .......... 34% 25.8M 4s 2024-09-12T12:57:41.847 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 57800K .......... .......... .......... .......... .......... 34% 15.6M 4s 2024-09-12T12:57:41.850 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 57850K .......... .......... .......... .......... .......... 34% 16.1M 4s 2024-09-12T12:57:41.852 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 57900K .......... .......... .......... .......... .......... 34% 24.8M 4s 2024-09-12T12:57:41.856 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 57950K .......... .......... .......... .......... .......... 34% 14.1M 4s 2024-09-12T12:57:41.863 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 58000K .......... .......... .......... .......... .......... 34% 6.99M 4s 2024-09-12T12:57:41.864 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 58050K .......... .......... .......... .......... .......... 34% 45.6M 4s 2024-09-12T12:57:41.866 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 58100K .......... .......... .......... .......... .......... 34% 24.2M 4s 2024-09-12T12:57:41.868 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 58150K .......... .......... .......... .......... .......... 34% 31.1M 4s 2024-09-12T12:57:41.870 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 58200K .......... .......... .......... .......... .......... 34% 18.3M 4s 2024-09-12T12:57:41.872 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 58250K .......... .......... .......... .......... .......... 34% 22.2M 4s 2024-09-12T12:57:41.876 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 58300K .......... .......... .......... .......... .......... 34% 12.5M 4s 2024-09-12T12:57:41.881 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 58350K .......... .......... .......... .......... .......... 34% 9.26M 4s 2024-09-12T12:57:41.886 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 58400K .......... .......... .......... .......... .......... 34% 12.4M 4s 2024-09-12T12:57:41.887 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 58450K .......... .......... .......... .......... .......... 34% 24.7M 4s 2024-09-12T12:57:41.891 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 58500K .......... .......... .......... .......... .......... 34% 15.5M 4s 2024-09-12T12:57:41.892 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 58550K .......... .......... .......... .......... .......... 34% 26.8M 4s 2024-09-12T12:57:41.895 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 58600K .......... .......... .......... .......... .......... 34% 19.0M 4s 2024-09-12T12:57:41.896 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 58650K .......... .......... .......... .......... .......... 34% 28.0M 4s 2024-09-12T12:57:41.899 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 58700K .......... .......... .......... .......... .......... 34% 28.4M 4s 2024-09-12T12:57:41.900 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 58750K .......... .......... .......... .......... .......... 34% 25.2M 4s 2024-09-12T12:57:41.901 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 58800K .......... .......... .......... .......... .......... 34% 41.0M 4s 2024-09-12T12:57:41.904 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 58850K .......... .......... .......... .......... .......... 34% 27.7M 4s 2024-09-12T12:57:41.905 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 58900K .......... .......... .......... .......... .......... 34% 29.5M 4s 2024-09-12T12:57:41.906 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 58950K .......... .......... .......... .......... .......... 35% 31.4M 4s 2024-09-12T12:57:41.909 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 59000K .......... .......... .......... .......... .......... 35% 22.8M 4s 2024-09-12T12:57:41.910 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 59050K .......... .......... .......... .......... .......... 35% 32.2M 4s 2024-09-12T12:57:41.911 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 59100K .......... .......... .......... .......... .......... 35% 35.5M 4s 2024-09-12T12:57:41.914 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 59150K .......... .......... .......... .......... .......... 35% 23.9M 4s 2024-09-12T12:57:41.916 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 59200K .......... .......... .......... .......... .......... 35% 24.1M 4s 2024-09-12T12:57:41.917 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 59250K .......... .......... .......... .......... .......... 35% 33.7M 4s 2024-09-12T12:57:41.918 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 59300K .......... .......... .......... .......... .......... 35% 38.1M 4s 2024-09-12T12:57:41.920 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 59350K .......... .......... .......... .......... .......... 35% 26.1M 4s 2024-09-12T12:57:41.923 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 59400K .......... .......... .......... .......... .......... 35% 13.9M 4s 2024-09-12T12:57:41.926 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 59450K .......... .......... .......... .......... .......... 35% 16.6M 4s 2024-09-12T12:57:41.929 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 59500K .......... .......... .......... .......... .......... 35% 19.6M 4s 2024-09-12T12:57:41.934 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 59550K .......... .......... .......... .......... .......... 35% 10.2M 4s 2024-09-12T12:57:41.935 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 59600K .......... .......... .......... .......... .......... 35% 32.5M 4s 2024-09-12T12:57:41.938 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 59650K .......... .......... .......... .......... .......... 35% 19.4M 4s 2024-09-12T12:57:41.941 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 59700K .......... .......... .......... .......... .......... 35% 21.2M 4s 2024-09-12T12:57:41.943 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 59750K .......... .......... .......... .......... .......... 35% 18.0M 4s 2024-09-12T12:57:41.944 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 59800K .......... .......... .......... .......... .......... 35% 31.8M 4s 2024-09-12T12:57:41.947 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 59850K .......... .......... .......... .......... .......... 35% 16.2M 4s 2024-09-12T12:57:41.950 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 59900K .......... .......... .......... .......... .......... 35% 20.5M 4s 2024-09-12T12:57:41.953 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 59950K .......... .......... .......... .......... .......... 35% 13.6M 4s 2024-09-12T12:57:41.955 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 60000K .......... .......... .......... .......... .......... 35% 35.6M 4s 2024-09-12T12:57:41.958 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 60050K .......... .......... .......... .......... .......... 35% 19.0M 4s 2024-09-12T12:57:41.960 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 60100K .......... .......... .......... .......... .......... 35% 17.9M 4s 2024-09-12T12:57:41.962 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 60150K .......... .......... .......... .......... .......... 35% 20.0M 4s 2024-09-12T12:57:41.964 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 60200K .......... .......... .......... .......... .......... 35% 31.1M 4s 2024-09-12T12:57:41.967 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 60250K .......... .......... .......... .......... .......... 35% 18.0M 4s 2024-09-12T12:57:41.969 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 60300K .......... .......... .......... .......... .......... 35% 22.9M 4s 2024-09-12T12:57:41.971 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 60350K .......... .......... .......... .......... .......... 35% 18.7M 4s 2024-09-12T12:57:41.974 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 60400K .......... .......... .......... .......... .......... 35% 16.0M 4s 2024-09-12T12:57:41.977 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 60450K .......... .......... .......... .......... .......... 35% 21.5M 4s 2024-09-12T12:57:41.982 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 60500K .......... .......... .......... .......... .......... 35% 9.73M 4s 2024-09-12T12:57:41.985 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 60550K .......... .......... .......... .......... .......... 35% 14.0M 4s 2024-09-12T12:57:41.988 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 60600K .......... .......... .......... .......... .......... 35% 24.3M 4s 2024-09-12T12:57:41.989 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 60650K .......... .......... .......... .......... .......... 36% 24.4M 4s 2024-09-12T12:57:41.992 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 60700K .......... .......... .......... .......... .......... 36% 20.1M 4s 2024-09-12T12:57:41.995 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 60750K .......... .......... .......... .......... .......... 36% 17.6M 4s 2024-09-12T12:57:41.999 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 60800K .......... .......... .......... .......... .......... 36% 11.5M 4s 2024-09-12T12:57:42.001 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 60850K .......... .......... .......... .......... .......... 36% 27.5M 4s 2024-09-12T12:57:42.004 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 60900K .......... .......... .......... .......... .......... 36% 14.6M 4s 2024-09-12T12:57:42.006 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 60950K .......... .......... .......... .......... .......... 36% 20.9M 4s 2024-09-12T12:57:42.010 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 61000K .......... .......... .......... .......... .......... 36% 12.1M 4s 2024-09-12T12:57:42.012 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 61050K .......... .......... .......... .......... .......... 36% 28.7M 4s 2024-09-12T12:57:42.014 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 61100K .......... .......... .......... .......... .......... 36% 25.7M 4s 2024-09-12T12:57:42.019 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 61150K .......... .......... .......... .......... .......... 36% 11.4M 4s 2024-09-12T12:57:42.021 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 61200K .......... .......... .......... .......... .......... 36% 17.6M 4s 2024-09-12T12:57:42.023 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 61250K .......... .......... .......... .......... .......... 36% 37.9M 4s 2024-09-12T12:57:42.025 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 61300K .......... .......... .......... .......... .......... 36% 24.7M 4s 2024-09-12T12:57:42.025 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 61350K .......... .......... .......... .......... .......... 36% 40.6M 4s 2024-09-12T12:57:42.027 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 61400K .......... .......... .......... .......... .......... 36% 30.5M 4s 2024-09-12T12:57:42.029 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 61450K .......... .......... .......... .......... .......... 36% 32.1M 4s 2024-09-12T12:57:42.030 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 61500K .......... .......... .......... .......... .......... 36% 25.9M 4s 2024-09-12T12:57:42.033 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 61550K .......... .......... .......... .......... .......... 36% 19.6M 4s 2024-09-12T12:57:42.035 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 61600K .......... .......... .......... .......... .......... 36% 36.1M 4s 2024-09-12T12:57:42.036 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 61650K .......... .......... .......... .......... .......... 36% 29.8M 4s 2024-09-12T12:57:42.038 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 61700K .......... .......... .......... .......... .......... 36% 24.7M 4s 2024-09-12T12:57:42.041 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 61750K .......... .......... .......... .......... .......... 36% 19.0M 4s 2024-09-12T12:57:42.043 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 61800K .......... .......... .......... .......... .......... 36% 22.6M 4s 2024-09-12T12:57:42.043 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 61850K .......... .......... .......... .......... .......... 36% 39.4M 4s 2024-09-12T12:57:42.045 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 61900K .......... .......... .......... .......... .......... 36% 30.8M 4s 2024-09-12T12:57:42.047 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 61950K .......... .......... .......... .......... .......... 36% 22.5M 4s 2024-09-12T12:57:42.049 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 62000K .......... .......... .......... .......... .......... 36% 29.7M 4s 2024-09-12T12:57:42.051 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 62050K .......... .......... .......... .......... .......... 36% 28.4M 4s 2024-09-12T12:57:42.052 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 62100K .......... .......... .......... .......... .......... 36% 34.6M 4s 2024-09-12T12:57:42.055 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 62150K .......... .......... .......... .......... .......... 36% 24.7M 4s 2024-09-12T12:57:42.056 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 62200K .......... .......... .......... .......... .......... 36% 30.4M 4s 2024-09-12T12:57:42.057 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 62250K .......... .......... .......... .......... .......... 36% 35.5M 4s 2024-09-12T12:57:42.059 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 62300K .......... .......... .......... .......... .......... 36% 35.9M 4s 2024-09-12T12:57:42.062 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 62350K .......... .......... .......... .......... .......... 37% 17.0M 4s 2024-09-12T12:57:42.074 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 62400K .......... .......... .......... .......... .......... 37% 3.97M 4s 2024-09-12T12:57:42.075 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 62450K .......... .......... .......... .......... .......... 37% 31.4M 4s 2024-09-12T12:57:42.078 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 62500K .......... .......... .......... .......... .......... 37% 26.9M 4s 2024-09-12T12:57:42.080 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 62550K .......... .......... .......... .......... .......... 37% 18.8M 4s 2024-09-12T12:57:42.081 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 62600K .......... .......... .......... .......... .......... 37% 38.9M 4s 2024-09-12T12:57:42.083 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 62650K .......... .......... .......... .......... .......... 37% 29.1M 4s 2024-09-12T12:57:42.085 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 62700K .......... .......... .......... .......... .......... 37% 23.0M 4s 2024-09-12T12:57:42.087 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 62750K .......... .......... .......... .......... .......... 37% 23.5M 4s 2024-09-12T12:57:42.088 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 62800K .......... .......... .......... .......... .......... 37% 33.1M 4s 2024-09-12T12:57:42.094 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 62850K .......... .......... .......... .......... .......... 37% 8.01M 4s 2024-09-12T12:57:42.097 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 62900K .......... .......... .......... .......... .......... 37% 20.0M 4s 2024-09-12T12:57:42.100 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 62950K .......... .......... .......... .......... .......... 37% 15.1M 4s 2024-09-12T12:57:42.102 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 63000K .......... .......... .......... .......... .......... 37% 25.1M 4s 2024-09-12T12:57:42.106 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 63050K .......... .......... .......... .......... .......... 37% 13.8M 4s 2024-09-12T12:57:42.107 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 63100K .......... .......... .......... .......... .......... 37% 35.9M 4s 2024-09-12T12:57:42.111 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 63150K .......... .......... .......... .......... .......... 37% 12.0M 4s 2024-09-12T12:57:42.113 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 63200K .......... .......... .......... .......... .......... 37% 23.2M 4s 2024-09-12T12:57:42.115 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 63250K .......... .......... .......... .......... .......... 37% 33.5M 4s 2024-09-12T12:57:42.117 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 63300K .......... .......... .......... .......... .......... 37% 23.5M 4s 2024-09-12T12:57:42.119 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 63350K .......... .......... .......... .......... .......... 37% 21.1M 4s 2024-09-12T12:57:42.122 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 63400K .......... .......... .......... .......... .......... 37% 17.8M 4s 2024-09-12T12:57:42.124 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 63450K .......... .......... .......... .......... .......... 37% 35.3M 4s 2024-09-12T12:57:42.131 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 63500K .......... .......... .......... .......... .......... 37% 6.08M 4s 2024-09-12T12:57:42.135 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 63550K .......... .......... .......... .......... .......... 37% 13.9M 4s 2024-09-12T12:57:42.137 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 63600K .......... .......... .......... .......... .......... 37% 21.5M 4s 2024-09-12T12:57:42.139 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 63650K .......... .......... .......... .......... .......... 37% 25.5M 4s 2024-09-12T12:57:42.142 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 63700K .......... .......... .......... .......... .......... 37% 18.9M 4s 2024-09-12T12:57:42.144 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 63750K .......... .......... .......... .......... .......... 37% 25.1M 4s 2024-09-12T12:57:42.147 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 63800K .......... .......... .......... .......... .......... 37% 16.4M 4s 2024-09-12T12:57:42.148 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 63850K .......... .......... .......... .......... .......... 37% 31.8M 4s 2024-09-12T12:57:42.150 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 63900K .......... .......... .......... .......... .......... 37% 25.1M 4s 2024-09-12T12:57:42.153 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 63950K .......... .......... .......... .......... .......... 37% 21.8M 4s 2024-09-12T12:57:42.153 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 64000K .......... .......... .......... .......... .......... 37% 38.2M 4s 2024-09-12T12:57:42.156 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 64050K .......... .......... .......... .......... .......... 38% 22.4M 4s 2024-09-12T12:57:42.157 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 64100K .......... .......... .......... .......... .......... 38% 29.2M 4s 2024-09-12T12:57:42.159 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 64150K .......... .......... .......... .......... .......... 38% 28.2M 4s 2024-09-12T12:57:42.161 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 64200K .......... .......... .......... .......... .......... 38% 17.9M 4s 2024-09-12T12:57:42.164 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 64250K .......... .......... .......... .......... .......... 38% 27.7M 4s 2024-09-12T12:57:42.165 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 64300K .......... .......... .......... .......... .......... 38% 28.1M 4s 2024-09-12T12:57:42.168 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 64350K .......... .......... .......... .......... .......... 38% 22.0M 4s 2024-09-12T12:57:42.169 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 64400K .......... .......... .......... .......... .......... 38% 45.1M 4s 2024-09-12T12:57:42.171 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 64450K .......... .......... .......... .......... .......... 38% 22.8M 4s 2024-09-12T12:57:42.172 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 64500K .......... .......... .......... .......... .......... 38% 33.7M 4s 2024-09-12T12:57:42.174 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 64550K .......... .......... .......... .......... .......... 38% 20.4M 4s 2024-09-12T12:57:42.179 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 64600K .......... .......... .......... .......... .......... 38% 10.7M 4s 2024-09-12T12:57:42.182 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 64650K .......... .......... .......... .......... .......... 38% 17.4M 4s 2024-09-12T12:57:42.186 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 64700K .......... .......... .......... .......... .......... 38% 11.9M 4s 2024-09-12T12:57:42.191 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 64750K .......... .......... .......... .......... .......... 38% 8.89M 4s 2024-09-12T12:57:42.194 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 64800K .......... .......... .......... .......... .......... 38% 18.2M 4s 2024-09-12T12:57:42.195 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 64850K .......... .......... .......... .......... .......... 38% 34.7M 4s 2024-09-12T12:57:42.197 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 64900K .......... .......... .......... .......... .......... 38% 36.6M 4s 2024-09-12T12:57:42.199 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 64950K .......... .......... .......... .......... .......... 38% 23.5M 4s 2024-09-12T12:57:42.201 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 65000K .......... .......... .......... .......... .......... 38% 28.3M 4s 2024-09-12T12:57:42.202 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 65050K .......... .......... .......... .......... .......... 38% 41.8M 4s 2024-09-12T12:57:42.204 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 65100K .......... .......... .......... .......... .......... 38% 27.6M 4s 2024-09-12T12:57:42.205 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 65150K .......... .......... .......... .......... .......... 38% 26.8M 4s 2024-09-12T12:57:42.208 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 65200K .......... .......... .......... .......... .......... 38% 17.0M 4s 2024-09-12T12:57:42.213 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 65250K .......... .......... .......... .......... .......... 38% 10.9M 4s 2024-09-12T12:57:42.215 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 65300K .......... .......... .......... .......... .......... 38% 22.2M 4s 2024-09-12T12:57:42.218 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 65350K .......... .......... .......... .......... .......... 38% 16.9M 4s 2024-09-12T12:57:42.220 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 65400K .......... .......... .......... .......... .......... 38% 22.9M 4s 2024-09-12T12:57:42.222 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 65450K .......... .......... .......... .......... .......... 38% 22.3M 4s 2024-09-12T12:57:42.225 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 65500K .......... .......... .......... .......... .......... 38% 20.2M 4s 2024-09-12T12:57:42.228 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 65550K .......... .......... .......... .......... .......... 38% 14.7M 4s 2024-09-12T12:57:42.229 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 65600K .......... .......... .......... .......... .......... 38% 35.2M 4s 2024-09-12T12:57:42.232 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 65650K .......... .......... .......... .......... .......... 38% 20.9M 4s 2024-09-12T12:57:42.235 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 65700K .......... .......... .......... .......... .......... 39% 13.8M 4s 2024-09-12T12:57:42.237 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 65750K .......... .......... .......... .......... .......... 39% 24.2M 4s 2024-09-12T12:57:42.241 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 65800K .......... .......... .......... .......... .......... 39% 15.2M 4s 2024-09-12T12:57:42.245 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 65850K .......... .......... .......... .......... .......... 39% 10.4M 4s 2024-09-12T12:57:42.248 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 65900K .......... .......... .......... .......... .......... 39% 16.8M 4s 2024-09-12T12:57:42.252 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 65950K .......... .......... .......... .......... .......... 39% 12.0M 4s 2024-09-12T12:57:42.255 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 66000K .......... .......... .......... .......... .......... 39% 17.0M 4s 2024-09-12T12:57:42.258 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 66050K .......... .......... .......... .......... .......... 39% 18.5M 4s 2024-09-12T12:57:42.260 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 66100K .......... .......... .......... .......... .......... 39% 22.1M 4s 2024-09-12T12:57:42.261 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 66150K .......... .......... .......... .......... .......... 39% 29.9M 4s 2024-09-12T12:57:42.264 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 66200K .......... .......... .......... .......... .......... 39% 23.0M 4s 2024-09-12T12:57:42.267 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 66250K .......... .......... .......... .......... .......... 39% 15.2M 4s 2024-09-12T12:57:42.269 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 66300K .......... .......... .......... .......... .......... 39% 20.6M 4s 2024-09-12T12:57:42.273 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 66350K .......... .......... .......... .......... .......... 39% 13.4M 4s 2024-09-12T12:57:42.275 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 66400K .......... .......... .......... .......... .......... 39% 23.4M 4s 2024-09-12T12:57:42.276 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 66450K .......... .......... .......... .......... .......... 39% 37.4M 4s 2024-09-12T12:57:42.278 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 66500K .......... .......... .......... .......... .......... 39% 28.0M 4s 2024-09-12T12:57:42.280 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 66550K .......... .......... .......... .......... .......... 39% 29.8M 4s 2024-09-12T12:57:42.281 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 66600K .......... .......... .......... .......... .......... 39% 23.6M 4s 2024-09-12T12:57:42.283 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 66650K .......... .......... .......... .......... .......... 39% 26.2M 4s 2024-09-12T12:57:42.286 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 66700K .......... .......... .......... .......... .......... 39% 17.9M 4s 2024-09-12T12:57:42.289 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 66750K .......... .......... .......... .......... .......... 39% 23.6M 4s 2024-09-12T12:57:42.291 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 66800K .......... .......... .......... .......... .......... 39% 30.7M 4s 2024-09-12T12:57:42.291 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 66850K .......... .......... .......... .......... .......... 39% 34.0M 4s 2024-09-12T12:57:42.295 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 66900K .......... .......... .......... .......... .......... 39% 13.7M 4s 2024-09-12T12:57:42.299 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 66950K .......... .......... .......... .......... .......... 39% 12.1M 4s 2024-09-12T12:57:42.303 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 67000K .......... .......... .......... .......... .......... 39% 12.4M 4s 2024-09-12T12:57:42.305 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 67050K .......... .......... .......... .......... .......... 39% 17.8M 4s 2024-09-12T12:57:42.310 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 67100K .......... .......... .......... .......... .......... 39% 11.4M 4s 2024-09-12T12:57:42.313 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 67150K .......... .......... .......... .......... .......... 39% 15.8M 4s 2024-09-12T12:57:42.315 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 67200K .......... .......... .......... .......... .......... 39% 21.4M 4s 2024-09-12T12:57:42.317 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 67250K .......... .......... .......... .......... .......... 39% 33.0M 4s 2024-09-12T12:57:42.319 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 67300K .......... .......... .......... .......... .......... 39% 24.8M 4s 2024-09-12T12:57:42.320 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 67350K .......... .......... .......... .......... .......... 39% 29.1M 4s 2024-09-12T12:57:42.322 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 67400K .......... .......... .......... .......... .......... 40% 29.6M 4s 2024-09-12T12:57:42.323 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 67450K .......... .......... .......... .......... .......... 40% 40.1M 4s 2024-09-12T12:57:42.327 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 67500K .......... .......... .......... .......... .......... 40% 15.6M 4s 2024-09-12T12:57:42.329 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 67550K .......... .......... .......... .......... .......... 40% 16.4M 4s 2024-09-12T12:57:42.332 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 67600K .......... .......... .......... .......... .......... 40% 19.4M 4s 2024-09-12T12:57:42.335 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 67650K .......... .......... .......... .......... .......... 40% 14.3M 4s 2024-09-12T12:57:42.337 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 67700K .......... .......... .......... .......... .......... 40% 24.7M 4s 2024-09-12T12:57:42.340 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 67750K .......... .......... .......... .......... .......... 40% 21.0M 4s 2024-09-12T12:57:42.343 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 67800K .......... .......... .......... .......... .......... 40% 16.3M 4s 2024-09-12T12:57:42.344 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 67850K .......... .......... .......... .......... .......... 40% 26.4M 4s 2024-09-12T12:57:42.346 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 67900K .......... .......... .......... .......... .......... 40% 42.9M 4s 2024-09-12T12:57:42.348 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 67950K .......... .......... .......... .......... .......... 40% 18.7M 4s 2024-09-12T12:57:42.352 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 68000K .......... .......... .......... .......... .......... 40% 12.9M 4s 2024-09-12T12:57:42.356 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 68050K .......... .......... .......... .......... .......... 40% 12.4M 4s 2024-09-12T12:57:42.358 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 68100K .......... .......... .......... .......... .......... 40% 24.8M 4s 2024-09-12T12:57:42.362 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 68150K .......... .......... .......... .......... .......... 40% 10.8M 4s 2024-09-12T12:57:42.364 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 68200K .......... .......... .......... .......... .......... 40% 24.3M 4s 2024-09-12T12:57:42.367 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 68250K .......... .......... .......... .......... .......... 40% 16.8M 4s 2024-09-12T12:57:42.371 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 68300K .......... .......... .......... .......... .......... 40% 13.5M 4s 2024-09-12T12:57:42.374 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 68350K .......... .......... .......... .......... .......... 40% 15.5M 4s 2024-09-12T12:57:42.375 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 68400K .......... .......... .......... .......... .......... 40% 32.3M 4s 2024-09-12T12:57:42.378 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 68450K .......... .......... .......... .......... .......... 40% 19.0M 4s 2024-09-12T12:57:42.381 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 68500K .......... .......... .......... .......... .......... 40% 23.9M 4s 2024-09-12T12:57:42.383 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 68550K .......... .......... .......... .......... .......... 40% 20.4M 4s 2024-09-12T12:57:42.386 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 68600K .......... .......... .......... .......... .......... 40% 20.8M 4s 2024-09-12T12:57:42.388 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 68650K .......... .......... .......... .......... .......... 40% 14.9M 4s 2024-09-12T12:57:42.390 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 68700K .......... .......... .......... .......... .......... 40% 25.5M 4s 2024-09-12T12:57:42.393 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 68750K .......... .......... .......... .......... .......... 40% 15.8M 4s 2024-09-12T12:57:42.395 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 68800K .......... .......... .......... .......... .......... 40% 24.0M 4s 2024-09-12T12:57:42.399 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 68850K .......... .......... .......... .......... .......... 40% 14.4M 4s 2024-09-12T12:57:42.400 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 68900K .......... .......... .......... .......... .......... 40% 59.1M 4s 2024-09-12T12:57:42.401 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 68950K .......... .......... .......... .......... .......... 40% 27.5M 4s 2024-09-12T12:57:42.403 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 69000K .......... .......... .......... .......... .......... 40% 27.3M 4s 2024-09-12T12:57:42.406 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 69050K .......... .......... .......... .......... .......... 40% 15.0M 4s 2024-09-12T12:57:42.408 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 69100K .......... .......... .......... .......... .......... 41% 23.9M 4s 2024-09-12T12:57:42.411 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 69150K .......... .......... .......... .......... .......... 41% 20.8M 4s 2024-09-12T12:57:42.414 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 69200K .......... .......... .......... .......... .......... 41% 15.3M 4s 2024-09-12T12:57:42.416 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 69250K .......... .......... .......... .......... .......... 41% 19.9M 4s 2024-09-12T12:57:42.421 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 69300K .......... .......... .......... .......... .......... 41% 11.9M 4s 2024-09-12T12:57:42.423 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 69350K .......... .......... .......... .......... .......... 41% 18.5M 4s 2024-09-12T12:57:42.429 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 69400K .......... .......... .......... .......... .......... 41% 12.1M 4s 2024-09-12T12:57:42.429 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 69450K .......... .......... .......... .......... .......... 41% 23.4M 4s 2024-09-12T12:57:42.431 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 69500K .......... .......... .......... .......... .......... 41% 27.8M 4s 2024-09-12T12:57:42.437 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 69550K .......... .......... .......... .......... .......... 41% 10.6M 4s 2024-09-12T12:57:42.440 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 69600K .......... .......... .......... .......... .......... 41% 18.1M 4s 2024-09-12T12:57:42.440 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 69650K .......... .......... .......... .......... .......... 41% 37.5M 4s 2024-09-12T12:57:42.446 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 69700K .......... .......... .......... .......... .......... 41% 21.7M 4s 2024-09-12T12:57:42.446 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 69750K .......... .......... .......... .......... .......... 41% 36.0M 4s 2024-09-12T12:57:42.446 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 69800K .......... .......... .......... .......... .......... 41% 28.5M 4s 2024-09-12T12:57:42.447 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 69850K .......... .......... .......... .......... .......... 41% 25.2M 4s 2024-09-12T12:57:42.450 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 69900K .......... .......... .......... .......... .......... 41% 14.4M 4s 2024-09-12T12:57:42.454 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 69950K .......... .......... .......... .......... .......... 41% 16.4M 4s 2024-09-12T12:57:42.455 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 70000K .......... .......... .......... .......... .......... 41% 29.1M 4s 2024-09-12T12:57:42.458 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 70050K .......... .......... .......... .......... .......... 41% 18.8M 4s 2024-09-12T12:57:42.461 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 70100K .......... .......... .......... .......... .......... 41% 16.0M 4s 2024-09-12T12:57:42.462 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 70150K .......... .......... .......... .......... .......... 41% 31.2M 4s 2024-09-12T12:57:42.464 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 70200K .......... .......... .......... .......... .......... 41% 23.1M 4s 2024-09-12T12:57:42.467 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 70250K .......... .......... .......... .......... .......... 41% 15.1M 4s 2024-09-12T12:57:42.470 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 70300K .......... .......... .......... .......... .......... 41% 28.3M 4s 2024-09-12T12:57:42.473 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 70350K .......... .......... .......... .......... .......... 41% 14.0M 4s 2024-09-12T12:57:42.476 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 70400K .......... .......... .......... .......... .......... 41% 13.5M 4s 2024-09-12T12:57:42.482 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 70450K .......... .......... .......... .......... .......... 41% 9.13M 4s 2024-09-12T12:57:42.484 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 70500K .......... .......... .......... .......... .......... 41% 25.0M 4s 2024-09-12T12:57:42.486 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 70550K .......... .......... .......... .......... .......... 41% 19.3M 4s 2024-09-12T12:57:42.491 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 70600K .......... .......... .......... .......... .......... 41% 11.1M 4s 2024-09-12T12:57:42.494 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 70650K .......... .......... .......... .......... .......... 41% 21.0M 4s 2024-09-12T12:57:42.494 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 70700K .......... .......... .......... .......... .......... 41% 30.7M 4s 2024-09-12T12:57:42.498 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 70750K .......... .......... .......... .......... .......... 42% 17.7M 4s 2024-09-12T12:57:42.500 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 70800K .......... .......... .......... .......... .......... 42% 21.6M 4s 2024-09-12T12:57:42.502 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 70850K .......... .......... .......... .......... .......... 42% 23.3M 4s 2024-09-12T12:57:42.504 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 70900K .......... .......... .......... .......... .......... 42% 23.3M 4s 2024-09-12T12:57:42.506 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 70950K .......... .......... .......... .......... .......... 42% 21.4M 4s 2024-09-12T12:57:42.509 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 71000K .......... .......... .......... .......... .......... 42% 18.9M 4s 2024-09-12T12:57:42.513 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 71050K .......... .......... .......... .......... .......... 42% 14.0M 4s 2024-09-12T12:57:42.516 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 71100K .......... .......... .......... .......... .......... 42% 12.2M 4s 2024-09-12T12:57:42.520 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 71150K .......... .......... .......... .......... .......... 42% 14.9M 4s 2024-09-12T12:57:42.521 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 71200K .......... .......... .......... .......... .......... 42% 26.0M 4s 2024-09-12T12:57:42.523 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 71250K .......... .......... .......... .......... .......... 42% 26.2M 4s 2024-09-12T12:57:42.527 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 71300K .......... .......... .......... .......... .......... 42% 13.9M 4s 2024-09-12T12:57:42.528 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 71350K .......... .......... .......... .......... .......... 42% 37.0M 4s 2024-09-12T12:57:42.530 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 71400K .......... .......... .......... .......... .......... 42% 27.0M 4s 2024-09-12T12:57:42.533 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 71450K .......... .......... .......... .......... .......... 42% 21.0M 4s 2024-09-12T12:57:42.535 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 71500K .......... .......... .......... .......... .......... 42% 21.3M 4s 2024-09-12T12:57:42.537 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 71550K .......... .......... .......... .......... .......... 42% 16.7M 4s 2024-09-12T12:57:42.541 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 71600K .......... .......... .......... .......... .......... 42% 13.4M 4s 2024-09-12T12:57:42.545 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 71650K .......... .......... .......... .......... .......... 42% 15.8M 4s 2024-09-12T12:57:42.547 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 71700K .......... .......... .......... .......... .......... 42% 18.3M 4s 2024-09-12T12:57:42.549 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 71750K .......... .......... .......... .......... .......... 42% 18.9M 4s 2024-09-12T12:57:42.552 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 71800K .......... .......... .......... .......... .......... 42% 16.1M 4s 2024-09-12T12:57:42.556 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 71850K .......... .......... .......... .......... .......... 42% 16.0M 4s 2024-09-12T12:57:42.557 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 71900K .......... .......... .......... .......... .......... 42% 22.3M 4s 2024-09-12T12:57:42.561 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 71950K .......... .......... .......... .......... .......... 42% 14.7M 4s 2024-09-12T12:57:42.565 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 72000K .......... .......... .......... .......... .......... 42% 14.7M 4s 2024-09-12T12:57:42.566 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 72050K .......... .......... .......... .......... .......... 42% 30.3M 4s 2024-09-12T12:57:42.568 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 72100K .......... .......... .......... .......... .......... 42% 29.7M 4s 2024-09-12T12:57:42.569 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 72150K .......... .......... .......... .......... .......... 42% 29.5M 4s 2024-09-12T12:57:42.572 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 72200K .......... .......... .......... .......... .......... 42% 17.7M 4s 2024-09-12T12:57:42.575 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 72250K .......... .......... .......... .......... .......... 42% 15.4M 4s 2024-09-12T12:57:42.577 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 72300K .......... .......... .......... .......... .......... 42% 20.6M 4s 2024-09-12T12:57:42.581 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 72350K .......... .......... .......... .......... .......... 42% 12.6M 4s 2024-09-12T12:57:42.583 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 72400K .......... .......... .......... .......... .......... 42% 25.3M 4s 2024-09-12T12:57:42.586 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 72450K .......... .......... .......... .......... .......... 43% 21.5M 4s 2024-09-12T12:57:42.587 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 72500K .......... .......... .......... .......... .......... 43% 28.7M 4s 2024-09-12T12:57:42.590 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 72550K .......... .......... .......... .......... .......... 43% 15.3M 4s 2024-09-12T12:57:42.592 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 72600K .......... .......... .......... .......... .......... 43% 23.2M 4s 2024-09-12T12:57:42.595 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 72650K .......... .......... .......... .......... .......... 43% 18.1M 4s 2024-09-12T12:57:42.600 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 72700K .......... .......... .......... .......... .......... 43% 11.7M 4s 2024-09-12T12:57:42.604 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 72750K .......... .......... .......... .......... .......... 43% 10.7M 4s 2024-09-12T12:57:42.605 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 72800K .......... .......... .......... .......... .......... 43% 39.0M 4s 2024-09-12T12:57:42.608 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 72850K .......... .......... .......... .......... .......... 43% 16.1M 4s 2024-09-12T12:57:42.613 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 72900K .......... .......... .......... .......... .......... 43% 12.9M 4s 2024-09-12T12:57:42.615 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 72950K .......... .......... .......... .......... .......... 43% 18.0M 4s 2024-09-12T12:57:42.617 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 73000K .......... .......... .......... .......... .......... 43% 19.9M 4s 2024-09-12T12:57:42.620 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 73050K .......... .......... .......... .......... .......... 43% 19.5M 4s 2024-09-12T12:57:42.622 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 73100K .......... .......... .......... .......... .......... 43% 16.7M 4s 2024-09-12T12:57:42.625 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 73150K .......... .......... .......... .......... .......... 43% 18.2M 4s 2024-09-12T12:57:42.627 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 73200K .......... .......... .......... .......... .......... 43% 37.0M 4s 2024-09-12T12:57:42.630 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 73250K .......... .......... .......... .......... .......... 43% 16.6M 4s 2024-09-12T12:57:42.631 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 73300K .......... .......... .......... .......... .......... 43% 37.3M 4s 2024-09-12T12:57:42.635 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 73350K .......... .......... .......... .......... .......... 43% 16.4M 4s 2024-09-12T12:57:42.642 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 73400K .......... .......... .......... .......... .......... 43% 7.10M 4s 2024-09-12T12:57:42.645 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 73450K .......... .......... .......... .......... .......... 43% 24.1M 4s 2024-09-12T12:57:42.645 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 73500K .......... .......... .......... .......... .......... 43% 43.8M 4s 2024-09-12T12:57:42.648 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 73550K .......... .......... .......... .......... .......... 43% 16.8M 4s 2024-09-12T12:57:42.651 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 73600K .......... .......... .......... .......... .......... 43% 16.3M 4s 2024-09-12T12:57:42.652 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 73650K .......... .......... .......... .......... .......... 43% 21.0M 4s 2024-09-12T12:57:42.655 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 73700K .......... .......... .......... .......... .......... 43% 26.0M 4s 2024-09-12T12:57:42.657 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 73750K .......... .......... .......... .......... .......... 43% 21.5M 4s 2024-09-12T12:57:42.659 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 73800K .......... .......... .......... .......... .......... 43% 17.7M 4s 2024-09-12T12:57:42.662 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 73850K .......... .......... .......... .......... .......... 43% 15.9M 4s 2024-09-12T12:57:42.665 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 73900K .......... .......... .......... .......... .......... 43% 17.1M 4s 2024-09-12T12:57:42.668 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 73950K .......... .......... .......... .......... .......... 43% 14.9M 4s 2024-09-12T12:57:42.671 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 74000K .......... .......... .......... .......... .......... 43% 16.9M 4s 2024-09-12T12:57:42.674 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 74050K .......... .......... .......... .......... .......... 43% 14.5M 4s 2024-09-12T12:57:42.677 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 74100K .......... .......... .......... .......... .......... 43% 16.6M 4s 2024-09-12T12:57:42.679 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 74150K .......... .......... .......... .......... .......... 44% 22.8M 4s 2024-09-12T12:57:42.682 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 74200K .......... .......... .......... .......... .......... 44% 21.4M 4s 2024-09-12T12:57:42.685 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 74250K .......... .......... .......... .......... .......... 44% 15.0M 4s 2024-09-12T12:57:42.686 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 74300K .......... .......... .......... .......... .......... 44% 32.2M 4s 2024-09-12T12:57:42.689 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 74350K .......... .......... .......... .......... .......... 44% 18.4M 4s 2024-09-12T12:57:42.691 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 74400K .......... .......... .......... .......... .......... 44% 19.7M 4s 2024-09-12T12:57:42.694 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 74450K .......... .......... .......... .......... .......... 44% 18.2M 4s 2024-09-12T12:57:42.698 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 74500K .......... .......... .......... .......... .......... 44% 13.5M 4s 2024-09-12T12:57:42.700 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 74550K .......... .......... .......... .......... .......... 44% 34.4M 4s 2024-09-12T12:57:42.702 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 74600K .......... .......... .......... .......... .......... 44% 17.9M 4s 2024-09-12T12:57:42.705 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 74650K .......... .......... .......... .......... .......... 44% 18.7M 4s 2024-09-12T12:57:42.707 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 74700K .......... .......... .......... .......... .......... 44% 22.0M 4s 2024-09-12T12:57:42.709 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 74750K .......... .......... .......... .......... .......... 44% 21.1M 4s 2024-09-12T12:57:42.713 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 74800K .......... .......... .......... .......... .......... 44% 17.8M 4s 2024-09-12T12:57:42.714 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 74850K .......... .......... .......... .......... .......... 44% 22.6M 4s 2024-09-12T12:57:42.716 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 74900K .......... .......... .......... .......... .......... 44% 26.8M 4s 2024-09-12T12:57:42.720 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 74950K .......... .......... .......... .......... .......... 44% 14.6M 4s 2024-09-12T12:57:42.723 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 75000K .......... .......... .......... .......... .......... 44% 16.9M 4s 2024-09-12T12:57:42.725 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 75050K .......... .......... .......... .......... .......... 44% 16.8M 4s 2024-09-12T12:57:42.728 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 75100K .......... .......... .......... .......... .......... 44% 27.3M 4s 2024-09-12T12:57:42.732 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 75150K .......... .......... .......... .......... .......... 44% 11.5M 4s 2024-09-12T12:57:42.734 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 75200K .......... .......... .......... .......... .......... 44% 18.5M 4s 2024-09-12T12:57:42.737 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 75250K .......... .......... .......... .......... .......... 44% 15.6M 4s 2024-09-12T12:57:42.739 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 75300K .......... .......... .......... .......... .......... 44% 24.9M 4s 2024-09-12T12:57:42.742 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 75350K .......... .......... .......... .......... .......... 44% 13.3M 4s 2024-09-12T12:57:42.744 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 75400K .......... .......... .......... .......... .......... 44% 33.3M 4s 2024-09-12T12:57:42.746 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 75450K .......... .......... .......... .......... .......... 44% 23.7M 4s 2024-09-12T12:57:42.751 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 75500K .......... .......... .......... .......... .......... 44% 12.7M 4s 2024-09-12T12:57:42.753 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 75550K .......... .......... .......... .......... .......... 44% 18.7M 4s 2024-09-12T12:57:42.754 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 75600K .......... .......... .......... .......... .......... 44% 29.1M 4s 2024-09-12T12:57:42.757 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 75650K .......... .......... .......... .......... .......... 44% 19.1M 4s 2024-09-12T12:57:42.759 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 75700K .......... .......... .......... .......... .......... 44% 26.0M 4s 2024-09-12T12:57:42.762 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 75750K .......... .......... .......... .......... .......... 44% 15.7M 4s 2024-09-12T12:57:42.764 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 75800K .......... .......... .......... .......... .......... 44% 19.0M 4s 2024-09-12T12:57:42.772 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 75850K .......... .......... .......... .......... .......... 45% 6.32M 4s 2024-09-12T12:57:42.776 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 75900K .......... .......... .......... .......... .......... 45% 11.8M 4s 2024-09-12T12:57:42.779 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 75950K .......... .......... .......... .......... .......... 45% 16.9M 4s 2024-09-12T12:57:42.782 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 76000K .......... .......... .......... .......... .......... 45% 17.1M 4s 2024-09-12T12:57:42.785 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 76050K .......... .......... .......... .......... .......... 45% 16.9M 4s 2024-09-12T12:57:42.787 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 76100K .......... .......... .......... .......... .......... 45% 27.4M 4s 2024-09-12T12:57:42.788 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 76150K .......... .......... .......... .......... .......... 45% 25.6M 4s 2024-09-12T12:57:42.792 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 76200K .......... .......... .......... .......... .......... 45% 12.8M 4s 2024-09-12T12:57:42.798 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 76250K .......... .......... .......... .......... .......... 45% 11.2M 4s 2024-09-12T12:57:42.798 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 76300K .......... .......... .......... .......... .......... 45% 35.4M 4s 2024-09-12T12:57:42.803 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 76350K .......... .......... .......... .......... .......... 45% 11.4M 4s 2024-09-12T12:57:42.808 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 76400K .......... .......... .......... .......... .......... 45% 10.3M 4s 2024-09-12T12:57:42.808 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 76450K .......... .......... .......... .......... .......... 45% 47.2M 4s 2024-09-12T12:57:42.811 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 76500K .......... .......... .......... .......... .......... 45% 20.4M 4s 2024-09-12T12:57:42.815 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 76550K .......... .......... .......... .......... .......... 45% 15.1M 4s 2024-09-12T12:57:42.815 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 76600K .......... .......... .......... .......... .......... 45% 32.6M 4s 2024-09-12T12:57:42.817 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 76650K .......... .......... .......... .......... .......... 45% 21.8M 4s 2024-09-12T12:57:42.820 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 76700K .......... .......... .......... .......... .......... 45% 21.6M 4s 2024-09-12T12:57:42.824 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 76750K .......... .......... .......... .......... .......... 45% 12.9M 4s 2024-09-12T12:57:42.825 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 76800K .......... .......... .......... .......... .......... 45% 25.1M 4s 2024-09-12T12:57:42.827 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 76850K .......... .......... .......... .......... .......... 45% 24.8M 4s 2024-09-12T12:57:42.831 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 76900K .......... .......... .......... .......... .......... 45% 14.6M 4s 2024-09-12T12:57:42.835 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 76950K .......... .......... .......... .......... .......... 45% 10.3M 4s 2024-09-12T12:57:42.837 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 77000K .......... .......... .......... .......... .......... 45% 27.8M 4s 2024-09-12T12:57:42.840 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 77050K .......... .......... .......... .......... .......... 45% 20.8M 4s 2024-09-12T12:57:42.842 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 77100K .......... .......... .......... .......... .......... 45% 18.7M 4s 2024-09-12T12:57:42.847 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 77150K .......... .......... .......... .......... .......... 45% 14.6M 4s 2024-09-12T12:57:42.848 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 77200K .......... .......... .......... .......... .......... 45% 26.0M 4s 2024-09-12T12:57:42.851 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 77250K .......... .......... .......... .......... .......... 45% 14.5M 4s 2024-09-12T12:57:42.853 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 77300K .......... .......... .......... .......... .......... 45% 18.6M 4s 2024-09-12T12:57:42.857 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 77350K .......... .......... .......... .......... .......... 45% 21.5M 4s 2024-09-12T12:57:42.859 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 77400K .......... .......... .......... .......... .......... 45% 15.5M 4s 2024-09-12T12:57:42.861 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 77450K .......... .......... .......... .......... .......... 45% 19.9M 4s 2024-09-12T12:57:42.867 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 77500K .......... .......... .......... .......... .......... 46% 8.35M 4s 2024-09-12T12:57:42.870 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 77550K .......... .......... .......... .......... .......... 46% 16.7M 4s 2024-09-12T12:57:42.872 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 77600K .......... .......... .......... .......... .......... 46% 22.0M 4s 2024-09-12T12:57:42.875 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 77650K .......... .......... .......... .......... .......... 46% 19.5M 4s 2024-09-12T12:57:42.877 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 77700K .......... .......... .......... .......... .......... 46% 19.0M 4s 2024-09-12T12:57:42.880 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 77750K .......... .......... .......... .......... .......... 46% 17.4M 4s 2024-09-12T12:57:42.882 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 77800K .......... .......... .......... .......... .......... 46% 20.7M 4s 2024-09-12T12:57:42.884 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 77850K .......... .......... .......... .......... .......... 46% 33.7M 4s 2024-09-12T12:57:42.887 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 77900K .......... .......... .......... .......... .......... 46% 18.9M 4s 2024-09-12T12:57:42.890 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 77950K .......... .......... .......... .......... .......... 46% 16.7M 4s 2024-09-12T12:57:42.896 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 78000K .......... .......... .......... .......... .......... 46% 7.09M 4s 2024-09-12T12:57:42.901 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 78050K .......... .......... .......... .......... .......... 46% 10.0M 4s 2024-09-12T12:57:42.904 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 78100K .......... .......... .......... .......... .......... 46% 20.7M 4s 2024-09-12T12:57:42.909 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 78150K .......... .......... .......... .......... .......... 46% 9.13M 4s 2024-09-12T12:57:42.911 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 78200K .......... .......... .......... .......... .......... 46% 24.8M 4s 2024-09-12T12:57:42.913 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 78250K .......... .......... .......... .......... .......... 46% 27.6M 4s 2024-09-12T12:57:42.916 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 78300K .......... .......... .......... .......... .......... 46% 15.2M 4s 2024-09-12T12:57:42.919 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 78350K .......... .......... .......... .......... .......... 46% 13.7M 4s 2024-09-12T12:57:42.921 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 78400K .......... .......... .......... .......... .......... 46% 23.9M 4s 2024-09-12T12:57:42.924 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 78450K .......... .......... .......... .......... .......... 46% 23.6M 4s 2024-09-12T12:57:42.928 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 78500K .......... .......... .......... .......... .......... 46% 12.3M 4s 2024-09-12T12:57:42.931 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 78550K .......... .......... .......... .......... .......... 46% 14.1M 4s 2024-09-12T12:57:42.935 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 78600K .......... .......... .......... .......... .......... 46% 12.6M 4s 2024-09-12T12:57:42.938 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 78650K .......... .......... .......... .......... .......... 46% 14.0M 4s 2024-09-12T12:57:42.940 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 78700K .......... .......... .......... .......... .......... 46% 28.7M 4s 2024-09-12T12:57:42.944 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 78750K .......... .......... .......... .......... .......... 46% 12.7M 4s 2024-09-12T12:57:42.946 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 78800K .......... .......... .......... .......... .......... 46% 27.0M 4s 2024-09-12T12:57:42.949 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 78850K .......... .......... .......... .......... .......... 46% 16.9M 4s 2024-09-12T12:57:42.951 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 78900K .......... .......... .......... .......... .......... 46% 22.3M 4s 2024-09-12T12:57:42.953 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 78950K .......... .......... .......... .......... .......... 46% 26.6M 4s 2024-09-12T12:57:42.955 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 79000K .......... .......... .......... .......... .......... 46% 17.4M 4s 2024-09-12T12:57:42.958 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 79050K .......... .......... .......... .......... .......... 46% 19.3M 4s 2024-09-12T12:57:42.961 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 79100K .......... .......... .......... .......... .......... 46% 16.5M 4s 2024-09-12T12:57:42.967 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 79150K .......... .......... .......... .......... .......... 46% 8.56M 4s 2024-09-12T12:57:42.971 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 79200K .......... .......... .......... .......... .......... 47% 10.6M 4s 2024-09-12T12:57:42.973 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 79250K .......... .......... .......... .......... .......... 47% 30.5M 4s 2024-09-12T12:57:42.975 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 79300K .......... .......... .......... .......... .......... 47% 22.6M 4s 2024-09-12T12:57:42.979 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 79350K .......... .......... .......... .......... .......... 47% 13.4M 4s 2024-09-12T12:57:42.980 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 79400K .......... .......... .......... .......... .......... 47% 25.8M 4s 2024-09-12T12:57:42.983 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 79450K .......... .......... .......... .......... .......... 47% 21.2M 4s 2024-09-12T12:57:42.986 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 79500K .......... .......... .......... .......... .......... 47% 13.8M 4s 2024-09-12T12:57:42.990 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 79550K .......... .......... .......... .......... .......... 47% 14.8M 4s 2024-09-12T12:57:42.993 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 79600K .......... .......... .......... .......... .......... 47% 15.3M 4s 2024-09-12T12:57:42.998 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 79650K .......... .......... .......... .......... .......... 47% 9.48M 4s 2024-09-12T12:57:43.001 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 79700K .......... .......... .......... .......... .......... 47% 17.1M 4s 2024-09-12T12:57:43.003 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 79750K .......... .......... .......... .......... .......... 47% 19.2M 4s 2024-09-12T12:57:43.005 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 79800K .......... .......... .......... .......... .......... 47% 29.9M 4s 2024-09-12T12:57:43.008 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 79850K .......... .......... .......... .......... .......... 47% 18.3M 4s 2024-09-12T12:57:43.010 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 79900K .......... .......... .......... .......... .......... 47% 19.5M 4s 2024-09-12T12:57:43.013 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 79950K .......... .......... .......... .......... .......... 47% 15.1M 4s 2024-09-12T12:57:43.015 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 80000K .......... .......... .......... .......... .......... 47% 25.7M 4s 2024-09-12T12:57:43.019 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 80050K .......... .......... .......... .......... .......... 47% 16.4M 4s 2024-09-12T12:57:43.020 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 80100K .......... .......... .......... .......... .......... 47% 29.4M 4s 2024-09-12T12:57:43.024 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 80150K .......... .......... .......... .......... .......... 47% 12.4M 4s 2024-09-12T12:57:43.029 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 80200K .......... .......... .......... .......... .......... 47% 9.40M 4s 2024-09-12T12:57:43.033 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 80250K .......... .......... .......... .......... .......... 47% 11.4M 4s 2024-09-12T12:57:43.035 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 80300K .......... .......... .......... .......... .......... 47% 32.7M 4s 2024-09-12T12:57:43.038 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 80350K .......... .......... .......... .......... .......... 47% 18.1M 4s 2024-09-12T12:57:43.039 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 80400K .......... .......... .......... .......... .......... 47% 33.9M 4s 2024-09-12T12:57:43.041 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 80450K .......... .......... .......... .......... .......... 47% 23.7M 4s 2024-09-12T12:57:43.044 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 80500K .......... .......... .......... .......... .......... 47% 26.5M 4s 2024-09-12T12:57:43.046 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 80550K .......... .......... .......... .......... .......... 47% 21.2M 4s 2024-09-12T12:57:43.049 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 80600K .......... .......... .......... .......... .......... 47% 13.8M 4s 2024-09-12T12:57:43.051 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 80650K .......... .......... .......... .......... .......... 47% 24.2M 4s 2024-09-12T12:57:43.053 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 80700K .......... .......... .......... .......... .......... 47% 19.2M 4s 2024-09-12T12:57:43.059 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 80750K .......... .......... .......... .......... .......... 47% 9.84M 4s 2024-09-12T12:57:43.062 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 80800K .......... .......... .......... .......... .......... 47% 14.6M 4s 2024-09-12T12:57:43.067 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 80850K .......... .......... .......... .......... .......... 47% 9.91M 4s 2024-09-12T12:57:43.069 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 80900K .......... .......... .......... .......... .......... 48% 20.9M 4s 2024-09-12T12:57:43.072 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 80950K .......... .......... .......... .......... .......... 48% 15.5M 4s 2024-09-12T12:57:43.074 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 81000K .......... .......... .......... .......... .......... 48% 28.2M 4s 2024-09-12T12:57:43.077 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 81050K .......... .......... .......... .......... .......... 48% 16.2M 4s 2024-09-12T12:57:43.080 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 81100K .......... .......... .......... .......... .......... 48% 18.9M 4s 2024-09-12T12:57:43.083 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 81150K .......... .......... .......... .......... .......... 48% 14.5M 4s 2024-09-12T12:57:43.084 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 81200K .......... .......... .......... .......... .......... 48% 39.8M 4s 2024-09-12T12:57:43.088 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 81250K .......... .......... .......... .......... .......... 48% 13.1M 4s 2024-09-12T12:57:43.093 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 81300K .......... .......... .......... .......... .......... 48% 9.68M 4s 2024-09-12T12:57:43.096 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 81350K .......... .......... .......... .......... .......... 48% 15.5M 4s 2024-09-12T12:57:43.098 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 81400K .......... .......... .......... .......... .......... 48% 23.3M 4s 2024-09-12T12:57:43.102 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 81450K .......... .......... .......... .......... .......... 48% 13.0M 4s 2024-09-12T12:57:43.105 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 81500K .......... .......... .......... .......... .......... 48% 18.9M 4s 2024-09-12T12:57:43.107 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 81550K .......... .......... .......... .......... .......... 48% 17.6M 4s 2024-09-12T12:57:43.110 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 81600K .......... .......... .......... .......... .......... 48% 21.1M 4s 2024-09-12T12:57:43.112 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 81650K .......... .......... .......... .......... .......... 48% 19.4M 4s 2024-09-12T12:57:43.115 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 81700K .......... .......... .......... .......... .......... 48% 18.3M 4s 2024-09-12T12:57:43.117 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 81750K .......... .......... .......... .......... .......... 48% 22.4M 4s 2024-09-12T12:57:43.121 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 81800K .......... .......... .......... .......... .......... 48% 11.7M 4s 2024-09-12T12:57:43.123 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 81850K .......... .......... .......... .......... .......... 48% 29.2M 4s 2024-09-12T12:57:43.127 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 81900K .......... .......... .......... .......... .......... 48% 10.3M 4s 2024-09-12T12:57:43.132 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 81950K .......... .......... .......... .......... .......... 48% 10.9M 4s 2024-09-12T12:57:43.135 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 82000K .......... .......... .......... .......... .......... 48% 15.2M 4s 2024-09-12T12:57:43.137 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 82050K .......... .......... .......... .......... .......... 48% 21.3M 4s 2024-09-12T12:57:43.140 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 82100K .......... .......... .......... .......... .......... 48% 23.0M 4s 2024-09-12T12:57:43.143 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 82150K .......... .......... .......... .......... .......... 48% 15.9M 4s 2024-09-12T12:57:43.145 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 82200K .......... .......... .......... .......... .......... 48% 22.4M 4s 2024-09-12T12:57:43.147 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 82250K .......... .......... .......... .......... .......... 48% 21.1M 4s 2024-09-12T12:57:43.149 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 82300K .......... .......... .......... .......... .......... 48% 37.1M 4s 2024-09-12T12:57:43.153 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 82350K .......... .......... .......... .......... .......... 48% 11.6M 4s 2024-09-12T12:57:43.155 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 82400K .......... .......... .......... .......... .......... 48% 27.0M 4s 2024-09-12T12:57:43.156 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 82450K .......... .......... .......... .......... .......... 48% 27.2M 4s 2024-09-12T12:57:43.158 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 82500K .......... .......... .......... .......... .......... 48% 27.5M 4s 2024-09-12T12:57:43.161 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 82550K .......... .......... .......... .......... .......... 49% 22.6M 4s 2024-09-12T12:57:43.161 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 82600K .......... .......... .......... .......... .......... 49% 40.7M 4s 2024-09-12T12:57:43.163 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 82650K .......... .......... .......... .......... .......... 49% 34.6M 4s 2024-09-12T12:57:43.165 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 82700K .......... .......... .......... .......... .......... 49% 22.4M 4s 2024-09-12T12:57:43.168 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 82750K .......... .......... .......... .......... .......... 49% 19.0M 4s 2024-09-12T12:57:43.170 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 82800K .......... .......... .......... .......... .......... 49% 18.9M 4s 2024-09-12T12:57:43.174 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 82850K .......... .......... .......... .......... .......... 49% 13.4M 4s 2024-09-12T12:57:43.176 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 82900K .......... .......... .......... .......... .......... 49% 19.3M 4s 2024-09-12T12:57:43.179 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 82950K .......... .......... .......... .......... .......... 49% 19.9M 4s 2024-09-12T12:57:43.184 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 83000K .......... .......... .......... .......... .......... 49% 11.2M 4s 2024-09-12T12:57:43.188 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 83050K .......... .......... .......... .......... .......... 49% 17.3M 4s 2024-09-12T12:57:43.192 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 83100K .......... .......... .......... .......... .......... 49% 11.4M 4s 2024-09-12T12:57:43.195 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 83150K .......... .......... .......... .......... .......... 49% 12.6M 4s 2024-09-12T12:57:43.197 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 83200K .......... .......... .......... .......... .......... 49% 22.7M 4s 2024-09-12T12:57:43.199 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 83250K .......... .......... .......... .......... .......... 49% 19.7M 4s 2024-09-12T12:57:43.202 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 83300K .......... .......... .......... .......... .......... 49% 16.2M 4s 2024-09-12T12:57:43.205 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 83350K .......... .......... .......... .......... .......... 49% 16.5M 4s 2024-09-12T12:57:43.207 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 83400K .......... .......... .......... .......... .......... 49% 21.0M 4s 2024-09-12T12:57:43.209 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 83450K .......... .......... .......... .......... .......... 49% 27.4M 4s 2024-09-12T12:57:43.212 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 83500K .......... .......... .......... .......... .......... 49% 17.4M 4s 2024-09-12T12:57:43.226 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 83550K .......... .......... .......... .......... .......... 49% 3.40M 4s 2024-09-12T12:57:43.230 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 83600K .......... .......... .......... .......... .......... 49% 13.6M 4s 2024-09-12T12:57:43.232 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 83650K .......... .......... .......... .......... .......... 49% 16.6M 4s 2024-09-12T12:57:43.235 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 83700K .......... .......... .......... .......... .......... 49% 20.4M 4s 2024-09-12T12:57:43.238 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 83750K .......... .......... .......... .......... .......... 49% 16.1M 4s 2024-09-12T12:57:43.240 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 83800K .......... .......... .......... .......... .......... 49% 25.4M 4s 2024-09-12T12:57:43.243 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 83850K .......... .......... .......... .......... .......... 49% 19.4M 4s 2024-09-12T12:57:43.249 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 83900K .......... .......... .......... .......... .......... 49% 14.9M 4s 2024-09-12T12:57:43.249 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 83950K .......... .......... .......... .......... .......... 49% 16.2M 4s 2024-09-12T12:57:43.250 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 84000K .......... .......... .......... .......... .......... 49% 24.8M 4s 2024-09-12T12:57:43.253 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 84050K .......... .......... .......... .......... .......... 49% 22.2M 4s 2024-09-12T12:57:43.256 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 84100K .......... .......... .......... .......... .......... 49% 15.4M 4s 2024-09-12T12:57:43.259 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 84150K .......... .......... .......... .......... .......... 49% 17.4M 4s 2024-09-12T12:57:43.263 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 84200K .......... .......... .......... .......... .......... 49% 12.1M 4s 2024-09-12T12:57:43.268 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 84250K .......... .......... .......... .......... .......... 50% 9.44M 4s 2024-09-12T12:57:43.272 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 84300K .......... .......... .......... .......... .......... 50% 16.3M 4s 2024-09-12T12:57:43.274 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 84350K .......... .......... .......... .......... .......... 50% 14.7M 4s 2024-09-12T12:57:43.276 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 84400K .......... .......... .......... .......... .......... 50% 27.9M 4s 2024-09-12T12:57:43.280 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 84450K .......... .......... .......... .......... .......... 50% 14.6M 4s 2024-09-12T12:57:43.281 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 84500K .......... .......... .......... .......... .......... 50% 33.0M 4s 2024-09-12T12:57:43.283 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 84550K .......... .......... .......... .......... .......... 50% 19.9M 4s 2024-09-12T12:57:43.289 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 84600K .......... .......... .......... .......... .......... 50% 8.43M 4s 2024-09-12T12:57:43.291 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 84650K .......... .......... .......... .......... .......... 50% 23.1M 4s 2024-09-12T12:57:43.293 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 84700K .......... .......... .......... .......... .......... 50% 28.1M 4s 2024-09-12T12:57:43.295 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 84750K .......... .......... .......... .......... .......... 50% 25.2M 4s 2024-09-12T12:57:43.298 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 84800K .......... .......... .......... .......... .......... 50% 18.6M 4s 2024-09-12T12:57:43.299 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 84850K .......... .......... .......... .......... .......... 50% 24.0M 4s 2024-09-12T12:57:43.302 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 84900K .......... .......... .......... .......... .......... 50% 21.3M 4s 2024-09-12T12:57:43.304 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 84950K .......... .......... .......... .......... .......... 50% 20.2M 4s 2024-09-12T12:57:43.308 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 85000K .......... .......... .......... .......... .......... 50% 12.5M 4s 2024-09-12T12:57:43.310 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 85050K .......... .......... .......... .......... .......... 50% 27.2M 4s 2024-09-12T12:57:43.316 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 85100K .......... .......... .......... .......... .......... 50% 11.4M 4s 2024-09-12T12:57:43.317 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 85150K .......... .......... .......... .......... .......... 50% 17.4M 4s 2024-09-12T12:57:43.318 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 85200K .......... .......... .......... .......... .......... 50% 31.8M 4s 2024-09-12T12:57:43.321 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 85250K .......... .......... .......... .......... .......... 50% 17.6M 4s 2024-09-12T12:57:43.324 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 85300K .......... .......... .......... .......... .......... 50% 19.8M 4s 2024-09-12T12:57:43.327 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 85350K .......... .......... .......... .......... .......... 50% 16.8M 4s 2024-09-12T12:57:43.330 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 85400K .......... .......... .......... .......... .......... 50% 16.6M 4s 2024-09-12T12:57:43.331 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 85450K .......... .......... .......... .......... .......... 50% 28.4M 4s 2024-09-12T12:57:43.335 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 85500K .......... .......... .......... .......... .......... 50% 14.7M 4s 2024-09-12T12:57:43.341 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 85550K .......... .......... .......... .......... .......... 50% 7.73M 4s 2024-09-12T12:57:43.346 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 85600K .......... .......... .......... .......... .......... 50% 9.21M 4s 2024-09-12T12:57:43.350 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 85650K .......... .......... .......... .......... .......... 50% 14.3M 4s 2024-09-12T12:57:43.357 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 85700K .......... .......... .......... .......... .......... 50% 7.43M 4s 2024-09-12T12:57:43.358 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 85750K .......... .......... .......... .......... .......... 50% 40.6M 4s 2024-09-12T12:57:43.361 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 85800K .......... .......... .......... .......... .......... 50% 15.8M 4s 2024-09-12T12:57:43.362 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 85850K .......... .......... .......... .......... .......... 50% 30.6M 4s 2024-09-12T12:57:43.364 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 85900K .......... .......... .......... .......... .......... 50% 22.0M 4s 2024-09-12T12:57:43.368 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 85950K .......... .......... .......... .......... .......... 51% 12.7M 4s 2024-09-12T12:57:43.370 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 86000K .......... .......... .......... .......... .......... 51% 23.4M 4s 2024-09-12T12:57:43.374 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 86050K .......... .......... .......... .......... .......... 51% 14.3M 4s 2024-09-12T12:57:43.375 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 86100K .......... .......... .......... .......... .......... 51% 29.1M 4s 2024-09-12T12:57:43.379 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 86150K .......... .......... .......... .......... .......... 51% 17.3M 4s 2024-09-12T12:57:43.383 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 86200K .......... .......... .......... .......... .......... 51% 12.0M 4s 2024-09-12T12:57:43.383 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 86250K .......... .......... .......... .......... .......... 51% 42.2M 4s 2024-09-12T12:57:43.386 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 86300K .......... .......... .......... .......... .......... 51% 19.0M 4s 2024-09-12T12:57:43.390 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 86350K .......... .......... .......... .......... .......... 51% 14.6M 4s 2024-09-12T12:57:43.392 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 86400K .......... .......... .......... .......... .......... 51% 18.7M 4s 2024-09-12T12:57:43.395 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 86450K .......... .......... .......... .......... .......... 51% 18.6M 4s 2024-09-12T12:57:43.400 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 86500K .......... .......... .......... .......... .......... 51% 9.82M 4s 2024-09-12T12:57:43.403 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 86550K .......... .......... .......... .......... .......... 51% 15.0M 4s 2024-09-12T12:57:43.406 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 86600K .......... .......... .......... .......... .......... 51% 16.5M 4s 2024-09-12T12:57:43.408 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 86650K .......... .......... .......... .......... .......... 51% 24.4M 4s 2024-09-12T12:57:43.412 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 86700K .......... .......... .......... .......... .......... 51% 11.6M 4s 2024-09-12T12:57:43.415 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 86750K .......... .......... .......... .......... .......... 51% 15.7M 4s 2024-09-12T12:57:43.420 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 86800K .......... .......... .......... .......... .......... 51% 10.4M 4s 2024-09-12T12:57:43.422 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 86850K .......... .......... .......... .......... .......... 51% 21.4M 4s 2024-09-12T12:57:43.425 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 86900K .......... .......... .......... .......... .......... 51% 17.9M 4s 2024-09-12T12:57:43.427 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 86950K .......... .......... .......... .......... .......... 51% 25.5M 4s 2024-09-12T12:57:43.428 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 87000K .......... .......... .......... .......... .......... 51% 35.8M 4s 2024-09-12T12:57:43.431 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 87050K .......... .......... .......... .......... .......... 51% 15.9M 4s 2024-09-12T12:57:43.433 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 87100K .......... .......... .......... .......... .......... 51% 21.0M 4s 2024-09-12T12:57:43.437 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 87150K .......... .......... .......... .......... .......... 51% 14.7M 4s 2024-09-12T12:57:43.439 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 87200K .......... .......... .......... .......... .......... 51% 23.3M 4s 2024-09-12T12:57:43.443 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 87250K .......... .......... .......... .......... .......... 51% 14.0M 4s 2024-09-12T12:57:43.446 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 87300K .......... .......... .......... .......... .......... 51% 12.6M 4s 2024-09-12T12:57:43.448 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 87350K .......... .......... .......... .......... .......... 51% 26.5M 4s 2024-09-12T12:57:43.451 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 87400K .......... .......... .......... .......... .......... 51% 19.2M 4s 2024-09-12T12:57:43.453 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 87450K .......... .......... .......... .......... .......... 51% 21.7M 4s 2024-09-12T12:57:43.455 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 87500K .......... .......... .......... .......... .......... 51% 19.6M 4s 2024-09-12T12:57:43.459 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 87550K .......... .......... .......... .......... .......... 51% 12.6M 4s 2024-09-12T12:57:43.463 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 87600K .......... .......... .......... .......... .......... 51% 12.4M 4s 2024-09-12T12:57:43.467 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 87650K .......... .......... .......... .......... .......... 52% 13.0M 4s 2024-09-12T12:57:43.471 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 87700K .......... .......... .......... .......... .......... 52% 13.1M 4s 2024-09-12T12:57:43.473 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 87750K .......... .......... .......... .......... .......... 52% 19.9M 4s 2024-09-12T12:57:43.479 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 87800K .......... .......... .......... .......... .......... 52% 8.61M 4s 2024-09-12T12:57:43.481 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 87850K .......... .......... .......... .......... .......... 52% 20.7M 4s 2024-09-12T12:57:43.488 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 87900K .......... .......... .......... .......... .......... 52% 6.71M 4s 2024-09-12T12:57:43.589 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 87950K .......... .......... .......... .......... .......... 52% 743K 4s 2024-09-12T12:57:43.589 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 88000K .......... .......... .......... .......... .......... 52% 152M 4s 2024-09-12T12:57:43.589 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 88050K .......... .......... .......... .......... .......... 52% 171M 4s 2024-09-12T12:57:43.589 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 88100K .......... .......... .......... .......... .......... 52% 138M 4s 2024-09-12T12:57:43.589 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 88150K .......... .......... .......... .......... .......... 52% 163M 4s 2024-09-12T12:57:43.590 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 88200K .......... .......... .......... .......... .......... 52% 138M 4s 2024-09-12T12:57:43.590 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 88250K .......... .......... .......... .......... .......... 52% 151M 4s 2024-09-12T12:57:43.590 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 88300K .......... .......... .......... .......... .......... 52% 160M 4s 2024-09-12T12:57:43.590 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 88350K .......... .......... .......... .......... .......... 52% 108M 4s 2024-09-12T12:57:43.590 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 88400K .......... .......... .......... .......... .......... 52% 179M 4s 2024-09-12T12:57:43.590 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 88450K .......... .......... .......... .......... .......... 52% 180M 4s 2024-09-12T12:57:43.590 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 88500K .......... .......... .......... .......... .......... 52% 149M 4s 2024-09-12T12:57:43.590 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 88550K .......... .......... .......... .......... .......... 52% 139M 4s 2024-09-12T12:57:43.590 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 88600K .......... .......... .......... .......... .......... 52% 138M 4s 2024-09-12T12:57:43.590 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 88650K .......... .......... .......... .......... .......... 52% 169M 4s 2024-09-12T12:57:43.590 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 88700K .......... .......... .......... .......... .......... 52% 198M 4s 2024-09-12T12:57:43.590 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 88750K .......... .......... .......... .......... .......... 52% 142M 4s 2024-09-12T12:57:43.590 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 88800K .......... .......... .......... .......... .......... 52% 172M 4s 2024-09-12T12:57:43.590 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 88850K .......... .......... .......... .......... .......... 52% 164M 4s 2024-09-12T12:57:43.591 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 88900K .......... .......... .......... .......... .......... 52% 135M 4s 2024-09-12T12:57:43.591 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 88950K .......... .......... .......... .......... .......... 52% 161M 4s 2024-09-12T12:57:43.591 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 89000K .......... .......... .......... .......... .......... 52% 175M 4s 2024-09-12T12:57:43.591 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 89050K .......... .......... .......... .......... .......... 52% 160M 4s 2024-09-12T12:57:43.591 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 89100K .......... .......... .......... .......... .......... 52% 120M 4s 2024-09-12T12:57:43.591 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 89150K .......... .......... .......... .......... .......... 52% 125M 4s 2024-09-12T12:57:43.591 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 89200K .......... .......... .......... .......... .......... 52% 146M 4s 2024-09-12T12:57:43.591 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 89250K .......... .......... .......... .......... .......... 52% 161M 4s 2024-09-12T12:57:43.591 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 89300K .......... .......... .......... .......... .......... 53% 16.3M 4s 2024-09-12T12:57:43.591 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 89350K .......... .......... .......... .......... .......... 53% 19.7M 4s 2024-09-12T12:57:43.591 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 89400K .......... .......... .......... .......... .......... 53% 10.2M 4s 2024-09-12T12:57:43.591 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 89450K .......... .......... .......... .......... .......... 53% 42.0M 4s 2024-09-12T12:57:43.591 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 89500K .......... .......... .......... .......... .......... 53% 17.4M 4s 2024-09-12T12:57:43.591 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 89550K .......... .......... .......... .......... .......... 53% 8.39M 4s 2024-09-12T12:57:43.593 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 89600K .......... .......... .......... .......... .......... 53% 9.76M 4s 2024-09-12T12:57:43.593 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 89650K .......... .......... .......... .......... .......... 53% 15.1M 4s 2024-09-12T12:57:43.598 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 89700K .......... .......... .......... .......... .......... 53% 8.98M 4s 2024-09-12T12:57:43.601 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 89750K .......... .......... .......... .......... .......... 53% 15.1M 4s 2024-09-12T12:57:43.609 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 89800K .......... .......... .......... .......... .......... 53% 6.53M 4s 2024-09-12T12:57:43.612 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 89850K .......... .......... .......... .......... .......... 53% 21.0M 4s 2024-09-12T12:57:43.612 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 89900K .......... .......... .......... .......... .......... 53% 43.8M 4s 2024-09-12T12:57:43.614 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 89950K .......... .......... .......... .......... .......... 53% 20.1M 4s 2024-09-12T12:57:43.616 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 90000K .......... .......... .......... .......... .......... 53% 28.3M 4s 2024-09-12T12:57:43.619 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 90050K .......... .......... .......... .......... .......... 53% 21.5M 4s 2024-09-12T12:57:43.622 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 90100K .......... .......... .......... .......... .......... 53% 14.6M 4s 2024-09-12T12:57:43.625 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 90150K .......... .......... .......... .......... .......... 53% 13.4M 4s 2024-09-12T12:57:43.628 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 90200K .......... .......... .......... .......... .......... 53% 21.0M 4s 2024-09-12T12:57:43.631 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 90250K .......... .......... .......... .......... .......... 53% 17.1M 4s 2024-09-12T12:57:43.633 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 90300K .......... .......... .......... .......... .......... 53% 17.5M 4s 2024-09-12T12:57:43.637 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 90350K .......... .......... .......... .......... .......... 53% 12.9M 4s 2024-09-12T12:57:43.641 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 90400K .......... .......... .......... .......... .......... 53% 14.1M 4s 2024-09-12T12:57:43.643 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 90450K .......... .......... .......... .......... .......... 53% 19.7M 4s 2024-09-12T12:57:43.648 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 90500K .......... .......... .......... .......... .......... 53% 12.1M 4s 2024-09-12T12:57:43.652 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 90550K .......... .......... .......... .......... .......... 53% 9.95M 4s 2024-09-12T12:57:43.656 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 90600K .......... .......... .......... .......... .......... 53% 12.9M 4s 2024-09-12T12:57:43.658 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 90650K .......... .......... .......... .......... .......... 53% 26.8M 4s 2024-09-12T12:57:43.660 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 90700K .......... .......... .......... .......... .......... 53% 17.3M 4s 2024-09-12T12:57:43.664 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 90750K .......... .......... .......... .......... .......... 53% 15.4M 4s 2024-09-12T12:57:43.668 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 90800K .......... .......... .......... .......... .......... 53% 11.9M 4s 2024-09-12T12:57:43.670 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 90850K .......... .......... .......... .......... .......... 53% 19.4M 4s 2024-09-12T12:57:43.674 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 90900K .......... .......... .......... .......... .......... 53% 16.0M 4s 2024-09-12T12:57:43.675 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 90950K .......... .......... .......... .......... .......... 53% 25.1M 4s 2024-09-12T12:57:43.677 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 91000K .......... .......... .......... .......... .......... 54% 22.2M 4s 2024-09-12T12:57:43.681 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 91050K .......... .......... .......... .......... .......... 54% 16.0M 4s 2024-09-12T12:57:43.684 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 91100K .......... .......... .......... .......... .......... 54% 13.4M 4s 2024-09-12T12:57:43.687 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 91150K .......... .......... .......... .......... .......... 54% 16.0M 4s 2024-09-12T12:57:43.692 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 91200K .......... .......... .......... .......... .......... 54% 13.3M 4s 2024-09-12T12:57:43.694 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 91250K .......... .......... .......... .......... .......... 54% 17.6M 4s 2024-09-12T12:57:43.698 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 91300K .......... .......... .......... .......... .......... 54% 10.6M 4s 2024-09-12T12:57:43.705 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 91350K .......... .......... .......... .......... .......... 54% 7.52M 4s 2024-09-12T12:57:43.708 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 91400K .......... .......... .......... .......... .......... 54% 15.2M 4s 2024-09-12T12:57:43.712 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 91450K .......... .......... .......... .......... .......... 54% 13.9M 4s 2024-09-12T12:57:43.715 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 91500K .......... .......... .......... .......... .......... 54% 15.6M 4s 2024-09-12T12:57:43.720 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 91550K .......... .......... .......... .......... .......... 54% 9.28M 4s 2024-09-12T12:57:43.724 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 91600K .......... .......... .......... .......... .......... 54% 11.9M 4s 2024-09-12T12:57:43.728 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 91650K .......... .......... .......... .......... .......... 54% 13.0M 3s 2024-09-12T12:57:43.730 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 91700K .......... .......... .......... .......... .......... 54% 18.8M 3s 2024-09-12T12:57:43.733 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 91750K .......... .......... .......... .......... .......... 54% 23.9M 3s 2024-09-12T12:57:43.734 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 91800K .......... .......... .......... .......... .......... 54% 26.6M 3s 2024-09-12T12:57:43.736 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 91850K .......... .......... .......... .......... .......... 54% 39.3M 3s 2024-09-12T12:57:43.738 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 91900K .......... .......... .......... .......... .......... 54% 24.4M 3s 2024-09-12T12:57:43.742 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 91950K .......... .......... .......... .......... .......... 54% 11.1M 3s 2024-09-12T12:57:43.745 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 92000K .......... .......... .......... .......... .......... 54% 19.5M 3s 2024-09-12T12:57:43.748 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 92050K .......... .......... .......... .......... .......... 54% 16.7M 3s 2024-09-12T12:57:43.751 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 92100K .......... .......... .......... .......... .......... 54% 13.3M 3s 2024-09-12T12:57:43.755 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 92150K .......... .......... .......... .......... .......... 54% 12.1M 3s 2024-09-12T12:57:43.758 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 92200K .......... .......... .......... .......... .......... 54% 13.8M 3s 2024-09-12T12:57:43.761 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 92250K .......... .......... .......... .......... .......... 54% 16.6M 3s 2024-09-12T12:57:43.765 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 92300K .......... .......... .......... .......... .......... 54% 12.3M 3s 2024-09-12T12:57:43.772 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 92350K .......... .......... .......... .......... .......... 54% 7.19M 3s 2024-09-12T12:57:43.775 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 92400K .......... .......... .......... .......... .......... 54% 20.2M 3s 2024-09-12T12:57:43.778 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 92450K .......... .......... .......... .......... .......... 54% 14.1M 3s 2024-09-12T12:57:43.780 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 92500K .......... .......... .......... .......... .......... 54% 23.2M 3s 2024-09-12T12:57:43.783 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 92550K .......... .......... .......... .......... .......... 54% 17.7M 3s 2024-09-12T12:57:43.786 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 92600K .......... .......... .......... .......... .......... 54% 19.4M 3s 2024-09-12T12:57:43.788 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 92650K .......... .......... .......... .......... .......... 54% 21.3M 3s 2024-09-12T12:57:43.791 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 92700K .......... .......... .......... .......... .......... 55% 17.3M 3s 2024-09-12T12:57:43.794 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 92750K .......... .......... .......... .......... .......... 55% 15.4M 3s 2024-09-12T12:57:43.797 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 92800K .......... .......... .......... .......... .......... 55% 18.2M 3s 2024-09-12T12:57:43.800 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 92850K .......... .......... .......... .......... .......... 55% 16.5M 3s 2024-09-12T12:57:43.807 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 92900K .......... .......... .......... .......... .......... 55% 7.19M 3s 2024-09-12T12:57:43.807 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 92950K .......... .......... .......... .......... .......... 55% 108M 3s 2024-09-12T12:57:43.811 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 93000K .......... .......... .......... .......... .......... 55% 11.3M 3s 2024-09-12T12:57:43.814 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 93050K .......... .......... .......... .......... .......... 55% 21.0M 3s 2024-09-12T12:57:43.816 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 93100K .......... .......... .......... .......... .......... 55% 23.1M 3s 2024-09-12T12:57:43.820 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 93150K .......... .......... .......... .......... .......... 55% 10.1M 3s 2024-09-12T12:57:43.824 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 93200K .......... .......... .......... .......... .......... 55% 15.9M 3s 2024-09-12T12:57:43.826 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 93250K .......... .......... .......... .......... .......... 55% 20.4M 3s 2024-09-12T12:57:43.828 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 93300K .......... .......... .......... .......... .......... 55% 20.1M 3s 2024-09-12T12:57:43.831 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 93350K .......... .......... .......... .......... .......... 55% 17.1M 3s 2024-09-12T12:57:43.834 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 93400K .......... .......... .......... .......... .......... 55% 19.5M 3s 2024-09-12T12:57:43.837 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 93450K .......... .......... .......... .......... .......... 55% 14.1M 3s 2024-09-12T12:57:43.841 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 93500K .......... .......... .......... .......... .......... 55% 13.2M 3s 2024-09-12T12:57:43.845 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 93550K .......... .......... .......... .......... .......... 55% 11.7M 3s 2024-09-12T12:57:43.847 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 93600K .......... .......... .......... .......... .......... 55% 24.2M 3s 2024-09-12T12:57:43.848 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 93650K .......... .......... .......... .......... .......... 55% 38.7M 3s 2024-09-12T12:57:43.850 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 93700K .......... .......... .......... .......... .......... 55% 25.1M 3s 2024-09-12T12:57:43.852 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 93750K .......... .......... .......... .......... .......... 55% 30.3M 3s 2024-09-12T12:57:43.853 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 93800K .......... .......... .......... .......... .......... 55% 34.6M 3s 2024-09-12T12:57:43.855 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 93850K .......... .......... .......... .......... .......... 55% 27.3M 3s 2024-09-12T12:57:43.857 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 93900K .......... .......... .......... .......... .......... 55% 30.8M 3s 2024-09-12T12:57:43.859 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 93950K .......... .......... .......... .......... .......... 55% 23.1M 3s 2024-09-12T12:57:43.860 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 94000K .......... .......... .......... .......... .......... 55% 31.7M 3s 2024-09-12T12:57:43.865 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 94050K .......... .......... .......... .......... .......... 55% 22.7M 3s 2024-09-12T12:57:43.865 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 94100K .......... .......... .......... .......... .......... 55% 22.5M 3s 2024-09-12T12:57:43.866 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 94150K .......... .......... .......... .......... .......... 55% 40.4M 3s 2024-09-12T12:57:43.867 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 94200K .......... .......... .......... .......... .......... 55% 31.2M 3s 2024-09-12T12:57:43.870 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 94250K .......... .......... .......... .......... .......... 55% 22.9M 3s 2024-09-12T12:57:43.871 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 94300K .......... .......... .......... .......... .......... 55% 34.4M 3s 2024-09-12T12:57:43.873 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 94350K .......... .......... .......... .......... .......... 56% 25.7M 3s 2024-09-12T12:57:43.874 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 94400K .......... .......... .......... .......... .......... 56% 36.1M 3s 2024-09-12T12:57:43.876 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 94450K .......... .......... .......... .......... .......... 56% 23.6M 3s 2024-09-12T12:57:43.878 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 94500K .......... .......... .......... .......... .......... 56% 33.0M 3s 2024-09-12T12:57:43.879 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 94550K .......... .......... .......... .......... .......... 56% 32.4M 3s 2024-09-12T12:57:43.881 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 94600K .......... .......... .......... .......... .......... 56% 30.1M 3s 2024-09-12T12:57:43.883 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 94650K .......... .......... .......... .......... .......... 56% 25.5M 3s 2024-09-12T12:57:43.884 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 94700K .......... .......... .......... .......... .......... 56% 34.2M 3s 2024-09-12T12:57:43.887 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 94750K .......... .......... .......... .......... .......... 56% 22.4M 3s 2024-09-12T12:57:43.888 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 94800K .......... .......... .......... .......... .......... 56% 30.9M 3s 2024-09-12T12:57:43.890 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 94850K .......... .......... .......... .......... .......... 56% 25.7M 3s 2024-09-12T12:57:43.891 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 94900K .......... .......... .......... .......... .......... 56% 41.6M 3s 2024-09-12T12:57:43.893 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 94950K .......... .......... .......... .......... .......... 56% 27.6M 3s 2024-09-12T12:57:43.895 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 95000K .......... .......... .......... .......... .......... 56% 21.3M 3s 2024-09-12T12:57:43.896 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 95050K .......... .......... .......... .......... .......... 56% 38.3M 3s 2024-09-12T12:57:43.898 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 95100K .......... .......... .......... .......... .......... 56% 39.0M 3s 2024-09-12T12:57:43.900 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 95150K .......... .......... .......... .......... .......... 56% 21.6M 3s 2024-09-12T12:57:43.902 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 95200K .......... .......... .......... .......... .......... 56% 24.2M 3s 2024-09-12T12:57:43.904 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 95250K .......... .......... .......... .......... .......... 56% 21.7M 3s 2024-09-12T12:57:43.906 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 95300K .......... .......... .......... .......... .......... 56% 24.1M 3s 2024-09-12T12:57:43.909 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 95350K .......... .......... .......... .......... .......... 56% 16.7M 3s 2024-09-12T12:57:43.910 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 95400K .......... .......... .......... .......... .......... 56% 35.6M 3s 2024-09-12T12:57:43.914 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 95450K .......... .......... .......... .......... .......... 56% 14.4M 3s 2024-09-12T12:57:43.916 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 95500K .......... .......... .......... .......... .......... 56% 28.8M 3s 2024-09-12T12:57:43.919 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 95550K .......... .......... .......... .......... .......... 56% 15.7M 3s 2024-09-12T12:57:43.922 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 95600K .......... .......... .......... .......... .......... 56% 17.5M 3s 2024-09-12T12:57:43.924 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 95650K .......... .......... .......... .......... .......... 56% 20.9M 3s 2024-09-12T12:57:43.927 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 95700K .......... .......... .......... .......... .......... 56% 17.8M 3s 2024-09-12T12:57:43.928 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 95750K .......... .......... .......... .......... .......... 56% 36.5M 3s 2024-09-12T12:57:43.932 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 95800K .......... .......... .......... .......... .......... 56% 15.4M 3s 2024-09-12T12:57:43.934 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 95850K .......... .......... .......... .......... .......... 56% 15.9M 3s 2024-09-12T12:57:43.936 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 95900K .......... .......... .......... .......... .......... 56% 30.4M 3s 2024-09-12T12:57:43.939 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 95950K .......... .......... .......... .......... .......... 56% 16.5M 3s 2024-09-12T12:57:43.943 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 96000K .......... .......... .......... .......... .......... 56% 12.4M 3s 2024-09-12T12:57:43.944 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 96050K .......... .......... .......... .......... .......... 57% 31.3M 3s 2024-09-12T12:57:43.948 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 96100K .......... .......... .......... .......... .......... 57% 12.9M 3s 2024-09-12T12:57:43.950 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 96150K .......... .......... .......... .......... .......... 57% 25.3M 3s 2024-09-12T12:57:43.953 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 96200K .......... .......... .......... .......... .......... 57% 20.2M 3s 2024-09-12T12:57:43.954 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 96250K .......... .......... .......... .......... .......... 57% 35.1M 3s 2024-09-12T12:57:43.955 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 96300K .......... .......... .......... .......... .......... 57% 24.6M 3s 2024-09-12T12:57:43.958 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 96350K .......... .......... .......... .......... .......... 57% 21.3M 3s 2024-09-12T12:57:43.960 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 96400K .......... .......... .......... .......... .......... 57% 23.1M 3s 2024-09-12T12:57:43.961 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 96450K .......... .......... .......... .......... .......... 57% 43.7M 3s 2024-09-12T12:57:43.964 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 96500K .......... .......... .......... .......... .......... 57% 21.9M 3s 2024-09-12T12:57:43.965 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 96550K .......... .......... .......... .......... .......... 57% 40.5M 3s 2024-09-12T12:57:43.967 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 96600K .......... .......... .......... .......... .......... 57% 25.3M 3s 2024-09-12T12:57:43.968 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 96650K .......... .......... .......... .......... .......... 57% 42.1M 3s 2024-09-12T12:57:43.970 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 96700K .......... .......... .......... .......... .......... 57% 26.7M 3s 2024-09-12T12:57:43.972 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 96750K .......... .......... .......... .......... .......... 57% 21.7M 3s 2024-09-12T12:57:43.974 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 96800K .......... .......... .......... .......... .......... 57% 25.1M 3s 2024-09-12T12:57:43.975 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 96850K .......... .......... .......... .......... .......... 57% 36.0M 3s 2024-09-12T12:57:43.978 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 96900K .......... .......... .......... .......... .......... 57% 24.0M 3s 2024-09-12T12:57:43.978 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 96950K .......... .......... .......... .......... .......... 57% 44.3M 3s 2024-09-12T12:57:43.980 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 97000K .......... .......... .......... .......... .......... 57% 24.6M 3s 2024-09-12T12:57:43.982 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 97050K .......... .......... .......... .......... .......... 57% 40.0M 3s 2024-09-12T12:57:43.984 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 97100K .......... .......... .......... .......... .......... 57% 24.2M 3s 2024-09-12T12:57:43.986 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 97150K .......... .......... .......... .......... .......... 57% 22.9M 3s 2024-09-12T12:57:43.987 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 97200K .......... .......... .......... .......... .......... 57% 38.1M 3s 2024-09-12T12:57:43.990 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 97250K .......... .......... .......... .......... .......... 57% 23.6M 3s 2024-09-12T12:57:43.991 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 97300K .......... .......... .......... .......... .......... 57% 29.1M 3s 2024-09-12T12:57:43.992 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 97350K .......... .......... .......... .......... .......... 57% 34.6M 3s 2024-09-12T12:57:43.993 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 97400K .......... .......... .......... .......... .......... 57% 41.4M 3s 2024-09-12T12:57:43.995 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 97450K .......... .......... .......... .......... .......... 57% 23.6M 3s 2024-09-12T12:57:43.998 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 97500K .......... .......... .......... .......... .......... 57% 22.0M 3s 2024-09-12T12:57:44.002 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 97550K .......... .......... .......... .......... .......... 57% 12.8M 3s 2024-09-12T12:57:44.003 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 97600K .......... .......... .......... .......... .......... 57% 29.7M 3s 2024-09-12T12:57:44.006 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 97650K .......... .......... .......... .......... .......... 57% 19.4M 3s 2024-09-12T12:57:44.007 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 97700K .......... .......... .......... .......... .......... 57% 34.5M 3s 2024-09-12T12:57:44.010 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 97750K .......... .......... .......... .......... .......... 58% 15.0M 3s 2024-09-12T12:57:44.012 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 97800K .......... .......... .......... .......... .......... 58% 28.4M 3s 2024-09-12T12:57:44.014 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 97850K .......... .......... .......... .......... .......... 58% 20.3M 3s 2024-09-12T12:57:44.016 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 97900K .......... .......... .......... .......... .......... 58% 28.5M 3s 2024-09-12T12:57:44.019 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 97950K .......... .......... .......... .......... .......... 58% 17.5M 3s 2024-09-12T12:57:44.021 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 98000K .......... .......... .......... .......... .......... 58% 18.6M 3s 2024-09-12T12:57:44.023 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 98050K .......... .......... .......... .......... .......... 58% 35.4M 3s 2024-09-12T12:57:44.026 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 98100K .......... .......... .......... .......... .......... 58% 17.3M 3s 2024-09-12T12:57:44.028 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 98150K .......... .......... .......... .......... .......... 58% 22.0M 3s 2024-09-12T12:57:44.030 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 98200K .......... .......... .......... .......... .......... 58% 27.9M 3s 2024-09-12T12:57:44.032 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 98250K .......... .......... .......... .......... .......... 58% 19.2M 3s 2024-09-12T12:57:44.034 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 98300K .......... .......... .......... .......... .......... 58% 29.1M 3s 2024-09-12T12:57:44.038 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 98350K .......... .......... .......... .......... .......... 58% 13.0M 3s 2024-09-12T12:57:44.040 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 98400K .......... .......... .......... .......... .......... 58% 22.3M 3s 2024-09-12T12:57:44.042 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 98450K .......... .......... .......... .......... .......... 58% 23.9M 3s 2024-09-12T12:57:44.045 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 98500K .......... .......... .......... .......... .......... 58% 14.7M 3s 2024-09-12T12:57:44.047 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 98550K .......... .......... .......... .......... .......... 58% 24.5M 3s 2024-09-12T12:57:44.049 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 98600K .......... .......... .......... .......... .......... 58% 22.3M 3s 2024-09-12T12:57:44.054 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 98650K .......... .......... .......... .......... .......... 58% 23.9M 3s 2024-09-12T12:57:44.054 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 98700K .......... .......... .......... .......... .......... 58% 39.7M 3s 2024-09-12T12:57:44.055 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 98750K .......... .......... .......... .......... .......... 58% 22.5M 3s 2024-09-12T12:57:44.057 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 98800K .......... .......... .......... .......... .......... 58% 24.8M 3s 2024-09-12T12:57:44.058 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 98850K .......... .......... .......... .......... .......... 58% 44.4M 3s 2024-09-12T12:57:44.060 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 98900K .......... .......... .......... .......... .......... 58% 23.8M 3s 2024-09-12T12:57:44.061 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 98950K .......... .......... .......... .......... .......... 58% 44.6M 3s 2024-09-12T12:57:44.063 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 99000K .......... .......... .......... .......... .......... 58% 28.5M 3s 2024-09-12T12:57:44.065 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 99050K .......... .......... .......... .......... .......... 58% 19.7M 3s 2024-09-12T12:57:44.067 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 99100K .......... .......... .......... .......... .......... 58% 35.6M 3s 2024-09-12T12:57:44.069 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 99150K .......... .......... .......... .......... .......... 58% 23.4M 3s 2024-09-12T12:57:44.070 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 99200K .......... .......... .......... .......... .......... 58% 31.5M 3s 2024-09-12T12:57:44.072 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 99250K .......... .......... .......... .......... .......... 58% 28.3M 3s 2024-09-12T12:57:44.074 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 99300K .......... .......... .......... .......... .......... 58% 27.3M 3s 2024-09-12T12:57:44.075 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 99350K .......... .......... .......... .......... .......... 58% 36.8M 3s 2024-09-12T12:57:44.077 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 99400K .......... .......... .......... .......... .......... 58% 23.5M 3s 2024-09-12T12:57:44.078 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 99450K .......... .......... .......... .......... .......... 59% 44.2M 3s 2024-09-12T12:57:44.080 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 99500K .......... .......... .......... .......... .......... 59% 24.7M 3s 2024-09-12T12:57:44.083 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 99550K .......... .......... .......... .......... .......... 59% 24.1M 3s 2024-09-12T12:57:44.085 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 99600K .......... .......... .......... .......... .......... 59% 25.8M 3s 2024-09-12T12:57:44.085 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 99650K .......... .......... .......... .......... .......... 59% 39.3M 3s 2024-09-12T12:57:44.088 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 99700K .......... .......... .......... .......... .......... 59% 23.0M 3s 2024-09-12T12:57:44.089 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 99750K .......... .......... .......... .......... .......... 59% 37.1M 3s 2024-09-12T12:57:44.090 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 99800K .......... .......... .......... .......... .......... 59% 29.3M 3s 2024-09-12T12:57:44.092 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 99850K .......... .......... .......... .......... .......... 59% 31.2M 3s 2024-09-12T12:57:44.094 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 99900K .......... .......... .......... .......... .......... 59% 21.5M 3s 2024-09-12T12:57:44.097 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 99950K .......... .......... .......... .......... .......... 59% 21.6M 3s 2024-09-12T12:57:44.098 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:100000K .......... .......... .......... .......... .......... 59% 48.2M 3s 2024-09-12T12:57:44.100 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:100050K .......... .......... .......... .......... .......... 59% 21.6M 3s 2024-09-12T12:57:44.101 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:100100K .......... .......... .......... .......... .......... 59% 35.4M 3s 2024-09-12T12:57:44.103 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:100150K .......... .......... .......... .......... .......... 59% 26.9M 3s 2024-09-12T12:57:44.104 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:100200K .......... .......... .......... .......... .......... 59% 41.7M 3s 2024-09-12T12:57:44.107 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:100250K .......... .......... .......... .......... .......... 59% 22.7M 3s 2024-09-12T12:57:44.108 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:100300K .......... .......... .......... .......... .......... 59% 24.4M 3s 2024-09-12T12:57:44.111 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:100350K .......... .......... .......... .......... .......... 59% 22.8M 3s 2024-09-12T12:57:44.112 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:100400K .......... .......... .......... .......... .......... 59% 37.2M 3s 2024-09-12T12:57:44.114 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:100450K .......... .......... .......... .......... .......... 59% 24.3M 3s 2024-09-12T12:57:44.115 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:100500K .......... .......... .......... .......... .......... 59% 42.4M 3s 2024-09-12T12:57:44.117 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:100550K .......... .......... .......... .......... .......... 59% 25.1M 3s 2024-09-12T12:57:44.118 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:100600K .......... .......... .......... .......... .......... 59% 32.7M 3s 2024-09-12T12:57:44.120 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:100650K .......... .......... .......... .......... .......... 59% 29.6M 3s 2024-09-12T12:57:44.121 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:100700K .......... .......... .......... .......... .......... 59% 37.3M 3s 2024-09-12T12:57:44.123 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:100750K .......... .......... .......... .......... .......... 59% 26.0M 3s 2024-09-12T12:57:44.126 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:100800K .......... .......... .......... .......... .......... 59% 22.8M 3s 2024-09-12T12:57:44.128 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:100850K .......... .......... .......... .......... .......... 59% 24.1M 3s 2024-09-12T12:57:44.129 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:100900K .......... .......... .......... .......... .......... 59% 36.8M 3s 2024-09-12T12:57:44.131 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:100950K .......... .......... .......... .......... .......... 59% 24.2M 3s 2024-09-12T12:57:44.132 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:101000K .......... .......... .......... .......... .......... 59% 49.1M 3s 2024-09-12T12:57:44.134 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:101050K .......... .......... .......... .......... .......... 59% 28.5M 3s 2024-09-12T12:57:44.135 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:101100K .......... .......... .......... .......... .......... 60% 30.7M 3s 2024-09-12T12:57:44.137 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:101150K .......... .......... .......... .......... .......... 60% 22.0M 3s 2024-09-12T12:57:44.140 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:101200K .......... .......... .......... .......... .......... 60% 18.3M 3s 2024-09-12T12:57:44.141 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:101250K .......... .......... .......... .......... .......... 60% 60.5M 3s 2024-09-12T12:57:44.143 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:101300K .......... .......... .......... .......... .......... 60% 24.9M 3s 2024-09-12T12:57:44.145 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:101350K .......... .......... .......... .......... .......... 60% 25.5M 3s 2024-09-12T12:57:44.146 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:101400K .......... .......... .......... .......... .......... 60% 39.1M 3s 2024-09-12T12:57:44.148 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:101450K .......... .......... .......... .......... .......... 60% 22.4M 3s 2024-09-12T12:57:44.149 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:101500K .......... .......... .......... .......... .......... 60% 50.6M 3s 2024-09-12T12:57:44.151 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:101550K .......... .......... .......... .......... .......... 60% 23.7M 3s 2024-09-12T12:57:44.153 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:101600K .......... .......... .......... .......... .......... 60% 22.6M 3s 2024-09-12T12:57:44.155 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:101650K .......... .......... .......... .......... .......... 60% 37.3M 3s 2024-09-12T12:57:44.157 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:101700K .......... .......... .......... .......... .......... 60% 24.6M 3s 2024-09-12T12:57:44.157 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:101750K .......... .......... .......... .......... .......... 60% 49.3M 3s 2024-09-12T12:57:44.160 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:101800K .......... .......... .......... .......... .......... 60% 22.3M 3s 2024-09-12T12:57:44.161 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:101850K .......... .......... .......... .......... .......... 60% 29.8M 3s 2024-09-12T12:57:44.164 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:101900K .......... .......... .......... .......... .......... 60% 20.5M 3s 2024-09-12T12:57:44.224 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:101950K .......... .......... .......... .......... .......... 60% 1.57M 3s 2024-09-12T12:57:44.224 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:102000K .......... .......... .......... .......... .......... 60% 173M 3s 2024-09-12T12:57:44.224 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:102050K .......... .......... .......... .......... .......... 60% 146M 3s 2024-09-12T12:57:44.225 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:102100K .......... .......... .......... .......... .......... 60% 159M 3s 2024-09-12T12:57:44.225 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:102150K .......... .......... .......... .......... .......... 60% 152M 3s 2024-09-12T12:57:44.225 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:102200K .......... .......... .......... .......... .......... 60% 194M 3s 2024-09-12T12:57:44.225 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:102250K .......... .......... .......... .......... .......... 60% 193M 3s 2024-09-12T12:57:44.225 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:102300K .......... .......... .......... .......... .......... 60% 136M 3s 2024-09-12T12:57:44.225 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:102350K .......... .......... .......... .......... .......... 60% 129M 3s 2024-09-12T12:57:44.225 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:102400K .......... .......... .......... .......... .......... 60% 142M 3s 2024-09-12T12:57:44.225 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:102450K .......... .......... .......... .......... .......... 60% 165M 3s 2024-09-12T12:57:44.225 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:102500K .......... .......... .......... .......... .......... 60% 182M 3s 2024-09-12T12:57:44.225 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:102550K .......... .......... .......... .......... .......... 60% 166M 3s 2024-09-12T12:57:44.225 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:102600K .......... .......... .......... .......... .......... 60% 171M 3s 2024-09-12T12:57:44.225 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:102650K .......... .......... .......... .......... .......... 60% 148M 3s 2024-09-12T12:57:44.225 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:102700K .......... .......... .......... .......... .......... 60% 171M 3s 2024-09-12T12:57:44.226 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:102750K .......... .......... .......... .......... .......... 60% 133M 3s 2024-09-12T12:57:44.226 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:102800K .......... .......... .......... .......... .......... 61% 182M 3s 2024-09-12T12:57:44.226 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:102850K .......... .......... .......... .......... .......... 61% 191M 3s 2024-09-12T12:57:44.226 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:102900K .......... .......... .......... .......... .......... 61% 175M 3s 2024-09-12T12:57:44.226 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:102950K .......... .......... .......... .......... .......... 61% 111M 3s 2024-09-12T12:57:44.226 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:103000K .......... .......... .......... .......... .......... 61% 166M 3s 2024-09-12T12:57:44.226 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:103050K .......... .......... .......... .......... .......... 61% 20.1M 3s 2024-09-12T12:57:44.226 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:103100K .......... .......... .......... .......... .......... 61% 26.2M 3s 2024-09-12T12:57:44.226 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:103150K .......... .......... .......... .......... .......... 61% 38.6M 3s 2024-09-12T12:57:44.226 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:103200K .......... .......... .......... .......... .......... 61% 10.9M 3s 2024-09-12T12:57:44.226 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:103250K .......... .......... .......... .......... .......... 61% 31.1M 3s 2024-09-12T12:57:44.226 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:103300K .......... .......... .......... .......... .......... 61% 105M 3s 2024-09-12T12:57:44.226 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:103350K .......... .......... .......... .......... .......... 61% 23.7M 3s 2024-09-12T12:57:44.226 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:103400K .......... .......... .......... .......... .......... 61% 53.8M 3s 2024-09-12T12:57:44.227 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:103450K .......... .......... .......... .......... .......... 61% 19.0M 3s 2024-09-12T12:57:44.227 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:103500K .......... .......... .......... .......... .......... 61% 42.3M 3s 2024-09-12T12:57:44.227 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:103550K .......... .......... .......... .......... .......... 61% 24.9M 3s 2024-09-12T12:57:44.229 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:103600K .......... .......... .......... .......... .......... 61% 16.6M 3s 2024-09-12T12:57:44.229 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:103650K .......... .......... .......... .......... .......... 61% 32.3M 3s 2024-09-12T12:57:44.229 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:103700K .......... .......... .......... .......... .......... 61% 39.9M 3s 2024-09-12T12:57:44.231 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:103750K .......... .......... .......... .......... .......... 61% 16.5M 3s 2024-09-12T12:57:44.233 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:103800K .......... .......... .......... .......... .......... 61% 26.6M 3s 2024-09-12T12:57:44.235 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:103850K .......... .......... .......... .......... .......... 61% 26.0M 3s 2024-09-12T12:57:44.238 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:103900K .......... .......... .......... .......... .......... 61% 22.3M 3s 2024-09-12T12:57:44.238 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:103950K .......... .......... .......... .......... .......... 61% 42.6M 3s 2024-09-12T12:57:44.284 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:104000K .......... .......... .......... .......... .......... 61% 1.79M 3s 2024-09-12T12:57:44.285 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:104050K .......... .......... .......... .......... .......... 61% 177M 3s 2024-09-12T12:57:44.285 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:104100K .......... .......... .......... .......... .......... 61% 190M 3s 2024-09-12T12:57:44.285 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:104150K .......... .......... .......... .......... .......... 61% 125M 3s 2024-09-12T12:57:44.285 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:104200K .......... .......... .......... .......... .......... 61% 89.9M 3s 2024-09-12T12:57:44.285 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:104250K .......... .......... .......... .......... .......... 61% 171M 3s 2024-09-12T12:57:44.285 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:104300K .......... .......... .......... .......... .......... 61% 181M 3s 2024-09-12T12:57:44.285 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:104350K .......... .......... .......... .......... .......... 61% 155M 3s 2024-09-12T12:57:44.285 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:104400K .......... .......... .......... .......... .......... 61% 163M 3s 2024-09-12T12:57:44.285 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:104450K .......... .......... .......... .......... .......... 61% 136M 3s 2024-09-12T12:57:44.285 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:104500K .......... .......... .......... .......... .......... 62% 177M 3s 2024-09-12T12:57:44.285 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:104550K .......... .......... .......... .......... .......... 62% 166M 3s 2024-09-12T12:57:44.285 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:104600K .......... .......... .......... .......... .......... 62% 169M 3s 2024-09-12T12:57:44.285 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:104650K .......... .......... .......... .......... .......... 62% 138M 3s 2024-09-12T12:57:44.286 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:104700K .......... .......... .......... .......... .......... 62% 168M 3s 2024-09-12T12:57:44.286 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:104750K .......... .......... .......... .......... .......... 62% 125M 3s 2024-09-12T12:57:44.286 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:104800K .......... .......... .......... .......... .......... 62% 9.67M 3s 2024-09-12T12:57:44.286 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:104850K .......... .......... .......... .......... .......... 62% 29.3M 3s 2024-09-12T12:57:44.286 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:104900K .......... .......... .......... .......... .......... 62% 12.6M 3s 2024-09-12T12:57:44.286 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:104950K .......... .......... .......... .......... .......... 62% 21.4M 3s 2024-09-12T12:57:44.288 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:105000K .......... .......... .......... .......... .......... 62% 20.7M 3s 2024-09-12T12:57:44.288 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:105050K .......... .......... .......... .......... .......... 62% 73.9M 3s 2024-09-12T12:57:44.289 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:105100K .......... .......... .......... .......... .......... 62% 15.0M 3s 2024-09-12T12:57:44.294 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:105150K .......... .......... .......... .......... .......... 62% 11.5M 3s 2024-09-12T12:57:44.296 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:105200K .......... .......... .......... .......... .......... 62% 22.3M 3s 2024-09-12T12:57:44.297 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:105250K .......... .......... .......... .......... .......... 62% 26.5M 3s 2024-09-12T12:57:44.301 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:105300K .......... .......... .......... .......... .......... 62% 11.7M 3s 2024-09-12T12:57:44.304 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:105350K .......... .......... .......... .......... .......... 62% 21.4M 3s 2024-09-12T12:57:44.307 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:105400K .......... .......... .......... .......... .......... 62% 24.2M 3s 2024-09-12T12:57:44.307 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:105450K .......... .......... .......... .......... .......... 62% 96.5M 3s 2024-09-12T12:57:44.310 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:105500K .......... .......... .......... .......... .......... 62% 15.1M 3s 2024-09-12T12:57:44.314 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:105550K .......... .......... .......... .......... .......... 62% 10.8M 3s 2024-09-12T12:57:44.317 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:105600K .......... .......... .......... .......... .......... 62% 19.6M 3s 2024-09-12T12:57:44.319 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:105650K .......... .......... .......... .......... .......... 62% 26.8M 3s 2024-09-12T12:57:44.323 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:105700K .......... .......... .......... .......... .......... 62% 10.4M 3s 2024-09-12T12:57:44.324 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:105750K .......... .......... .......... .......... .......... 62% 36.1M 3s 2024-09-12T12:57:44.327 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:105800K .......... .......... .......... .......... .......... 62% 20.8M 3s 2024-09-12T12:57:44.329 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:105850K .......... .......... .......... .......... .......... 62% 26.1M 3s 2024-09-12T12:57:44.330 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:105900K .......... .......... .......... .......... .......... 62% 28.3M 3s 2024-09-12T12:57:44.335 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:105950K .......... .......... .......... .......... .......... 62% 10.8M 3s 2024-09-12T12:57:44.338 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:106000K .......... .......... .......... .......... .......... 62% 18.0M 3s 2024-09-12T12:57:44.342 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:106050K .......... .......... .......... .......... .......... 62% 12.0M 3s 2024-09-12T12:57:44.344 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:106100K .......... .......... .......... .......... .......... 62% 21.2M 3s 2024-09-12T12:57:44.345 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:106150K .......... .......... .......... .......... .......... 63% 35.1M 3s 2024-09-12T12:57:44.349 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:106200K .......... .......... .......... .......... .......... 63% 19.0M 3s 2024-09-12T12:57:44.349 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:106250K .......... .......... .......... .......... .......... 63% 31.5M 3s 2024-09-12T12:57:44.352 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:106300K .......... .......... .......... .......... .......... 63% 25.7M 3s 2024-09-12T12:57:44.356 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:106350K .......... .......... .......... .......... .......... 63% 10.7M 3s 2024-09-12T12:57:44.360 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:106400K .......... .......... .......... .......... .......... 63% 12.0M 3s 2024-09-12T12:57:44.363 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:106450K .......... .......... .......... .......... .......... 63% 17.6M 3s 2024-09-12T12:57:44.367 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:106500K .......... .......... .......... .......... .......... 63% 15.7M 3s 2024-09-12T12:57:44.369 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:106550K .......... .......... .......... .......... .......... 63% 20.2M 3s 2024-09-12T12:57:44.369 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:106600K .......... .......... .......... .......... .......... 63% 162M 3s 2024-09-12T12:57:44.370 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:106650K .......... .......... .......... .......... .......... 63% 26.9M 3s 2024-09-12T12:57:44.373 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:106700K .......... .......... .......... .......... .......... 63% 25.3M 3s 2024-09-12T12:57:44.378 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:106750K .......... .......... .......... .......... .......... 63% 8.64M 3s 2024-09-12T12:57:44.381 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:106800K .......... .......... .......... .......... .......... 63% 16.3M 3s 2024-09-12T12:57:44.384 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:106850K .......... .......... .......... .......... .......... 63% 17.9M 3s 2024-09-12T12:57:44.386 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:106900K .......... .......... .......... .......... .......... 63% 25.8M 3s 2024-09-12T12:57:44.390 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:106950K .......... .......... .......... .......... .......... 63% 14.4M 3s 2024-09-12T12:57:44.390 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:107000K .......... .......... .......... .......... .......... 63% 170M 3s 2024-09-12T12:57:44.391 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:107050K .......... .......... .......... .......... .......... 63% 27.5M 3s 2024-09-12T12:57:44.394 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:107100K .......... .......... .......... .......... .......... 63% 15.7M 3s 2024-09-12T12:57:44.399 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:107150K .......... .......... .......... .......... .......... 63% 11.1M 3s 2024-09-12T12:57:44.402 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:107200K .......... .......... .......... .......... .......... 63% 15.6M 3s 2024-09-12T12:57:44.404 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:107250K .......... .......... .......... .......... .......... 63% 19.0M 3s 2024-09-12T12:57:44.407 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:107300K .......... .......... .......... .......... .......... 63% 19.8M 3s 2024-09-12T12:57:44.410 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:107350K .......... .......... .......... .......... .......... 63% 18.4M 3s 2024-09-12T12:57:44.411 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:107400K .......... .......... .......... .......... .......... 63% 139M 3s 2024-09-12T12:57:44.414 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:107450K .......... .......... .......... .......... .......... 63% 13.5M 3s 2024-09-12T12:57:44.414 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:107500K .......... .......... .......... .......... .......... 63% 43.7M 3s 2024-09-12T12:57:44.419 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:107550K .......... .......... .......... .......... .......... 63% 11.1M 3s 2024-09-12T12:57:44.422 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:107600K .......... .......... .......... .......... .......... 63% 15.6M 3s 2024-09-12T12:57:44.425 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:107650K .......... .......... .......... .......... .......... 63% 14.5M 3s 2024-09-12T12:57:44.427 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:107700K .......... .......... .......... .......... .......... 63% 28.0M 3s 2024-09-12T12:57:44.431 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:107750K .......... .......... .......... .......... .......... 63% 17.0M 3s 2024-09-12T12:57:44.431 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:107800K .......... .......... .......... .......... .......... 63% 184M 3s 2024-09-12T12:57:44.434 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:107850K .......... .......... .......... .......... .......... 64% 14.4M 3s 2024-09-12T12:57:44.435 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:107900K .......... .......... .......... .......... .......... 64% 42.9M 3s 2024-09-12T12:57:44.440 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:107950K .......... .......... .......... .......... .......... 64% 10.7M 3s 2024-09-12T12:57:44.443 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:108000K .......... .......... .......... .......... .......... 64% 15.0M 3s 2024-09-12T12:57:44.446 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:108050K .......... .......... .......... .......... .......... 64% 15.6M 3s 2024-09-12T12:57:44.448 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:108100K .......... .......... .......... .......... .......... 64% 28.2M 3s 2024-09-12T12:57:44.451 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:108150K .......... .......... .......... .......... .......... 64% 18.7M 3s 2024-09-12T12:57:44.451 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:108200K .......... .......... .......... .......... .......... 64% 180M 3s 2024-09-12T12:57:44.454 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:108250K .......... .......... .......... .......... .......... 64% 13.0M 3s 2024-09-12T12:57:44.456 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:108300K .......... .......... .......... .......... .......... 64% 43.8M 3s 2024-09-12T12:57:44.460 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:108350K .......... .......... .......... .......... .......... 64% 10.9M 3s 2024-09-12T12:57:44.463 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:108400K .......... .......... .......... .......... .......... 64% 15.2M 3s 2024-09-12T12:57:44.466 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:108450K .......... .......... .......... .......... .......... 64% 15.5M 3s 2024-09-12T12:57:44.468 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:108500K .......... .......... .......... .......... .......... 64% 27.0M 3s 2024-09-12T12:57:44.471 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:108550K .......... .......... .......... .......... .......... 64% 19.2M 3s 2024-09-12T12:57:44.471 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:108600K .......... .......... .......... .......... .......... 64% 168M 3s 2024-09-12T12:57:44.477 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:108650K .......... .......... .......... .......... .......... 64% 12.8M 3s 2024-09-12T12:57:44.477 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:108700K .......... .......... .......... .......... .......... 64% 46.7M 3s 2024-09-12T12:57:44.480 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:108750K .......... .......... .......... .......... .......... 64% 10.8M 3s 2024-09-12T12:57:44.484 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:108800K .......... .......... .......... .......... .......... 64% 14.8M 3s 2024-09-12T12:57:44.487 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:108850K .......... .......... .......... .......... .......... 64% 14.1M 3s 2024-09-12T12:57:44.489 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:108900K .......... .......... .......... .......... .......... 64% 29.9M 3s 2024-09-12T12:57:44.492 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:108950K .......... .......... .......... .......... .......... 64% 21.1M 3s 2024-09-12T12:57:44.492 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:109000K .......... .......... .......... .......... .......... 64% 177M 3s 2024-09-12T12:57:44.495 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:109050K .......... .......... .......... .......... .......... 64% 12.9M 3s 2024-09-12T12:57:44.496 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:109100K .......... .......... .......... .......... .......... 64% 47.7M 3s 2024-09-12T12:57:44.500 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:109150K .......... .......... .......... .......... .......... 64% 10.6M 3s 2024-09-12T12:57:44.504 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:109200K .......... .......... .......... .......... .......... 64% 14.0M 3s 2024-09-12T12:57:44.507 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:109250K .......... .......... .......... .......... .......... 64% 15.2M 3s 2024-09-12T12:57:44.509 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:109300K .......... .......... .......... .......... .......... 64% 38.8M 3s 2024-09-12T12:57:44.512 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:109350K .......... .......... .......... .......... .......... 64% 18.5M 3s 2024-09-12T12:57:44.513 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:109400K .......... .......... .......... .......... .......... 64% 28.5M 3s 2024-09-12T12:57:44.516 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:109450K .......... .......... .......... .......... .......... 64% 19.9M 3s 2024-09-12T12:57:44.517 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:109500K .......... .......... .......... .......... .......... 64% 58.3M 3s 2024-09-12T12:57:44.521 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:109550K .......... .......... .......... .......... .......... 65% 9.95M 3s 2024-09-12T12:57:44.525 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:109600K .......... .......... .......... .......... .......... 65% 15.3M 3s 2024-09-12T12:57:44.528 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:109650K .......... .......... .......... .......... .......... 65% 15.2M 3s 2024-09-12T12:57:44.529 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:109700K .......... .......... .......... .......... .......... 65% 35.5M 3s 2024-09-12T12:57:44.532 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:109750K .......... .......... .......... .......... .......... 65% 18.5M 3s 2024-09-12T12:57:44.532 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:109800K .......... .......... .......... .......... .......... 65% 161M 3s 2024-09-12T12:57:44.536 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:109850K .......... .......... .......... .......... .......... 65% 12.7M 3s 2024-09-12T12:57:44.537 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:109900K .......... .......... .......... .......... .......... 65% 47.9M 3s 2024-09-12T12:57:44.541 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:109950K .......... .......... .......... .......... .......... 65% 10.5M 3s 2024-09-12T12:57:44.545 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:110000K .......... .......... .......... .......... .......... 65% 15.0M 3s 2024-09-12T12:57:44.548 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:110050K .......... .......... .......... .......... .......... 65% 15.9M 3s 2024-09-12T12:57:44.549 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:110100K .......... .......... .......... .......... .......... 65% 35.5M 3s 2024-09-12T12:57:44.553 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:110150K .......... .......... .......... .......... .......... 65% 17.8M 3s 2024-09-12T12:57:44.553 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:110200K .......... .......... .......... .......... .......... 65% 170M 3s 2024-09-12T12:57:44.556 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:110250K .......... .......... .......... .......... .......... 65% 13.3M 3s 2024-09-12T12:57:44.557 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:110300K .......... .......... .......... .......... .......... 65% 40.7M 3s 2024-09-12T12:57:44.560 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:110350K .......... .......... .......... .......... .......... 65% 15.9M 3s 2024-09-12T12:57:44.563 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:110400K .......... .......... .......... .......... .......... 65% 15.0M 3s 2024-09-12T12:57:44.569 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:110450K .......... .......... .......... .......... .......... 65% 10.3M 3s 2024-09-12T12:57:44.570 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:110500K .......... .......... .......... .......... .......... 65% 38.4M 3s 2024-09-12T12:57:44.570 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:110550K .......... .......... .......... .......... .......... 65% 40.4M 3s 2024-09-12T12:57:44.573 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:110600K .......... .......... .......... .......... .......... 65% 28.4M 3s 2024-09-12T12:57:44.577 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:110650K .......... .......... .......... .......... .......... 65% 12.8M 3s 2024-09-12T12:57:44.577 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:110700K .......... .......... .......... .......... .......... 65% 46.4M 3s 2024-09-12T12:57:44.579 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:110750K .......... .......... .......... .......... .......... 65% 23.6M 3s 2024-09-12T12:57:44.586 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:110800K .......... .......... .......... .......... .......... 65% 8.34M 3s 2024-09-12T12:57:44.587 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:110850K .......... .......... .......... .......... .......... 65% 24.9M 3s 2024-09-12T12:57:44.589 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:110900K .......... .......... .......... .......... .......... 65% 28.8M 3s 2024-09-12T12:57:44.591 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:110950K .......... .......... .......... .......... .......... 65% 21.6M 3s 2024-09-12T12:57:44.593 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:111000K .......... .......... .......... .......... .......... 65% 30.7M 3s 2024-09-12T12:57:44.597 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:111050K .......... .......... .......... .......... .......... 65% 12.1M 3s 2024-09-12T12:57:44.598 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:111100K .......... .......... .......... .......... .......... 65% 61.8M 3s 2024-09-12T12:57:44.601 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:111150K .......... .......... .......... .......... .......... 65% 14.6M 3s 2024-09-12T12:57:44.604 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:111200K .......... .......... .......... .......... .......... 65% 15.3M 3s 2024-09-12T12:57:44.607 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:111250K .......... .......... .......... .......... .......... 66% 14.4M 3s 2024-09-12T12:57:44.609 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:111300K .......... .......... .......... .......... .......... 66% 28.2M 3s 2024-09-12T12:57:44.611 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:111350K .......... .......... .......... .......... .......... 66% 24.7M 3s 2024-09-12T12:57:44.613 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:111400K .......... .......... .......... .......... .......... 66% 24.4M 3s 2024-09-12T12:57:44.614 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:111450K .......... .......... .......... .......... .......... 66% 38.3M 3s 2024-09-12T12:57:44.618 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:111500K .......... .......... .......... .......... .......... 66% 14.7M 3s 2024-09-12T12:57:44.621 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:111550K .......... .......... .......... .......... .......... 66% 14.4M 3s 2024-09-12T12:57:44.622 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:111600K .......... .......... .......... .......... .......... 66% 31.9M 3s 2024-09-12T12:57:44.628 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:111650K .......... .......... .......... .......... .......... 66% 9.55M 3s 2024-09-12T12:57:44.629 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:111700K .......... .......... .......... .......... .......... 66% 29.2M 3s 2024-09-12T12:57:44.632 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:111750K .......... .......... .......... .......... .......... 66% 22.9M 3s 2024-09-12T12:57:44.634 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:111800K .......... .......... .......... .......... .......... 66% 30.5M 3s 2024-09-12T12:57:44.634 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:111850K .......... .......... .......... .......... .......... 66% 35.8M 3s 2024-09-12T12:57:44.638 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:111900K .......... .......... .......... .......... .......... 66% 14.9M 3s 2024-09-12T12:57:44.639 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:111950K .......... .......... .......... .......... .......... 66% 39.5M 3s 2024-09-12T12:57:44.643 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:112000K .......... .......... .......... .......... .......... 66% 11.3M 3s 2024-09-12T12:57:44.646 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:112050K .......... .......... .......... .......... .......... 66% 18.1M 3s 2024-09-12T12:57:44.649 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:112100K .......... .......... .......... .......... .......... 66% 14.9M 3s 2024-09-12T12:57:44.651 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:112150K .......... .......... .......... .......... .......... 66% 30.7M 3s 2024-09-12T12:57:44.653 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:112200K .......... .......... .......... .......... .......... 66% 20.0M 3s 2024-09-12T12:57:44.655 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:112250K .......... .......... .......... .......... .......... 66% 29.7M 3s 2024-09-12T12:57:44.658 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:112300K .......... .......... .......... .......... .......... 66% 16.2M 3s 2024-09-12T12:57:44.659 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:112350K .......... .......... .......... .......... .......... 66% 41.1M 3s 2024-09-12T12:57:44.664 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:112400K .......... .......... .......... .......... .......... 66% 11.7M 3s 2024-09-12T12:57:44.665 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:112450K .......... .......... .......... .......... .......... 66% 26.5M 3s 2024-09-12T12:57:44.668 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:112500K .......... .......... .......... .......... .......... 66% 14.5M 3s 2024-09-12T12:57:44.672 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:112550K .......... .......... .......... .......... .......... 66% 14.6M 3s 2024-09-12T12:57:44.673 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:112600K .......... .......... .......... .......... .......... 66% 41.1M 3s 2024-09-12T12:57:44.674 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:112650K .......... .......... .......... .......... .......... 66% 73.4M 3s 2024-09-12T12:57:44.678 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:112700K .......... .......... .......... .......... .......... 66% 11.3M 3s 2024-09-12T12:57:44.679 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:112750K .......... .......... .......... .......... .......... 66% 34.6M 3s 2024-09-12T12:57:44.684 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:112800K .......... .......... .......... .......... .......... 66% 11.7M 3s 2024-09-12T12:57:44.685 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:112850K .......... .......... .......... .......... .......... 66% 30.6M 3s 2024-09-12T12:57:44.689 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:112900K .......... .......... .......... .......... .......... 67% 13.3M 3s 2024-09-12T12:57:44.691 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:112950K .......... .......... .......... .......... .......... 67% 19.4M 3s 2024-09-12T12:57:44.693 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:113000K .......... .......... .......... .......... .......... 67% 51.3M 3s 2024-09-12T12:57:44.694 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:113050K .......... .......... .......... .......... .......... 67% 32.3M 3s 2024-09-12T12:57:44.699 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:113100K .......... .......... .......... .......... .......... 67% 11.1M 3s 2024-09-12T12:57:44.700 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:113150K .......... .......... .......... .......... .......... 67% 37.3M 3s 2024-09-12T12:57:44.702 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:113200K .......... .......... .......... .......... .......... 67% 18.3M 3s 2024-09-12T12:57:44.704 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:113250K .......... .......... .......... .......... .......... 67% 27.9M 3s 2024-09-12T12:57:44.707 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:113300K .......... .......... .......... .......... .......... 67% 17.9M 3s 2024-09-12T12:57:44.710 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:113350K .......... .......... .......... .......... .......... 67% 14.3M 3s 2024-09-12T12:57:44.713 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:113400K .......... .......... .......... .......... .......... 67% 19.3M 3s 2024-09-12T12:57:44.714 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:113450K .......... .......... .......... .......... .......... 67% 36.5M 3s 2024-09-12T12:57:44.719 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:113500K .......... .......... .......... .......... .......... 67% 10.4M 3s 2024-09-12T12:57:44.719 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:113550K .......... .......... .......... .......... .......... 67% 153M 2s 2024-09-12T12:57:44.721 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:113600K .......... .......... .......... .......... .......... 67% 24.5M 2s 2024-09-12T12:57:44.725 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:113650K .......... .......... .......... .......... .......... 67% 14.1M 2s 2024-09-12T12:57:44.726 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:113700K .......... .......... .......... .......... .......... 67% 33.9M 2s 2024-09-12T12:57:44.730 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:113750K .......... .......... .......... .......... .......... 67% 10.9M 2s 2024-09-12T12:57:44.733 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:113800K .......... .......... .......... .......... .......... 67% 18.1M 2s 2024-09-12T12:57:44.734 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:113850K .......... .......... .......... .......... .......... 67% 41.4M 2s 2024-09-12T12:57:44.735 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:113900K .......... .......... .......... .......... .......... 67% 39.1M 2s 2024-09-12T12:57:44.739 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:113950K .......... .......... .......... .......... .......... 67% 12.9M 2s 2024-09-12T12:57:44.740 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:114000K .......... .......... .......... .......... .......... 67% 45.2M 2s 2024-09-12T12:57:44.745 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:114050K .......... .......... .......... .......... .......... 67% 10.6M 2s 2024-09-12T12:57:44.746 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:114100K .......... .......... .......... .......... .......... 67% 38.6M 2s 2024-09-12T12:57:44.750 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:114150K .......... .......... .......... .......... .......... 67% 12.5M 2s 2024-09-12T12:57:44.752 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:114200K .......... .......... .......... .......... .......... 67% 23.4M 2s 2024-09-12T12:57:44.755 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:114250K .......... .......... .......... .......... .......... 67% 30.8M 2s 2024-09-12T12:57:44.755 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:114300K .......... .......... .......... .......... .......... 67% 54.3M 2s 2024-09-12T12:57:44.761 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:114350K .......... .......... .......... .......... .......... 67% 10.0M 2s 2024-09-12T12:57:44.761 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:114400K .......... .......... .......... .......... .......... 67% 44.8M 2s 2024-09-12T12:57:44.765 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:114450K .......... .......... .......... .......... .......... 67% 10.9M 2s 2024-09-12T12:57:44.767 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:114500K .......... .......... .......... .......... .......... 67% 40.6M 2s 2024-09-12T12:57:44.767 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:114550K .......... .......... .......... .......... .......... 67% 35.1M 2s 2024-09-12T12:57:44.771 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:114600K .......... .......... .......... .......... .......... 68% 14.8M 2s 2024-09-12T12:57:44.774 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:114650K .......... .......... .......... .......... .......... 68% 16.3M 2s 2024-09-12T12:57:44.775 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:114700K .......... .......... .......... .......... .......... 68% 44.0M 2s 2024-09-12T12:57:44.781 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:114750K .......... .......... .......... .......... .......... 68% 10.2M 2s 2024-09-12T12:57:44.781 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:114800K .......... .......... .......... .......... .......... 68% 160M 2s 2024-09-12T12:57:44.783 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:114850K .......... .......... .......... .......... .......... 68% 15.4M 2s 2024-09-12T12:57:44.786 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:114900K .......... .......... .......... .......... .......... 68% 19.1M 2s 2024-09-12T12:57:44.787 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:114950K .......... .......... .......... .......... .......... 68% 41.6M 2s 2024-09-12T12:57:44.791 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:115000K .......... .......... .......... .......... .......... 68% 13.2M 2s 2024-09-12T12:57:44.794 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:115050K .......... .......... .......... .......... .......... 68% 14.5M 2s 2024-09-12T12:57:44.796 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:115100K .......... .......... .......... .......... .......... 68% 42.0M 2s 2024-09-12T12:57:44.796 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:115150K .......... .......... .......... .......... .......... 68% 47.2M 2s 2024-09-12T12:57:44.801 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:115200K .......... .......... .......... .......... .......... 68% 12.5M 2s 2024-09-12T12:57:44.802 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:115250K .......... .......... .......... .......... .......... 68% 46.9M 2s 2024-09-12T12:57:44.806 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:115300K .......... .......... .......... .......... .......... 68% 10.7M 2s 2024-09-12T12:57:44.807 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:115350K .......... .......... .......... .......... .......... 68% 36.8M 2s 2024-09-12T12:57:44.811 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:115400K .......... .......... .......... .......... .......... 68% 12.8M 2s 2024-09-12T12:57:44.813 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:115450K .......... .......... .......... .......... .......... 68% 21.0M 2s 2024-09-12T12:57:44.815 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:115500K .......... .......... .......... .......... .......... 68% 43.2M 2s 2024-09-12T12:57:44.816 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:115550K .......... .......... .......... .......... .......... 68% 39.7M 2s 2024-09-12T12:57:44.822 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:115600K .......... .......... .......... .......... .......... 68% 9.92M 2s 2024-09-12T12:57:44.822 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:115650K .......... .......... .......... .......... .......... 68% 45.5M 2s 2024-09-12T12:57:44.827 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:115700K .......... .......... .......... .......... .......... 68% 11.0M 2s 2024-09-12T12:57:44.828 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:115750K .......... .......... .......... .......... .......... 68% 44.5M 2s 2024-09-12T12:57:44.828 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:115800K .......... .......... .......... .......... .......... 68% 41.6M 2s 2024-09-12T12:57:44.834 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:115850K .......... .......... .......... .......... .......... 68% 9.16M 2s 2024-09-12T12:57:44.835 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:115900K .......... .......... .......... .......... .......... 68% 39.5M 2s 2024-09-12T12:57:44.837 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:115950K .......... .......... .......... .......... .......... 68% 39.4M 2s 2024-09-12T12:57:44.842 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:116000K .......... .......... .......... .......... .......... 68% 10.7M 2s 2024-09-12T12:57:44.842 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:116050K .......... .......... .......... .......... .......... 68% 172M 2s 2024-09-12T12:57:44.848 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:116100K .......... .......... .......... .......... .......... 68% 9.46M 2s 2024-09-12T12:57:44.848 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:116150K .......... .......... .......... .......... .......... 68% 109M 2s 2024-09-12T12:57:44.849 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:116200K .......... .......... .......... .......... .......... 68% 24.5M 2s 2024-09-12T12:57:44.852 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:116250K .......... .......... .......... .......... .......... 68% 15.3M 2s 2024-09-12T12:57:44.855 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:116300K .......... .......... .......... .......... .......... 69% 15.1M 2s 2024-09-12T12:57:44.857 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:116350K .......... .......... .......... .......... .......... 69% 39.1M 2s 2024-09-12T12:57:44.857 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:116400K .......... .......... .......... .......... .......... 69% 60.3M 2s 2024-09-12T12:57:44.862 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:116450K .......... .......... .......... .......... .......... 69% 10.9M 2s 2024-09-12T12:57:44.863 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:116500K .......... .......... .......... .......... .......... 69% 99.9M 2s 2024-09-12T12:57:44.867 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:116550K .......... .......... .......... .......... .......... 69% 10.5M 2s 2024-09-12T12:57:44.869 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:116600K .......... .......... .......... .......... .......... 69% 38.8M 2s 2024-09-12T12:57:44.873 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:116650K .......... .......... .......... .......... .......... 69% 11.2M 2s 2024-09-12T12:57:44.875 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:116700K .......... .......... .......... .......... .......... 69% 17.6M 2s 2024-09-12T12:57:44.876 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:116750K .......... .......... .......... .......... .......... 69% 40.0M 2s 2024-09-12T12:57:44.877 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:116800K .......... .......... .......... .......... .......... 69% 56.9M 2s 2024-09-12T12:57:44.883 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:116850K .......... .......... .......... .......... .......... 69% 11.0M 2s 2024-09-12T12:57:44.883 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:116900K .......... .......... .......... .......... .......... 69% 70.8M 2s 2024-09-12T12:57:44.888 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:116950K .......... .......... .......... .......... .......... 69% 10.4M 2s 2024-09-12T12:57:44.888 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:117000K .......... .......... .......... .......... .......... 69% 46.5M 2s 2024-09-12T12:57:44.889 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:117050K .......... .......... .......... .......... .......... 69% 45.6M 2s 2024-09-12T12:57:44.895 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:117100K .......... .......... .......... .......... .......... 69% 8.79M 2s 2024-09-12T12:57:44.897 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:117150K .......... .......... .......... .......... .......... 69% 27.9M 2s 2024-09-12T12:57:44.898 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:117200K .......... .......... .......... .......... .......... 69% 152M 2s 2024-09-12T12:57:44.903 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:117250K .......... .......... .......... .......... .......... 69% 9.68M 2s 2024-09-12T12:57:44.903 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:117300K .......... .......... .......... .......... .......... 69% 174M 2s 2024-09-12T12:57:44.908 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:117350K .......... .......... .......... .......... .......... 69% 9.57M 2s 2024-09-12T12:57:44.908 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:117400K .......... .......... .......... .......... .......... 69% 186M 2s 2024-09-12T12:57:44.909 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:117450K .......... .......... .......... .......... .......... 69% 27.2M 2s 2024-09-12T12:57:44.913 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:117500K .......... .......... .......... .......... .......... 69% 12.7M 2s 2024-09-12T12:57:44.916 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:117550K .......... .......... .......... .......... .......... 69% 19.1M 2s 2024-09-12T12:57:44.917 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:117600K .......... .......... .......... .......... .......... 69% 38.6M 2s 2024-09-12T12:57:44.923 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:117650K .......... .......... .......... .......... .......... 69% 9.35M 2s 2024-09-12T12:57:44.923 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:117700K .......... .......... .......... .......... .......... 69% 173M 2s 2024-09-12T12:57:44.924 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:117750K .......... .......... .......... .......... .......... 69% 109M 2s 2024-09-12T12:57:44.928 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:117800K .......... .......... .......... .......... .......... 69% 10.4M 2s 2024-09-12T12:57:44.929 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:117850K .......... .......... .......... .......... .......... 69% 46.9M 2s 2024-09-12T12:57:44.934 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:117900K .......... .......... .......... .......... .......... 69% 9.88M 2s 2024-09-12T12:57:44.936 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:117950K .......... .......... .......... .......... .......... 70% 17.9M 2s 2024-09-12T12:57:44.938 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:118000K .......... .......... .......... .......... .......... 70% 61.2M 2s 2024-09-12T12:57:44.938 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:118050K .......... .......... .......... .......... .......... 70% 51.8M 2s 2024-09-12T12:57:44.944 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:118100K .......... .......... .......... .......... .......... 70% 10.5M 2s 2024-09-12T12:57:44.944 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:118150K .......... .......... .......... .......... .......... 70% 107M 2s 2024-09-12T12:57:44.948 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:118200K .......... .......... .......... .......... .......... 70% 10.1M 2s 2024-09-12T12:57:44.950 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:118250K .......... .......... .......... .......... .......... 70% 26.1M 2s 2024-09-12T12:57:44.951 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:118300K .......... .......... .......... .......... .......... 70% 43.4M 2s 2024-09-12T12:57:44.996 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:118350K .......... .......... .......... .......... .......... 70% 1.40M 2s 2024-09-12T12:57:44.997 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:118400K .......... .......... .......... .......... .......... 70% 158M 2s 2024-09-12T12:57:44.997 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:118450K .......... .......... .......... .......... .......... 70% 189M 2s 2024-09-12T12:57:44.997 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:118500K .......... .......... .......... .......... .......... 70% 197M 2s 2024-09-12T12:57:44.997 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:118550K .......... .......... .......... .......... .......... 70% 168M 2s 2024-09-12T12:57:44.997 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:118600K .......... .......... .......... .......... .......... 70% 169M 2s 2024-09-12T12:57:44.997 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:118650K .......... .......... .......... .......... .......... 70% 206M 2s 2024-09-12T12:57:44.997 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:118700K .......... .......... .......... .......... .......... 70% 198M 2s 2024-09-12T12:57:44.997 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:118750K .......... .......... .......... .......... .......... 70% 158M 2s 2024-09-12T12:57:44.997 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:118800K .......... .......... .......... .......... .......... 70% 200M 2s 2024-09-12T12:57:44.997 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:118850K .......... .......... .......... .......... .......... 70% 190M 2s 2024-09-12T12:57:44.997 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:118900K .......... .......... .......... .......... .......... 70% 190M 2s 2024-09-12T12:57:44.997 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:118950K .......... .......... .......... .......... .......... 70% 172M 2s 2024-09-12T12:57:44.997 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:119000K .......... .......... .......... .......... .......... 70% 14.9M 2s 2024-09-12T12:57:44.998 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:119050K .......... .......... .......... .......... .......... 70% 8.58M 2s 2024-09-12T12:57:45.000 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:119100K .......... .......... .......... .......... .......... 70% 25.0M 2s 2024-09-12T12:57:45.003 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:119150K .......... .......... .......... .......... .......... 70% 19.3M 2s 2024-09-12T12:57:45.007 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:119200K .......... .......... .......... .......... .......... 70% 11.0M 2s 2024-09-12T12:57:45.008 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:119250K .......... .......... .......... .......... .......... 70% 51.2M 2s 2024-09-12T12:57:45.013 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:119300K .......... .......... .......... .......... .......... 70% 9.52M 2s 2024-09-12T12:57:45.019 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:119350K .......... .......... .......... .......... .......... 70% 7.95M 2s 2024-09-12T12:57:45.024 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:119400K .......... .......... .......... .......... .......... 70% 14.4M 2s 2024-09-12T12:57:45.024 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:119450K .......... .......... .......... .......... .......... 70% 52.9M 2s 2024-09-12T12:57:45.028 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:119500K .......... .......... .......... .......... .......... 70% 11.5M 2s 2024-09-12T12:57:45.030 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:119550K .......... .......... .......... .......... .......... 70% 22.4M 2s 2024-09-12T12:57:45.034 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:119600K .......... .......... .......... .......... .......... 70% 12.7M 2s 2024-09-12T12:57:45.040 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:119650K .......... .......... .......... .......... .......... 71% 7.83M 2s 2024-09-12T12:57:45.044 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:119700K .......... .......... .......... .......... .......... 71% 13.3M 2s 2024-09-12T12:57:45.045 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:119750K .......... .......... .......... .......... .......... 71% 69.7M 2s 2024-09-12T12:57:45.049 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:119800K .......... .......... .......... .......... .......... 71% 11.5M 2s 2024-09-12T12:57:45.050 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:119850K .......... .......... .......... .......... .......... 71% 47.4M 2s 2024-09-12T12:57:45.054 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:119900K .......... .......... .......... .......... .......... 71% 12.8M 2s 2024-09-12T12:57:45.061 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:119950K .......... .......... .......... .......... .......... 71% 6.50M 2s 2024-09-12T12:57:45.064 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:120000K .......... .......... .......... .......... .......... 71% 13.7M 2s 2024-09-12T12:57:45.065 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:120050K .......... .......... .......... .......... .......... 71% 77.2M 2s 2024-09-12T12:57:45.070 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:120100K .......... .......... .......... .......... .......... 71% 10.9M 2s 2024-09-12T12:57:45.072 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:120150K .......... .......... .......... .......... .......... 71% 23.0M 2s 2024-09-12T12:57:45.075 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:120200K .......... .......... .......... .......... .......... 71% 13.6M 2s 2024-09-12T12:57:45.082 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:120250K .......... .......... .......... .......... .......... 71% 6.94M 2s 2024-09-12T12:57:45.088 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:120300K .......... .......... .......... .......... .......... 71% 14.7M 2s 2024-09-12T12:57:45.088 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:120350K .......... .......... .......... .......... .......... 71% 89.7M 2s 2024-09-12T12:57:45.091 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:120400K .......... .......... .......... .......... .......... 71% 11.5M 2s 2024-09-12T12:57:45.093 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:120450K .......... .......... .......... .......... .......... 71% 22.7M 2s 2024-09-12T12:57:45.096 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:120500K .......... .......... .......... .......... .......... 71% 14.0M 2s 2024-09-12T12:57:45.103 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:120550K .......... .......... .......... .......... .......... 71% 7.07M 2s 2024-09-12T12:57:45.108 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:120600K .......... .......... .......... .......... .......... 71% 13.7M 2s 2024-09-12T12:57:45.108 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:120650K .......... .......... .......... .......... .......... 71% 140M 2s 2024-09-12T12:57:45.112 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:120700K .......... .......... .......... .......... .......... 71% 12.1M 2s 2024-09-12T12:57:45.116 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:120750K .......... .......... .......... .......... .......... 71% 9.87M 2s 2024-09-12T12:57:45.119 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:120800K .......... .......... .......... .......... .......... 71% 18.1M 2s 2024-09-12T12:57:45.125 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:120850K .......... .......... .......... .......... .......... 71% 7.74M 2s 2024-09-12T12:57:45.128 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:120900K .......... .......... .......... .......... .......... 71% 20.4M 2s 2024-09-12T12:57:45.128 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:120950K .......... .......... .......... .......... .......... 71% 161M 2s 2024-09-12T12:57:45.132 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:121000K .......... .......... .......... .......... .......... 71% 11.8M 2s 2024-09-12T12:57:45.134 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:121050K .......... .......... .......... .......... .......... 71% 20.1M 2s 2024-09-12T12:57:45.138 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:121100K .......... .......... .......... .......... .......... 71% 13.6M 2s 2024-09-12T12:57:45.146 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:121150K .......... .......... .......... .......... .......... 71% 6.22M 2s 2024-09-12T12:57:45.148 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:121200K .......... .......... .......... .......... .......... 71% 21.8M 2s 2024-09-12T12:57:45.149 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:121250K .......... .......... .......... .......... .......... 71% 38.7M 2s 2024-09-12T12:57:45.153 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:121300K .......... .......... .......... .......... .......... 71% 14.8M 2s 2024-09-12T12:57:45.158 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:121350K .......... .......... .......... .......... .......... 72% 9.18M 2s 2024-09-12T12:57:45.159 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:121400K .......... .......... .......... .......... .......... 72% 25.5M 2s 2024-09-12T12:57:45.167 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:121450K .......... .......... .......... .......... .......... 72% 6.74M 2s 2024-09-12T12:57:45.170 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:121500K .......... .......... .......... .......... .......... 72% 24.6M 2s 2024-09-12T12:57:45.170 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:121550K .......... .......... .......... .......... .......... 72% 38.2M 2s 2024-09-12T12:57:45.174 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:121600K .......... .......... .......... .......... .......... 72% 15.0M 2s 2024-09-12T12:57:45.179 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:121650K .......... .......... .......... .......... .......... 72% 9.74M 2s 2024-09-12T12:57:45.180 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:121700K .......... .......... .......... .......... .......... 72% 34.5M 2s 2024-09-12T12:57:45.187 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:121750K .......... .......... .......... .......... .......... 72% 6.76M 2s 2024-09-12T12:57:45.190 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:121800K .......... .......... .......... .......... .......... 72% 21.5M 2s 2024-09-12T12:57:45.190 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:121850K .......... .......... .......... .......... .......... 72% 198M 2s 2024-09-12T12:57:45.194 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:121900K .......... .......... .......... .......... .......... 72% 11.4M 2s 2024-09-12T12:57:45.200 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:121950K .......... .......... .......... .......... .......... 72% 9.15M 2s 2024-09-12T12:57:45.207 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:122000K .......... .......... .......... .......... .......... 72% 6.72M 2s 2024-09-12T12:57:45.211 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:122050K .......... .......... .......... .......... .......... 72% 17.1M 2s 2024-09-12T12:57:45.211 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:122100K .......... .......... .......... .......... .......... 72% 171M 2s 2024-09-12T12:57:45.211 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:122150K .......... .......... .......... .......... .......... 72% 40.3M 2s 2024-09-12T12:57:45.214 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:122200K .......... .......... .......... .......... .......... 72% 14.5M 2s 2024-09-12T12:57:45.220 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:122250K .......... .......... .......... .......... .......... 72% 9.29M 2s 2024-09-12T12:57:45.221 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:122300K .......... .......... .......... .......... .......... 72% 34.6M 2s 2024-09-12T12:57:45.231 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:122350K .......... .......... .......... .......... .......... 72% 5.53M 2s 2024-09-12T12:57:45.231 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:122400K .......... .......... .......... .......... .......... 72% 168M 2s 2024-09-12T12:57:45.231 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:122450K .......... .......... .......... .......... .......... 72% 39.2M 2s 2024-09-12T12:57:45.235 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:122500K .......... .......... .......... .......... .......... 72% 12.4M 2s 2024-09-12T12:57:45.240 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:122550K .......... .......... .......... .......... .......... 72% 10.1M 2s 2024-09-12T12:57:45.241 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:122600K .......... .......... .......... .......... .......... 72% 50.3M 2s 2024-09-12T12:57:45.248 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:122650K .......... .......... .......... .......... .......... 72% 6.84M 2s 2024-09-12T12:57:45.251 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:122700K .......... .......... .......... .......... .......... 72% 22.5M 2s 2024-09-12T12:57:45.252 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:122750K .......... .......... .......... .......... .......... 72% 38.1M 2s 2024-09-12T12:57:45.255 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:122800K .......... .......... .......... .......... .......... 72% 13.8M 2s 2024-09-12T12:57:45.261 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:122850K .......... .......... .......... .......... .......... 72% 9.77M 2s 2024-09-12T12:57:45.261 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:122900K .......... .......... .......... .......... .......... 72% 120M 2s 2024-09-12T12:57:45.268 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:122950K .......... .......... .......... .......... .......... 72% 6.70M 2s 2024-09-12T12:57:45.271 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:123000K .......... .......... .......... .......... .......... 72% 17.6M 2s 2024-09-12T12:57:45.271 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:123050K .......... .......... .......... .......... .......... 73% 178M 2s 2024-09-12T12:57:45.276 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:123100K .......... .......... .......... .......... .......... 73% 10.7M 2s 2024-09-12T12:57:45.282 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:123150K .......... .......... .......... .......... .......... 73% 9.48M 2s 2024-09-12T12:57:45.282 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:123200K .......... .......... .......... .......... .......... 73% 96.2M 2s 2024-09-12T12:57:45.289 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:123250K .......... .......... .......... .......... .......... 73% 6.65M 2s 2024-09-12T12:57:45.294 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:123300K .......... .......... .......... .......... .......... 73% 18.5M 2s 2024-09-12T12:57:45.294 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:123350K .......... .......... .......... .......... .......... 73% 187M 2s 2024-09-12T12:57:45.296 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:123400K .......... .......... .......... .......... .......... 73% 11.0M 2s 2024-09-12T12:57:45.297 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:123450K .......... .......... .......... .......... .......... 73% 135M 2s 2024-09-12T12:57:45.302 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:123500K .......... .......... .......... .......... .......... 73% 9.63M 2s 2024-09-12T12:57:45.309 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:123550K .......... .......... .......... .......... .......... 73% 6.38M 2s 2024-09-12T12:57:45.312 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:123600K .......... .......... .......... .......... .......... 73% 20.3M 2s 2024-09-12T12:57:45.313 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:123650K .......... .......... .......... .......... .......... 73% 107M 2s 2024-09-12T12:57:45.317 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:123700K .......... .......... .......... .......... .......... 73% 11.6M 2s 2024-09-12T12:57:45.317 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:123750K .......... .......... .......... .......... .......... 73% 138M 2s 2024-09-12T12:57:45.322 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:123800K .......... .......... .......... .......... .......... 73% 9.22M 2s 2024-09-12T12:57:45.323 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:123850K .......... .......... .......... .......... .......... 73% 63.0M 2s 2024-09-12T12:57:45.329 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:123900K .......... .......... .......... .......... .......... 73% 6.70M 2s 2024-09-12T12:57:45.333 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:123950K .......... .......... .......... .......... .......... 73% 21.8M 2s 2024-09-12T12:57:45.333 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:124000K .......... .......... .......... .......... .......... 73% 42.1M 2s 2024-09-12T12:57:45.337 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:124050K .......... .......... .......... .......... .......... 73% 13.8M 2s 2024-09-12T12:57:45.343 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:124100K .......... .......... .......... .......... .......... 73% 9.45M 2s 2024-09-12T12:57:45.343 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:124150K .......... .......... .......... .......... .......... 73% 151M 2s 2024-09-12T12:57:45.350 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:124200K .......... .......... .......... .......... .......... 73% 6.40M 2s 2024-09-12T12:57:45.354 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:124250K .......... .......... .......... .......... .......... 73% 19.2M 2s 2024-09-12T12:57:45.354 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:124300K .......... .......... .......... .......... .......... 73% 208M 2s 2024-09-12T12:57:45.357 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:124350K .......... .......... .......... .......... .......... 73% 11.2M 2s 2024-09-12T12:57:45.364 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:124400K .......... .......... .......... .......... .......... 73% 9.34M 2s 2024-09-12T12:57:45.364 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:124450K .......... .......... .......... .......... .......... 73% 171M 2s 2024-09-12T12:57:45.370 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:124500K .......... .......... .......... .......... .......... 73% 6.15M 2s 2024-09-12T12:57:45.374 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:124550K .......... .......... .......... .......... .......... 73% 23.5M 2s 2024-09-12T12:57:45.374 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:124600K .......... .......... .......... .......... .......... 73% 193M 2s 2024-09-12T12:57:45.378 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:124650K .......... .......... .......... .......... .......... 73% 10.6M 2s 2024-09-12T12:57:45.378 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:124700K .......... .......... .......... .......... .......... 74% 146M 2s 2024-09-12T12:57:45.383 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:124750K .......... .......... .......... .......... .......... 74% 9.73M 2s 2024-09-12T12:57:45.391 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:124800K .......... .......... .......... .......... .......... 74% 6.02M 2s 2024-09-12T12:57:45.394 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:124850K .......... .......... .......... .......... .......... 74% 23.8M 2s 2024-09-12T12:57:45.394 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:124900K .......... .......... .......... .......... .......... 74% 97.7M 2s 2024-09-12T12:57:45.398 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:124950K .......... .......... .......... .......... .......... 74% 11.7M 2s 2024-09-12T12:57:45.398 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:125000K .......... .......... .......... .......... .......... 74% 109M 2s 2024-09-12T12:57:45.404 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:125050K .......... .......... .......... .......... .......... 74% 9.82M 2s 2024-09-12T12:57:45.404 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:125100K .......... .......... .......... .......... .......... 74% 101M 2s 2024-09-12T12:57:45.414 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:125150K .......... .......... .......... .......... .......... 74% 5.10M 2s 2024-09-12T12:57:45.415 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:125200K .......... .......... .......... .......... .......... 74% 182M 2s 2024-09-12T12:57:45.415 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:125250K .......... .......... .......... .......... .......... 74% 63.8M 2s 2024-09-12T12:57:45.418 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:125300K .......... .......... .......... .......... .......... 74% 12.0M 2s 2024-09-12T12:57:45.424 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:125350K .......... .......... .......... .......... .......... 74% 9.56M 2s 2024-09-12T12:57:45.424 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:125400K .......... .......... .......... .......... .......... 74% 207M 2s 2024-09-12T12:57:45.432 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:125450K .......... .......... .......... .......... .......... 74% 5.93M 2s 2024-09-12T12:57:45.434 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:125500K .......... .......... .......... .......... .......... 74% 24.9M 2s 2024-09-12T12:57:45.435 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:125550K .......... .......... .......... .......... .......... 74% 84.5M 2s 2024-09-12T12:57:45.438 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:125600K .......... .......... .......... .......... .......... 74% 12.0M 2s 2024-09-12T12:57:45.445 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:125650K .......... .......... .......... .......... .......... 74% 9.34M 2s 2024-09-12T12:57:45.445 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:125700K .......... .......... .......... .......... .......... 74% 193M 2s 2024-09-12T12:57:45.452 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:125750K .......... .......... .......... .......... .......... 74% 5.89M 2s 2024-09-12T12:57:45.455 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:125800K .......... .......... .......... .......... .......... 74% 26.3M 2s 2024-09-12T12:57:45.455 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:125850K .......... .......... .......... .......... .......... 74% 170M 2s 2024-09-12T12:57:45.459 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:125900K .......... .......... .......... .......... .......... 74% 11.6M 2s 2024-09-12T12:57:45.460 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:125950K .......... .......... .......... .......... .......... 74% 24.7M 2s 2024-09-12T12:57:45.464 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:126000K .......... .......... .......... .......... .......... 74% 13.2M 2s 2024-09-12T12:57:45.473 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:126050K .......... .......... .......... .......... .......... 74% 5.85M 2s 2024-09-12T12:57:45.475 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:126100K .......... .......... .......... .......... .......... 74% 29.9M 2s 2024-09-12T12:57:45.475 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:126150K .......... .......... .......... .......... .......... 74% 171M 2s 2024-09-12T12:57:45.479 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:126200K .......... .......... .......... .......... .......... 74% 11.3M 2s 2024-09-12T12:57:45.480 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:126250K .......... .......... .......... .......... .......... 74% 170M 2s 2024-09-12T12:57:45.485 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:126300K .......... .......... .......... .......... .......... 74% 8.68M 2s 2024-09-12T12:57:45.493 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:126350K .......... .......... .......... .......... .......... 74% 6.06M 2s 2024-09-12T12:57:45.496 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:126400K .......... .......... .......... .......... .......... 75% 28.6M 2s 2024-09-12T12:57:45.496 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:126450K .......... .......... .......... .......... .......... 75% 182M 2s 2024-09-12T12:57:45.496 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:126500K .......... .......... .......... .......... .......... 75% 34.8M 2s 2024-09-12T12:57:45.499 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:126550K .......... .......... .......... .......... .......... 75% 14.8M 2s 2024-09-12T12:57:45.506 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:126600K .......... .......... .......... .......... .......... 75% 8.93M 2s 2024-09-12T12:57:45.506 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:126650K .......... .......... .......... .......... .......... 75% 178M 2s 2024-09-12T12:57:45.513 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:126700K .......... .......... .......... .......... .......... 75% 5.94M 2s 2024-09-12T12:57:45.516 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:126750K .......... .......... .......... .......... .......... 75% 27.8M 2s 2024-09-12T12:57:45.516 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:126800K .......... .......... .......... .......... .......... 75% 184M 2s 2024-09-12T12:57:45.520 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:126850K .......... .......... .......... .......... .......... 75% 10.9M 2s 2024-09-12T12:57:45.526 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:126900K .......... .......... .......... .......... .......... 75% 8.92M 2s 2024-09-12T12:57:45.526 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:126950K .......... .......... .......... .......... .......... 75% 96.6M 2s 2024-09-12T12:57:45.534 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:127000K .......... .......... .......... .......... .......... 75% 6.12M 2s 2024-09-12T12:57:45.536 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:127050K .......... .......... .......... .......... .......... 75% 30.4M 2s 2024-09-12T12:57:45.537 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:127100K .......... .......... .......... .......... .......... 75% 108M 2s 2024-09-12T12:57:45.540 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:127150K .......... .......... .......... .......... .......... 75% 11.7M 2s 2024-09-12T12:57:45.546 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:127200K .......... .......... .......... .......... .......... 75% 9.03M 2s 2024-09-12T12:57:45.546 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:127250K .......... .......... .......... .......... .......... 75% 109M 2s 2024-09-12T12:57:45.554 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:127300K .......... .......... .......... .......... .......... 75% 5.92M 2s 2024-09-12T12:57:45.557 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:127350K .......... .......... .......... .......... .......... 75% 33.4M 2s 2024-09-12T12:57:45.557 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:127400K .......... .......... .......... .......... .......... 75% 103M 2s 2024-09-12T12:57:45.557 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:127450K .......... .......... .......... .......... .......... 75% 55.5M 2s 2024-09-12T12:57:45.560 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:127500K .......... .......... .......... .......... .......... 75% 13.7M 2s 2024-09-12T12:57:45.566 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:127550K .......... .......... .......... .......... .......... 75% 8.74M 2s 2024-09-12T12:57:45.575 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:127600K .......... .......... .......... .......... .......... 75% 5.76M 2s 2024-09-12T12:57:45.578 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:127650K .......... .......... .......... .......... .......... 75% 37.4M 2s 2024-09-12T12:57:45.578 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:127700K .......... .......... .......... .......... .......... 75% 79.8M 2s 2024-09-12T12:57:45.578 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:127750K .......... .......... .......... .......... .......... 75% 79.8M 2s 2024-09-12T12:57:45.581 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:127800K .......... .......... .......... .......... .......... 75% 13.7M 2s 2024-09-12T12:57:45.587 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:127850K .......... .......... .......... .......... .......... 75% 8.53M 2s 2024-09-12T12:57:45.587 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:127900K .......... .......... .......... .......... .......... 75% 87.2M 2s 2024-09-12T12:57:45.595 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:127950K .......... .......... .......... .......... .......... 75% 6.03M 2s 2024-09-12T12:57:45.597 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:128000K .......... .......... .......... .......... .......... 75% 33.5M 2s 2024-09-12T12:57:45.597 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:128050K .......... .......... .......... .......... .......... 75% 112M 2s 2024-09-12T12:57:45.601 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:128100K .......... .......... .......... .......... .......... 76% 12.2M 2s 2024-09-12T12:57:45.602 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:128150K .......... .......... .......... .......... .......... 76% 30.0M 2s 2024-09-12T12:57:45.607 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:128200K .......... .......... .......... .......... .......... 76% 11.0M 2s 2024-09-12T12:57:45.615 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:128250K .......... .......... .......... .......... .......... 76% 5.97M 2s 2024-09-12T12:57:45.618 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:128300K .......... .......... .......... .......... .......... 76% 27.3M 2s 2024-09-12T12:57:45.619 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:128350K .......... .......... .......... .......... .......... 76% 95.3M 2s 2024-09-12T12:57:45.622 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:128400K .......... .......... .......... .......... .......... 76% 13.3M 2s 2024-09-12T12:57:45.622 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:128450K .......... .......... .......... .......... .......... 76% 108M 2s 2024-09-12T12:57:45.627 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:128500K .......... .......... .......... .......... .......... 76% 8.70M 2s 2024-09-12T12:57:45.636 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:128550K .......... .......... .......... .......... .......... 76% 5.81M 2s 2024-09-12T12:57:45.637 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:128600K .......... .......... .......... .......... .......... 76% 40.4M 2s 2024-09-12T12:57:45.638 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:128650K .......... .......... .......... .......... .......... 76% 59.9M 2s 2024-09-12T12:57:45.638 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:128700K .......... .......... .......... .......... .......... 76% 99.9M 2s 2024-09-12T12:57:45.642 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:128750K .......... .......... .......... .......... .......... 76% 12.7M 2s 2024-09-12T12:57:45.648 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:128800K .......... .......... .......... .......... .......... 76% 8.66M 2s 2024-09-12T12:57:45.656 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:128850K .......... .......... .......... .......... .......... 76% 5.88M 2s 2024-09-12T12:57:45.657 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:128900K .......... .......... .......... .......... .......... 76% 40.5M 2s 2024-09-12T12:57:45.659 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:128950K .......... .......... .......... .......... .......... 76% 59.7M 2s 2024-09-12T12:57:45.659 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:129000K .......... .......... .......... .......... .......... 76% 108M 2s 2024-09-12T12:57:45.662 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:129050K .......... .......... .......... .......... .......... 76% 13.9M 2s 2024-09-12T12:57:45.664 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:129100K .......... .......... .......... .......... .......... 76% 40.4M 2s 2024-09-12T12:57:45.669 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:129150K .......... .......... .......... .......... .......... 76% 8.95M 2s 2024-09-12T12:57:45.677 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:129200K .......... .......... .......... .......... .......... 76% 5.50M 2s 2024-09-12T12:57:45.679 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:129250K .......... .......... .......... .......... .......... 76% 74.8M 2s 2024-09-12T12:57:45.679 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:129300K .......... .......... .......... .......... .......... 76% 203M 2s 2024-09-12T12:57:45.682 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:129350K .......... .......... .......... .......... .......... 76% 13.0M 2s 2024-09-12T12:57:45.683 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:129400K .......... .......... .......... .......... .......... 76% 35.6M 2s 2024-09-12T12:57:45.689 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:129450K .......... .......... .......... .......... .......... 76% 9.77M 2s 2024-09-12T12:57:45.697 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:129500K .......... .......... .......... .......... .......... 76% 5.96M 2s 2024-09-12T12:57:45.700 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:129550K .......... .......... .......... .......... .......... 76% 25.0M 2s 2024-09-12T12:57:45.700 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:129600K .......... .......... .......... .......... .......... 76% 177M 2s 2024-09-12T12:57:45.700 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:129650K .......... .......... .......... .......... .......... 76% 61.1M 2s 2024-09-12T12:57:45.703 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:129700K .......... .......... .......... .......... .......... 76% 16.8M 2s 2024-09-12T12:57:45.709 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:129750K .......... .......... .......... .......... .......... 77% 8.05M 2s 2024-09-12T12:57:45.717 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:129800K .......... .......... .......... .......... .......... 77% 5.90M 2s 2024-09-12T12:57:45.720 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:129850K .......... .......... .......... .......... .......... 77% 30.9M 2s 2024-09-12T12:57:45.720 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:129900K .......... .......... .......... .......... .......... 77% 156M 2s 2024-09-12T12:57:45.720 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:129950K .......... .......... .......... .......... .......... 77% 46.3M 2s 2024-09-12T12:57:45.723 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:130000K .......... .......... .......... .......... .......... 77% 15.9M 2s 2024-09-12T12:57:45.729 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:130050K .......... .......... .......... .......... .......... 77% 8.05M 2s 2024-09-12T12:57:45.737 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:130100K .......... .......... .......... .......... .......... 77% 6.01M 2s 2024-09-12T12:57:45.740 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:130150K .......... .......... .......... .......... .......... 77% 41.6M 2s 2024-09-12T12:57:45.740 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:130200K .......... .......... .......... .......... .......... 77% 78.7M 2s 2024-09-12T12:57:45.740 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:130250K .......... .......... .......... .......... .......... 77% 139M 2s 2024-09-12T12:57:45.743 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:130300K .......... .......... .......... .......... .......... 77% 12.6M 2s 2024-09-12T12:57:45.749 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:130350K .......... .......... .......... .......... .......... 77% 8.18M 2s 2024-09-12T12:57:45.758 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:130400K .......... .......... .......... .......... .......... 77% 5.78M 2s 2024-09-12T12:57:45.760 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:130450K .......... .......... .......... .......... .......... 77% 43.8M 2s 2024-09-12T12:57:45.760 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:130500K .......... .......... .......... .......... .......... 77% 106M 2s 2024-09-12T12:57:45.760 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:130550K .......... .......... .......... .......... .......... 77% 128M 2s 2024-09-12T12:57:45.766 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:130600K .......... .......... .......... .......... .......... 77% 7.95M 2s 2024-09-12T12:57:45.766 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:130650K .......... .......... .......... .......... .......... 77% 170M 2s 2024-09-12T12:57:45.771 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:130700K .......... .......... .......... .......... .......... 77% 10.3M 2s 2024-09-12T12:57:45.779 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:130750K .......... .......... .......... .......... .......... 77% 5.88M 2s 2024-09-12T12:57:45.781 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:130800K .......... .......... .......... .......... .......... 77% 44.0M 2s 2024-09-12T12:57:45.781 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:130850K .......... .......... .......... .......... .......... 77% 144M 2s 2024-09-12T12:57:45.782 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:130900K .......... .......... .......... .......... .......... 77% 27.4M 2s 2024-09-12T12:57:45.789 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:130950K .......... .......... .......... .......... .......... 77% 8.30M 2s 2024-09-12T12:57:45.792 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:131000K .......... .......... .......... .......... .......... 77% 14.0M 2s 2024-09-12T12:57:45.800 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:131050K .......... .......... .......... .......... .......... 77% 6.01M 2s 2024-09-12T12:57:45.802 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:131100K .......... .......... .......... .......... .......... 77% 32.5M 2s 2024-09-12T12:57:45.802 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:131150K .......... .......... .......... .......... .......... 77% 174M 2s 2024-09-12T12:57:45.806 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:131200K .......... .......... .......... .......... .......... 77% 10.9M 2s 2024-09-12T12:57:45.810 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:131250K .......... .......... .......... .......... .......... 77% 15.4M 2s 2024-09-12T12:57:45.813 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:131300K .......... .......... .......... .......... .......... 77% 14.9M 2s 2024-09-12T12:57:45.813 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:131350K .......... .......... .......... .......... .......... 77% 138M 2s 2024-09-12T12:57:45.821 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:131400K .......... .......... .......... .......... .......... 77% 6.03M 2s 2024-09-12T12:57:45.824 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:131450K .......... .......... .......... .......... .......... 78% 32.8M 2s 2024-09-12T12:57:45.824 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:131500K .......... .......... .......... .......... .......... 78% 157M 2s 2024-09-12T12:57:45.831 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:131550K .......... .......... .......... .......... .......... 78% 6.83M 2s 2024-09-12T12:57:45.831 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:131600K .......... .......... .......... .......... .......... 78% 116M 2s 2024-09-12T12:57:45.834 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:131650K .......... .......... .......... .......... .......... 78% 13.6M 2s 2024-09-12T12:57:45.842 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:131700K .......... .......... .......... .......... .......... 78% 6.03M 2s 2024-09-12T12:57:45.845 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:131750K .......... .......... .......... .......... .......... 78% 40.5M 2s 2024-09-12T12:57:45.845 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:131800K .......... .......... .......... .......... .......... 78% 131M 2s 2024-09-12T12:57:45.848 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:131850K .......... .......... .......... .......... .......... 78% 9.91M 2s 2024-09-12T12:57:45.852 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:131900K .......... .......... .......... .......... .......... 78% 17.4M 2s 2024-09-12T12:57:45.855 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:131950K .......... .......... .......... .......... .......... 78% 13.5M 2s 2024-09-12T12:57:45.866 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:132000K .......... .......... .......... .......... .......... 78% 6.04M 2s 2024-09-12T12:57:45.866 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:132050K .......... .......... .......... .......... .......... 78% 106M 2s 2024-09-12T12:57:45.866 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:132100K .......... .......... .......... .......... .......... 78% 47.7M 2s 2024-09-12T12:57:45.866 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:132150K .......... .......... .......... .......... .......... 78% 121M 2s 2024-09-12T12:57:45.873 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:132200K .......... .......... .......... .......... .......... 78% 6.60M 2s 2024-09-12T12:57:45.873 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:132250K .......... .......... .......... .......... .......... 78% 175M 2s 2024-09-12T12:57:45.876 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:132300K .......... .......... .......... .......... .......... 78% 13.8M 2s 2024-09-12T12:57:45.884 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:132350K .......... .......... .......... .......... .......... 78% 5.94M 2s 2024-09-12T12:57:45.887 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:132400K .......... .......... .......... .......... .......... 78% 45.5M 2s 2024-09-12T12:57:45.887 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:132450K .......... .......... .......... .......... .......... 78% 129M 2s 2024-09-12T12:57:45.890 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:132500K .......... .......... .......... .......... .......... 78% 10.4M 2s 2024-09-12T12:57:45.895 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:132550K .......... .......... .......... .......... .......... 78% 15.8M 2s 2024-09-12T12:57:45.895 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:132600K .......... .......... .......... .......... .......... 78% 164M 2s 2024-09-12T12:57:45.898 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:132650K .......... .......... .......... .......... .......... 78% 14.3M 2s 2024-09-12T12:57:45.905 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:132700K .......... .......... .......... .......... .......... 78% 6.03M 2s 2024-09-12T12:57:45.908 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:132750K .......... .......... .......... .......... .......... 78% 37.2M 2s 2024-09-12T12:57:45.908 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:132800K .......... .......... .......... .......... .......... 78% 123M 2s 2024-09-12T12:57:45.916 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:132850K .......... .......... .......... .......... .......... 78% 6.25M 2s 2024-09-12T12:57:45.916 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:132900K .......... .......... .......... .......... .......... 78% 179M 2s 2024-09-12T12:57:45.956 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:132950K .......... .......... .......... .......... .......... 78% 1.50M 2s 2024-09-12T12:57:45.956 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:133000K .......... .......... .......... .......... .......... 78% 182M 2s 2024-09-12T12:57:45.956 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:133050K .......... .......... .......... .......... .......... 78% 202M 2s 2024-09-12T12:57:45.956 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:133100K .......... .......... .......... .......... .......... 78% 204M 2s 2024-09-12T12:57:45.956 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:133150K .......... .......... .......... .......... .......... 79% 134M 2s 2024-09-12T12:57:45.956 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:133200K .......... .......... .......... .......... .......... 79% 202M 2s 2024-09-12T12:57:45.956 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:133250K .......... .......... .......... .......... .......... 79% 210M 2s 2024-09-12T12:57:45.956 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:133300K .......... .......... .......... .......... .......... 79% 205M 2s 2024-09-12T12:57:45.957 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:133350K .......... .......... .......... .......... .......... 79% 159M 2s 2024-09-12T12:57:45.958 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:133400K .......... .......... .......... .......... .......... 79% 6.62M 2s 2024-09-12T12:57:45.958 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:133450K .......... .......... .......... .......... .......... 79% 150M 2s 2024-09-12T12:57:45.961 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:133500K .......... .......... .......... .......... .......... 79% 13.8M 2s 2024-09-12T12:57:45.969 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:133550K .......... .......... .......... .......... .......... 79% 6.39M 2s 2024-09-12T12:57:45.976 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:133600K .......... .......... .......... .......... .......... 79% 7.02M 2s 2024-09-12T12:57:45.979 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:133650K .......... .......... .......... .......... .......... 79% 19.2M 2s 2024-09-12T12:57:45.979 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:133700K .......... .......... .......... .......... .......... 79% 193M 2s 2024-09-12T12:57:45.983 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:133750K .......... .......... .......... .......... .......... 79% 11.3M 2s 2024-09-12T12:57:45.990 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:133800K .......... .......... .......... .......... .......... 79% 6.82M 2s 2024-09-12T12:57:45.996 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:133850K .......... .......... .......... .......... .......... 79% 7.95M 2s 2024-09-12T12:57:46.000 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:133900K .......... .......... .......... .......... .......... 79% 17.6M 2s 2024-09-12T12:57:46.003 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:133950K .......... .......... .......... .......... .......... 79% 12.3M 2s 2024-09-12T12:57:46.009 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:134000K .......... .......... .......... .......... .......... 79% 7.45M 2s 2024-09-12T12:57:46.017 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:134050K .......... .......... .......... .......... .......... 79% 6.93M 2s 2024-09-12T12:57:46.020 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:134100K .......... .......... .......... .......... .......... 79% 19.7M 2s 2024-09-12T12:57:46.020 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:134150K .......... .......... .......... .......... .......... 79% 146M 2s 2024-09-12T12:57:46.023 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:134200K .......... .......... .......... .......... .......... 79% 12.1M 2s 2024-09-12T12:57:46.031 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:134250K .......... .......... .......... .......... .......... 79% 6.79M 2s 2024-09-12T12:57:46.037 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:134300K .......... .......... .......... .......... .......... 79% 7.54M 2s 2024-09-12T12:57:46.041 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:134350K .......... .......... .......... .......... .......... 79% 19.3M 2s 2024-09-12T12:57:46.044 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:134400K .......... .......... .......... .......... .......... 79% 11.9M 2s 2024-09-12T12:57:46.050 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:134450K .......... .......... .......... .......... .......... 79% 7.33M 2s 2024-09-12T12:57:46.057 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:134500K .......... .......... .......... .......... .......... 79% 7.16M 2s 2024-09-12T12:57:46.061 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:134550K .......... .......... .......... .......... .......... 79% 20.1M 2s 2024-09-12T12:57:46.061 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:134600K .......... .......... .......... .......... .......... 79% 150M 2s 2024-09-12T12:57:46.064 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:134650K .......... .......... .......... .......... .......... 79% 11.6M 2s 2024-09-12T12:57:46.071 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:134700K .......... .......... .......... .......... .......... 79% 7.36M 2s 2024-09-12T12:57:46.081 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:134750K .......... .......... .......... .......... .......... 79% 5.35M 2s 2024-09-12T12:57:46.081 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:134800K .......... .......... .......... .......... .......... 79% 165M 2s 2024-09-12T12:57:46.083 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:134850K .......... .......... .......... .......... .......... 80% 16.9M 2s 2024-09-12T12:57:46.085 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:134900K .......... .......... .......... .......... .......... 80% 35.5M 2s 2024-09-12T12:57:46.091 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:134950K .......... .......... .......... .......... .......... 80% 6.93M 2s 2024-09-12T12:57:46.102 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:135000K .......... .......... .......... .......... .......... 80% 5.50M 2s 2024-09-12T12:57:46.102 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:135050K .......... .......... .......... .......... .......... 80% 173M 2s 2024-09-12T12:57:46.103 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:135100K .......... .......... .......... .......... .......... 80% 17.3M 2s 2024-09-12T12:57:46.111 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:135150K .......... .......... .......... .......... .......... 80% 6.35M 2s 2024-09-12T12:57:46.118 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:135200K .......... .......... .......... .......... .......... 80% 6.79M 2s 2024-09-12T12:57:46.122 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:135250K .......... .......... .......... .......... .......... 80% 18.3M 2s 2024-09-12T12:57:46.122 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:135300K .......... .......... .......... .......... .......... 80% 199M 2s 2024-09-12T12:57:46.125 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:135350K .......... .......... .......... .......... .......... 80% 12.4M 2s 2024-09-12T12:57:46.132 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:135400K .......... .......... .......... .......... .......... 80% 7.45M 2s 2024-09-12T12:57:46.139 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:135450K .......... .......... .......... .......... .......... 80% 6.81M 2s 2024-09-12T12:57:46.142 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:135500K .......... .......... .......... .......... .......... 80% 19.9M 2s 2024-09-12T12:57:46.144 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:135550K .......... .......... .......... .......... .......... 80% 17.9M 2s 2024-09-12T12:57:46.146 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:135600K .......... .......... .......... .......... .......... 80% 34.0M 2s 2024-09-12T12:57:46.152 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:135650K .......... .......... .......... .......... .......... 80% 7.15M 2s 2024-09-12T12:57:46.162 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:135700K .......... .......... .......... .......... .......... 80% 5.39M 2s 2024-09-12T12:57:46.163 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:135750K .......... .......... .......... .......... .......... 80% 129M 2s 2024-09-12T12:57:46.165 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:135800K .......... .......... .......... .......... .......... 80% 16.8M 2s 2024-09-12T12:57:46.173 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:135850K .......... .......... .......... .......... .......... 80% 6.45M 2s 2024-09-12T12:57:46.180 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:135900K .......... .......... .......... .......... .......... 80% 6.80M 2s 2024-09-12T12:57:46.183 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:135950K .......... .......... .......... .......... .......... 80% 20.4M 2s 2024-09-12T12:57:46.183 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:136000K .......... .......... .......... .......... .......... 80% 161M 2s 2024-09-12T12:57:46.186 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:136050K .......... .......... .......... .......... .......... 80% 12.5M 2s 2024-09-12T12:57:46.193 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:136100K .......... .......... .......... .......... .......... 80% 7.13M 2s 2024-09-12T12:57:46.200 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:136150K .......... .......... .......... .......... .......... 80% 6.81M 2s 2024-09-12T12:57:46.203 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:136200K .......... .......... .......... .......... .......... 80% 23.1M 2s 2024-09-12T12:57:46.203 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:136250K .......... .......... .......... .......... .......... 80% 188M 2s 2024-09-12T12:57:46.207 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:136300K .......... .......... .......... .......... .......... 80% 12.2M 2s 2024-09-12T12:57:46.213 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:136350K .......... .......... .......... .......... .......... 80% 7.13M 2s 2024-09-12T12:57:46.224 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:136400K .......... .......... .......... .......... .......... 80% 5.30M 2s 2024-09-12T12:57:46.224 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:136450K .......... .......... .......... .......... .......... 80% 193M 2s 2024-09-12T12:57:46.225 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:136500K .......... .......... .......... .......... .......... 81% 17.4M 2s 2024-09-12T12:57:46.234 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:136550K .......... .......... .......... .......... .......... 81% 6.37M 2s 2024-09-12T12:57:46.234 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:136600K .......... .......... .......... .......... .......... 81% 136M 2s 2024-09-12T12:57:46.244 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:136650K .......... .......... .......... .......... .......... 81% 5.20M 2s 2024-09-12T12:57:46.244 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:136700K .......... .......... .......... .......... .......... 81% 202M 2s 2024-09-12T12:57:46.248 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:136750K .......... .......... .......... .......... .......... 81% 12.8M 2s 2024-09-12T12:57:46.255 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:136800K .......... .......... .......... .......... .......... 81% 6.85M 2s 2024-09-12T12:57:46.261 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:136850K .......... .......... .......... .......... .......... 81% 7.20M 2s 2024-09-12T12:57:46.264 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:136900K .......... .......... .......... .......... .......... 81% 21.1M 2s 2024-09-12T12:57:46.264 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:136950K .......... .......... .......... .......... .......... 81% 137M 2s 2024-09-12T12:57:46.268 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:137000K .......... .......... .......... .......... .......... 81% 12.7M 2s 2024-09-12T12:57:46.275 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:137050K .......... .......... .......... .......... .......... 81% 6.93M 2s 2024-09-12T12:57:46.282 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:137100K .......... .......... .......... .......... .......... 81% 6.89M 2s 2024-09-12T12:57:46.285 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:137150K .......... .......... .......... .......... .......... 81% 21.4M 2s 2024-09-12T12:57:46.286 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:137200K .......... .......... .......... .......... .......... 81% 19.3M 2s 2024-09-12T12:57:46.288 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:137250K .......... .......... .......... .......... .......... 81% 35.8M 2s 2024-09-12T12:57:46.295 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:137300K .......... .......... .......... .......... .......... 81% 6.98M 2s 2024-09-12T12:57:46.305 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:137350K .......... .......... .......... .......... .......... 81% 5.40M 2s 2024-09-12T12:57:46.305 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:137400K .......... .......... .......... .......... .......... 81% 93.6M 2s 2024-09-12T12:57:46.307 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:137450K .......... .......... .......... .......... .......... 81% 17.8M 2s 2024-09-12T12:57:46.308 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:137500K .......... .......... .......... .......... .......... 81% 40.0M 2s 2024-09-12T12:57:46.322 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:137550K .......... .......... .......... .......... .......... 81% 3.52M 2s 2024-09-12T12:57:46.325 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:137600K .......... .......... .......... .......... .......... 81% 21.4M 2s 2024-09-12T12:57:46.326 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:137650K .......... .......... .......... .......... .......... 81% 162M 2s 2024-09-12T12:57:46.329 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:137700K .......... .......... .......... .......... .......... 81% 12.8M 2s 2024-09-12T12:57:46.336 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:137750K .......... .......... .......... .......... .......... 81% 6.96M 2s 2024-09-12T12:57:46.342 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:137800K .......... .......... .......... .......... .......... 81% 6.98M 2s 2024-09-12T12:57:46.348 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:137850K .......... .......... .......... .......... .......... 81% 19.4M 2s 2024-09-12T12:57:46.348 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:137900K .......... .......... .......... .......... .......... 81% 201M 2s 2024-09-12T12:57:46.350 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:137950K .......... .......... .......... .......... .......... 81% 12.3M 2s 2024-09-12T12:57:46.357 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:138000K .......... .......... .......... .......... .......... 81% 7.14M 2s 2024-09-12T12:57:46.366 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:138050K .......... .......... .......... .......... .......... 81% 5.43M 2s 2024-09-12T12:57:46.367 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:138100K .......... .......... .......... .......... .......... 81% 148M 1s 2024-09-12T12:57:46.368 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:138150K .......... .......... .......... .......... .......... 81% 19.8M 1s 2024-09-12T12:57:46.370 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:138200K .......... .......... .......... .......... .......... 82% 29.4M 1s 2024-09-12T12:57:46.376 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:138250K .......... .......... .......... .......... .......... 82% 7.01M 1s 2024-09-12T12:57:46.387 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:138300K .......... .......... .......... .......... .......... 82% 5.30M 1s 2024-09-12T12:57:46.387 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:138350K .......... .......... .......... .......... .......... 82% 143M 1s 2024-09-12T12:57:46.390 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:138400K .......... .......... .......... .......... .......... 82% 13.3M 1s 2024-09-12T12:57:46.397 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:138450K .......... .......... .......... .......... .......... 82% 7.17M 1s 2024-09-12T12:57:46.404 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:138500K .......... .......... .......... .......... .......... 82% 6.81M 1s 2024-09-12T12:57:46.408 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:138550K .......... .......... .......... .......... .......... 82% 20.4M 1s 2024-09-12T12:57:46.408 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:138600K .......... .......... .......... .......... .......... 82% 195M 1s 2024-09-12T12:57:46.412 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:138650K .......... .......... .......... .......... .......... 82% 12.2M 1s 2024-09-12T12:57:46.417 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:138700K .......... .......... .......... .......... .......... 82% 7.55M 1s 2024-09-12T12:57:46.429 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:138750K .......... .......... .......... .......... .......... 82% 5.26M 1s 2024-09-12T12:57:46.429 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:138800K .......... .......... .......... .......... .......... 82% 140M 1s 2024-09-12T12:57:46.429 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:138850K .......... .......... .......... .......... .......... 82% 19.7M 1s 2024-09-12T12:57:46.432 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:138900K .......... .......... .......... .......... .......... 82% 25.8M 1s 2024-09-12T12:57:46.438 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:138950K .......... .......... .......... .......... .......... 82% 7.37M 1s 2024-09-12T12:57:46.448 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:139000K .......... .......... .......... .......... .......... 82% 5.40M 1s 2024-09-12T12:57:46.448 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:139050K .......... .......... .......... .......... .......... 82% 176M 1s 2024-09-12T12:57:46.449 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:139100K .......... .......... .......... .......... .......... 82% 17.8M 1s 2024-09-12T12:57:46.458 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:139150K .......... .......... .......... .......... .......... 82% 6.01M 1s 2024-09-12T12:57:46.465 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:139200K .......... .......... .......... .......... .......... 82% 6.58M 1s 2024-09-12T12:57:46.469 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:139250K .......... .......... .......... .......... .......... 82% 24.1M 1s 2024-09-12T12:57:46.469 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:139300K .......... .......... .......... .......... .......... 82% 176M 1s 2024-09-12T12:57:46.472 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:139350K .......... .......... .......... .......... .......... 82% 12.6M 1s 2024-09-12T12:57:46.478 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:139400K .......... .......... .......... .......... .......... 82% 7.13M 1s 2024-09-12T12:57:46.485 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:139450K .......... .......... .......... .......... .......... 82% 6.67M 1s 2024-09-12T12:57:46.489 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:139500K .......... .......... .......... .......... .......... 82% 23.0M 1s 2024-09-12T12:57:46.489 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:139550K .......... .......... .......... .......... .......... 82% 171M 1s 2024-09-12T12:57:46.492 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:139600K .......... .......... .......... .......... .......... 82% 13.4M 1s 2024-09-12T12:57:46.499 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:139650K .......... .......... .......... .......... .......... 82% 7.13M 1s 2024-09-12T12:57:46.506 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:139700K .......... .......... .......... .......... .......... 82% 6.58M 1s 2024-09-12T12:57:46.510 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:139750K .......... .......... .......... .......... .......... 82% 20.7M 1s 2024-09-12T12:57:46.510 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:139800K .......... .......... .......... .......... .......... 82% 199M 1s 2024-09-12T12:57:46.511 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:139850K .......... .......... .......... .......... .......... 82% 22.6M 1s 2024-09-12T12:57:46.519 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:139900K .......... .......... .......... .......... .......... 83% 5.98M 1s 2024-09-12T12:57:46.526 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:139950K .......... .......... .......... .......... .......... 83% 6.41M 1s 2024-09-12T12:57:46.530 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:140000K .......... .......... .......... .......... .......... 83% 20.9M 1s 2024-09-12T12:57:46.530 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:140050K .......... .......... .......... .......... .......... 83% 196M 1s 2024-09-12T12:57:46.533 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:140100K .......... .......... .......... .......... .......... 83% 13.7M 1s 2024-09-12T12:57:46.539 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:140150K .......... .......... .......... .......... .......... 83% 7.35M 1s 2024-09-12T12:57:46.546 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:140200K .......... .......... .......... .......... .......... 83% 6.58M 1s 2024-09-12T12:57:46.551 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:140250K .......... .......... .......... .......... .......... 83% 19.4M 1s 2024-09-12T12:57:46.551 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:140300K .......... .......... .......... .......... .......... 83% 203M 1s 2024-09-12T12:57:46.553 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:140350K .......... .......... .......... .......... .......... 83% 12.8M 1s 2024-09-12T12:57:46.560 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:140400K .......... .......... .......... .......... .......... 83% 7.33M 1s 2024-09-12T12:57:46.568 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:140450K .......... .......... .......... .......... .......... 83% 6.02M 1s 2024-09-12T12:57:46.572 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:140500K .......... .......... .......... .......... .......... 83% 19.6M 1s 2024-09-12T12:57:46.572 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:140550K .......... .......... .......... .......... .......... 83% 160M 1s 2024-09-12T12:57:46.575 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:140600K .......... .......... .......... .......... .......... 83% 14.4M 1s 2024-09-12T12:57:46.581 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:140650K .......... .......... .......... .......... .......... 83% 7.71M 1s 2024-09-12T12:57:46.589 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:140700K .......... .......... .......... .......... .......... 83% 5.72M 1s 2024-09-12T12:57:46.593 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:140750K .......... .......... .......... .......... .......... 83% 20.3M 1s 2024-09-12T12:57:46.593 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:140800K .......... .......... .......... .......... .......... 83% 201M 1s 2024-09-12T12:57:46.596 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:140850K .......... .......... .......... .......... .......... 83% 14.1M 1s 2024-09-12T12:57:46.602 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:140900K .......... .......... .......... .......... .......... 83% 7.63M 1s 2024-09-12T12:57:46.610 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:140950K .......... .......... .......... .......... .......... 83% 5.53M 1s 2024-09-12T12:57:46.614 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:141000K .......... .......... .......... .......... .......... 83% 23.8M 1s 2024-09-12T12:57:46.615 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:141050K .......... .......... .......... .......... .......... 83% 166M 1s 2024-09-12T12:57:46.617 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:141100K .......... .......... .......... .......... .......... 83% 13.0M 1s 2024-09-12T12:57:46.623 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:141150K .......... .......... .......... .......... .......... 83% 7.92M 1s 2024-09-12T12:57:46.631 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:141200K .......... .......... .......... .......... .......... 83% 5.41M 1s 2024-09-12T12:57:46.635 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:141250K .......... .......... .......... .......... .......... 83% 26.7M 1s 2024-09-12T12:57:46.635 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:141300K .......... .......... .......... .......... .......... 83% 142M 1s 2024-09-12T12:57:46.638 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:141350K .......... .......... .......... .......... .......... 83% 12.5M 1s 2024-09-12T12:57:46.644 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:141400K .......... .......... .......... .......... .......... 83% 8.10M 1s 2024-09-12T12:57:46.652 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:141450K .......... .......... .......... .......... .......... 83% 5.63M 1s 2024-09-12T12:57:46.657 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:141500K .......... .......... .......... .......... .......... 83% 22.2M 1s 2024-09-12T12:57:46.657 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:141550K .......... .......... .......... .......... .......... 84% 138M 1s 2024-09-12T12:57:46.659 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:141600K .......... .......... .......... .......... .......... 84% 12.8M 1s 2024-09-12T12:57:46.665 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:141650K .......... .......... .......... .......... .......... 84% 7.68M 1s 2024-09-12T12:57:46.674 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:141700K .......... .......... .......... .......... .......... 84% 5.75M 1s 2024-09-12T12:57:46.677 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:141750K .......... .......... .......... .......... .......... 84% 21.2M 1s 2024-09-12T12:57:46.677 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:141800K .......... .......... .......... .......... .......... 84% 181M 1s 2024-09-12T12:57:46.680 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:141850K .......... .......... .......... .......... .......... 84% 12.6M 1s 2024-09-12T12:57:46.687 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:141900K .......... .......... .......... .......... .......... 84% 7.62M 1s 2024-09-12T12:57:46.699 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:141950K .......... .......... .......... .......... .......... 84% 4.79M 1s 2024-09-12T12:57:46.700 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:142000K .......... .......... .......... .......... .......... 84% 119M 1s 2024-09-12T12:57:46.700 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:142050K .......... .......... .......... .......... .......... 84% 201M 1s 2024-09-12T12:57:46.701 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:142100K .......... .......... .......... .......... .......... 84% 13.2M 1s 2024-09-12T12:57:46.708 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:142150K .......... .......... .......... .......... .......... 84% 7.62M 1s 2024-09-12T12:57:46.716 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:142200K .......... .......... .......... .......... .......... 84% 5.75M 1s 2024-09-12T12:57:46.723 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:142250K .......... .......... .......... .......... .......... 84% 21.0M 1s 2024-09-12T12:57:46.723 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:142300K .......... .......... .......... .......... .......... 84% 191M 1s 2024-09-12T12:57:46.723 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:142350K .......... .......... .......... .......... .......... 84% 12.7M 1s 2024-09-12T12:57:46.729 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:142400K .......... .......... .......... .......... .......... 84% 7.69M 1s 2024-09-12T12:57:46.737 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:142450K .......... .......... .......... .......... .......... 84% 5.65M 1s 2024-09-12T12:57:46.741 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:142500K .......... .......... .......... .......... .......... 84% 23.1M 1s 2024-09-12T12:57:46.741 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:142550K .......... .......... .......... .......... .......... 84% 99.7M 1s 2024-09-12T12:57:46.744 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:142600K .......... .......... .......... .......... .......... 84% 13.2M 1s 2024-09-12T12:57:46.751 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:142650K .......... .......... .......... .......... .......... 84% 7.46M 1s 2024-09-12T12:57:46.758 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:142700K .......... .......... .......... .......... .......... 84% 5.90M 1s 2024-09-12T12:57:46.761 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:142750K .......... .......... .......... .......... .......... 84% 23.3M 1s 2024-09-12T12:57:46.762 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:142800K .......... .......... .......... .......... .......... 84% 102M 1s 2024-09-12T12:57:46.765 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:142850K .......... .......... .......... .......... .......... 84% 12.9M 1s 2024-09-12T12:57:46.771 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:142900K .......... .......... .......... .......... .......... 84% 7.52M 1s 2024-09-12T12:57:46.779 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:142950K .......... .......... .......... .......... .......... 84% 6.07M 1s 2024-09-12T12:57:46.783 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:143000K .......... .......... .......... .......... .......... 84% 19.3M 1s 2024-09-12T12:57:46.783 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:143050K .......... .......... .......... .......... .......... 84% 105M 1s 2024-09-12T12:57:46.786 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:143100K .......... .......... .......... .......... .......... 84% 12.8M 1s 2024-09-12T12:57:46.793 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:143150K .......... .......... .......... .......... .......... 84% 7.50M 1s 2024-09-12T12:57:46.801 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:143200K .......... .......... .......... .......... .......... 84% 6.33M 1s 2024-09-12T12:57:46.803 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:143250K .......... .......... .......... .......... .......... 85% 22.6M 1s 2024-09-12T12:57:46.806 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:143300K .......... .......... .......... .......... .......... 85% 16.8M 1s 2024-09-12T12:57:46.807 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:143350K .......... .......... .......... .......... .......... 85% 30.3M 1s 2024-09-12T12:57:46.813 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:143400K .......... .......... .......... .......... .......... 85% 8.26M 1s 2024-09-12T12:57:46.821 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:143450K .......... .......... .......... .......... .......... 85% 6.19M 1s 2024-09-12T12:57:46.828 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:143500K .......... .......... .......... .......... .......... 85% 9.51M 1s 2024-09-12T12:57:46.828 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:143550K .......... .......... .......... .......... .......... 85% 90.9M 1s 2024-09-12T12:57:46.829 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:143600K .......... .......... .......... .......... .......... 85% 74.5M 1s 2024-09-12T12:57:46.834 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:143650K .......... .......... .......... .......... .......... 85% 7.66M 1s 2024-09-12T12:57:46.841 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:143700K .......... .......... .......... .......... .......... 85% 6.46M 1s 2024-09-12T12:57:46.849 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:143750K .......... .......... .......... .......... .......... 85% 9.30M 1s 2024-09-12T12:57:46.849 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:143800K .......... .......... .......... .......... .......... 85% 111M 1s 2024-09-12T12:57:46.849 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:143850K .......... .......... .......... .......... .......... 85% 95.1M 1s 2024-09-12T12:57:46.854 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:143900K .......... .......... .......... .......... .......... 85% 7.66M 1s 2024-09-12T12:57:46.869 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:143950K .......... .......... .......... .......... .......... 85% 3.89M 1s 2024-09-12T12:57:46.869 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:144000K .......... .......... .......... .......... .......... 85% 107M 1s 2024-09-12T12:57:46.870 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:144050K .......... .......... .......... .......... .......... 85% 108M 1s 2024-09-12T12:57:46.870 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:144100K .......... .......... .......... .......... .......... 85% 83.4M 1s 2024-09-12T12:57:46.874 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:144150K .......... .......... .......... .......... .......... 85% 7.67M 1s 2024-09-12T12:57:46.881 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:144200K .......... .......... .......... .......... .......... 85% 6.37M 1s 2024-09-12T12:57:46.889 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:144250K .......... .......... .......... .......... .......... 85% 9.26M 1s 2024-09-12T12:57:46.889 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:144300K .......... .......... .......... .......... .......... 85% 112M 1s 2024-09-12T12:57:46.889 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:144350K .......... .......... .......... .......... .......... 85% 91.6M 1s 2024-09-12T12:57:46.894 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:144400K .......... .......... .......... .......... .......... 85% 7.71M 1s 2024-09-12T12:57:46.910 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:144450K .......... .......... .......... .......... .......... 85% 3.84M 1s 2024-09-12T12:57:46.910 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:144500K .......... .......... .......... .......... .......... 85% 110M 1s 2024-09-12T12:57:46.910 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:144550K .......... .......... .......... .......... .......... 85% 94.5M 1s 2024-09-12T12:57:46.910 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:144600K .......... .......... .......... .......... .......... 85% 108M 1s 2024-09-12T12:57:46.914 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:144650K .......... .......... .......... .......... .......... 85% 7.80M 1s 2024-09-12T12:57:46.924 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:144700K .......... .......... .......... .......... .......... 85% 6.09M 1s 2024-09-12T12:57:46.930 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:144750K .......... .......... .......... .......... .......... 85% 9.54M 1s 2024-09-12T12:57:46.930 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:144800K .......... .......... .......... .......... .......... 85% 104M 1s 2024-09-12T12:57:46.930 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:144850K .......... .......... .......... .......... .......... 85% 112M 1s 2024-09-12T12:57:46.935 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:144900K .......... .......... .......... .......... .......... 85% 7.97M 1s 2024-09-12T12:57:46.943 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:144950K .......... .......... .......... .......... .......... 86% 5.90M 1s 2024-09-12T12:57:46.950 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:145000K .......... .......... .......... .......... .......... 86% 10.3M 1s 2024-09-12T12:57:46.950 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:145050K .......... .......... .......... .......... .......... 86% 105M 1s 2024-09-12T12:57:46.950 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:145100K .......... .......... .......... .......... .......... 86% 111M 1s 2024-09-12T12:57:46.955 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:145150K .......... .......... .......... .......... .......... 86% 7.58M 1s 2024-09-12T12:57:46.963 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:145200K .......... .......... .......... .......... .......... 86% 5.97M 1s 2024-09-12T12:57:46.971 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:145250K .......... .......... .......... .......... .......... 86% 10.1M 1s 2024-09-12T12:57:46.971 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:145300K .......... .......... .......... .......... .......... 86% 108M 1s 2024-09-12T12:57:46.971 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:145350K .......... .......... .......... .......... .......... 86% 92.9M 1s 2024-09-12T12:57:46.971 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:145400K .......... .......... .......... .......... .......... 86% 114M 1s 2024-09-12T12:57:46.976 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:145450K .......... .......... .......... .......... .......... 86% 7.88M 1s 2024-09-12T12:57:46.984 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:145500K .......... .......... .......... .......... .......... 86% 6.11M 1s 2024-09-12T12:57:46.990 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:145550K .......... .......... .......... .......... .......... 86% 9.70M 1s 2024-09-12T12:57:46.990 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:145600K .......... .......... .......... .......... .......... 86% 106M 1s 2024-09-12T12:57:46.990 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:145650K .......... .......... .......... .......... .......... 86% 116M 1s 2024-09-12T12:57:46.996 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:145700K .......... .......... .......... .......... .......... 86% 7.83M 1s 2024-09-12T12:57:47.004 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:145750K .......... .......... .......... .......... .......... 86% 5.98M 1s 2024-09-12T12:57:47.011 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:145800K .......... .......... .......... .......... .......... 86% 9.56M 1s 2024-09-12T12:57:47.012 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:145850K .......... .......... .......... .......... .......... 86% 106M 1s 2024-09-12T12:57:47.012 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:145900K .......... .......... .......... .......... .......... 86% 103M 1s 2024-09-12T12:57:47.016 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:145950K .......... .......... .......... .......... .......... 86% 8.20M 1s 2024-09-12T12:57:47.024 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:146000K .......... .......... .......... .......... .......... 86% 6.02M 1s 2024-09-12T12:57:47.031 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:146050K .......... .......... .......... .......... .......... 86% 9.76M 1s 2024-09-12T12:57:47.031 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:146100K .......... .......... .......... .......... .......... 86% 109M 1s 2024-09-12T12:57:47.032 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:146150K .......... .......... .......... .......... .......... 86% 90.3M 1s 2024-09-12T12:57:47.036 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:146200K .......... .......... .......... .......... .......... 86% 8.04M 1s 2024-09-12T12:57:47.045 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:146250K .......... .......... .......... .......... .......... 86% 5.98M 1s 2024-09-12T12:57:47.052 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:146300K .......... .......... .......... .......... .......... 86% 9.66M 1s 2024-09-12T12:57:47.052 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:146350K .......... .......... .......... .......... .......... 86% 89.8M 1s 2024-09-12T12:57:47.052 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:146400K .......... .......... .......... .......... .......... 86% 109M 1s 2024-09-12T12:57:47.052 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:146450K .......... .......... .......... .......... .......... 86% 102M 1s 2024-09-12T12:57:47.057 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:146500K .......... .......... .......... .......... .......... 86% 7.97M 1s 2024-09-12T12:57:47.065 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:146550K .......... .......... .......... .......... .......... 86% 6.11M 1s 2024-09-12T12:57:47.072 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:146600K .......... .......... .......... .......... .......... 86% 9.60M 1s 2024-09-12T12:57:47.072 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:146650K .......... .......... .......... .......... .......... 87% 106M 1s 2024-09-12T12:57:47.072 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:146700K .......... .......... .......... .......... .......... 87% 111M 1s 2024-09-12T12:57:47.077 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:146750K .......... .......... .......... .......... .......... 87% 8.15M 1s 2024-09-12T12:57:47.085 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:146800K .......... .......... .......... .......... .......... 87% 5.93M 1s 2024-09-12T12:57:47.092 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:146850K .......... .......... .......... .......... .......... 87% 9.45M 1s 2024-09-12T12:57:47.092 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:146900K .......... .......... .......... .......... .......... 87% 102M 1s 2024-09-12T12:57:47.092 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:146950K .......... .......... .......... .......... .......... 87% 98.7M 1s 2024-09-12T12:57:47.098 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:147000K .......... .......... .......... .......... .......... 87% 8.22M 1s 2024-09-12T12:57:47.105 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:147050K .......... .......... .......... .......... .......... 87% 6.12M 1s 2024-09-12T12:57:47.113 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:147100K .......... .......... .......... .......... .......... 87% 9.41M 1s 2024-09-12T12:57:47.113 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:147150K .......... .......... .......... .......... .......... 87% 84.0M 1s 2024-09-12T12:57:47.113 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:147200K .......... .......... .......... .......... .......... 87% 114M 1s 2024-09-12T12:57:47.113 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:147250K .......... .......... .......... .......... .......... 87% 86.1M 1s 2024-09-12T12:57:47.126 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:147300K .......... .......... .......... .......... .......... 87% 3.55M 1s 2024-09-12T12:57:47.134 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:147350K .......... .......... .......... .......... .......... 87% 9.18M 1s 2024-09-12T12:57:47.134 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:147400K .......... .......... .......... .......... .......... 87% 94.6M 1s 2024-09-12T12:57:47.134 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:147450K .......... .......... .......... .......... .......... 87% 107M 1s 2024-09-12T12:57:47.134 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:147500K .......... .......... .......... .......... .......... 87% 115M 1s 2024-09-12T12:57:47.138 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:147550K .......... .......... .......... .......... .......... 87% 8.03M 1s 2024-09-12T12:57:47.146 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:147600K .......... .......... .......... .......... .......... 87% 6.21M 1s 2024-09-12T12:57:47.154 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:147650K .......... .......... .......... .......... .......... 87% 9.51M 1s 2024-09-12T12:57:47.154 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:147700K .......... .......... .......... .......... .......... 87% 108M 1s 2024-09-12T12:57:47.154 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:147750K .......... .......... .......... .......... .......... 87% 105M 1s 2024-09-12T12:57:47.159 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:147800K .......... .......... .......... .......... .......... 87% 7.61M 1s 2024-09-12T12:57:47.167 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:147850K .......... .......... .......... .......... .......... 87% 6.25M 1s 2024-09-12T12:57:47.174 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:147900K .......... .......... .......... .......... .......... 87% 9.44M 1s 2024-09-12T12:57:47.174 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:147950K .......... .......... .......... .......... .......... 87% 91.9M 1s 2024-09-12T12:57:47.174 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:148000K .......... .......... .......... .......... .......... 87% 122M 1s 2024-09-12T12:57:47.179 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:148050K .......... .......... .......... .......... .......... 87% 8.81M 1s 2024-09-12T12:57:47.187 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:148100K .......... .......... .......... .......... .......... 87% 5.79M 1s 2024-09-12T12:57:47.194 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:148150K .......... .......... .......... .......... .......... 87% 9.36M 1s 2024-09-12T12:57:47.194 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:148200K .......... .......... .......... .......... .......... 87% 93.1M 1s 2024-09-12T12:57:47.195 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:148250K .......... .......... .......... .......... .......... 87% 109M 1s 2024-09-12T12:57:47.195 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:148300K .......... .......... .......... .......... .......... 88% 112M 1s 2024-09-12T12:57:47.208 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:148350K .......... .......... .......... .......... .......... 88% 3.54M 1s 2024-09-12T12:57:47.214 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:148400K .......... .......... .......... .......... .......... 88% 10.1M 1s 2024-09-12T12:57:47.215 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:148450K .......... .......... .......... .......... .......... 88% 61.1M 1s 2024-09-12T12:57:47.215 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:148500K .......... .......... .......... .......... .......... 88% 111M 1s 2024-09-12T12:57:47.215 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:148550K .......... .......... .......... .......... .......... 88% 107M 1s 2024-09-12T12:57:47.219 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:148600K .......... .......... .......... .......... .......... 88% 8.35M 1s 2024-09-12T12:57:47.228 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:148650K .......... .......... .......... .......... .......... 88% 5.80M 1s 2024-09-12T12:57:47.235 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:148700K .......... .......... .......... .......... .......... 88% 10.2M 1s 2024-09-12T12:57:47.235 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:148750K .......... .......... .......... .......... .......... 88% 86.8M 1s 2024-09-12T12:57:47.235 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:148800K .......... .......... .......... .......... .......... 88% 116M 1s 2024-09-12T12:57:47.240 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:148850K .......... .......... .......... .......... .......... 88% 8.92M 1s 2024-09-12T12:57:47.248 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:148900K .......... .......... .......... .......... .......... 88% 5.67M 1s 2024-09-12T12:57:47.255 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:148950K .......... .......... .......... .......... .......... 88% 9.59M 1s 2024-09-12T12:57:47.256 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:149000K .......... .......... .......... .......... .......... 88% 108M 1s 2024-09-12T12:57:47.256 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:149050K .......... .......... .......... .......... .......... 88% 108M 1s 2024-09-12T12:57:47.256 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:149100K .......... .......... .......... .......... .......... 88% 112M 1s 2024-09-12T12:57:47.268 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:149150K .......... .......... .......... .......... .......... 88% 3.52M 1s 2024-09-12T12:57:47.276 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:149200K .......... .......... .......... .......... .......... 88% 9.72M 1s 2024-09-12T12:57:47.276 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:149250K .......... .......... .......... .......... .......... 88% 80.1M 1s 2024-09-12T12:57:47.276 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:149300K .......... .......... .......... .......... .......... 88% 103M 1s 2024-09-12T12:57:47.276 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:149350K .......... .......... .......... .......... .......... 88% 108M 1s 2024-09-12T12:57:47.280 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:149400K .......... .......... .......... .......... .......... 88% 8.44M 1s 2024-09-12T12:57:47.289 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:149450K .......... .......... .......... .......... .......... 88% 5.77M 1s 2024-09-12T12:57:47.296 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:149500K .......... .......... .......... .......... .......... 88% 9.78M 1s 2024-09-12T12:57:47.296 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:149550K .......... .......... .......... .......... .......... 88% 93.9M 1s 2024-09-12T12:57:47.296 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:149600K .......... .......... .......... .......... .......... 88% 112M 1s 2024-09-12T12:57:47.301 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:149650K .......... .......... .......... .......... .......... 88% 8.31M 1s 2024-09-12T12:57:47.309 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:149700K .......... .......... .......... .......... .......... 88% 5.98M 1s 2024-09-12T12:57:47.316 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:149750K .......... .......... .......... .......... .......... 88% 9.79M 1s 2024-09-12T12:57:47.316 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:149800K .......... .......... .......... .......... .......... 88% 109M 1s 2024-09-12T12:57:47.316 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:149850K .......... .......... .......... .......... .......... 88% 112M 1s 2024-09-12T12:57:47.316 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:149900K .......... .......... .......... .......... .......... 88% 100M 1s 2024-09-12T12:57:47.322 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:149950K .......... .......... .......... .......... .......... 88% 7.08M 1s 2024-09-12T12:57:47.329 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:150000K .......... .......... .......... .......... .......... 89% 6.62M 1s 2024-09-12T12:57:47.336 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:150050K .......... .......... .......... .......... .......... 89% 10.0M 1s 2024-09-12T12:57:47.336 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:150100K .......... .......... .......... .......... .......... 89% 109M 1s 2024-09-12T12:57:47.336 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:150150K .......... .......... .......... .......... .......... 89% 99.2M 1s 2024-09-12T12:57:47.341 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:150200K .......... .......... .......... .......... .......... 89% 7.81M 1s 2024-09-12T12:57:47.350 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:150250K .......... .......... .......... .......... .......... 89% 5.99M 1s 2024-09-12T12:57:47.357 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:150300K .......... .......... .......... .......... .......... 89% 8.63M 1s 2024-09-12T12:57:47.358 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:150350K .......... .......... .......... .......... .......... 89% 88.3M 1s 2024-09-12T12:57:47.358 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:150400K .......... .......... .......... .......... .......... 89% 112M 1s 2024-09-12T12:57:47.362 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:150450K .......... .......... .......... .......... .......... 89% 8.07M 1s 2024-09-12T12:57:47.371 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:150500K .......... .......... .......... .......... .......... 89% 5.92M 1s 2024-09-12T12:57:47.372 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:150550K .......... .......... .......... .......... .......... 89% 134M 1s 2024-09-12T12:57:47.378 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:150600K .......... .......... .......... .......... .......... 89% 9.37M 1s 2024-09-12T12:57:47.378 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:150650K .......... .......... .......... .......... .......... 89% 151M 1s 2024-09-12T12:57:47.378 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:150700K .......... .......... .......... .......... .......... 89% 193M 1s 2024-09-12T12:57:47.385 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:150750K .......... .......... .......... .......... .......... 89% 5.92M 1s 2024-09-12T12:57:47.392 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:150800K .......... .......... .......... .......... .......... 89% 7.15M 1s 2024-09-12T12:57:47.400 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:150850K .......... .......... .......... .......... .......... 89% 9.40M 1s 2024-09-12T12:57:47.400 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:150900K .......... .......... .......... .......... .......... 89% 134M 1s 2024-09-12T12:57:47.400 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:150950K .......... .......... .......... .......... .......... 89% 147M 1s 2024-09-12T12:57:47.404 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:151000K .......... .......... .......... .......... .......... 89% 7.59M 1s 2024-09-12T12:57:47.414 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:151050K .......... .......... .......... .......... .......... 89% 5.62M 1s 2024-09-12T12:57:47.414 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:151100K .......... .......... .......... .......... .......... 89% 88.5M 1s 2024-09-12T12:57:47.420 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:151150K .......... .......... .......... .......... .......... 89% 10.1M 1s 2024-09-12T12:57:47.420 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:151200K .......... .......... .......... .......... .......... 89% 185M 1s 2024-09-12T12:57:47.425 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:151250K .......... .......... .......... .......... .......... 89% 7.72M 1s 2024-09-12T12:57:47.427 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:151300K .......... .......... .......... .......... .......... 89% 21.1M 1s 2024-09-12T12:57:47.434 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:151350K .......... .......... .......... .......... .......... 89% 7.05M 1s 2024-09-12T12:57:47.441 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:151400K .......... .......... .......... .......... .......... 89% 9.49M 1s 2024-09-12T12:57:47.441 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:151450K .......... .......... .......... .......... .......... 89% 187M 1s 2024-09-12T12:57:47.441 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:151500K .......... .......... .......... .......... .......... 89% 173M 1s 2024-09-12T12:57:47.446 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:151550K .......... .......... .......... .......... .......... 89% 7.65M 1s 2024-09-12T12:57:47.455 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:151600K .......... .......... .......... .......... .......... 89% 5.93M 1s 2024-09-12T12:57:47.456 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:151650K .......... .......... .......... .......... .......... 89% 37.1M 1s 2024-09-12T12:57:47.463 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:151700K .......... .......... .......... .......... .......... 90% 10.9M 1s 2024-09-12T12:57:47.463 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:151750K .......... .......... .......... .......... .......... 90% 182M 1s 2024-09-12T12:57:47.463 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:151800K .......... .......... .......... .......... .......... 90% 197M 1s 2024-09-12T12:57:47.469 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:151850K .......... .......... .......... .......... .......... 90% 6.27M 1s 2024-09-12T12:57:47.475 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:151900K .......... .......... .......... .......... .......... 90% 7.42M 1s 2024-09-12T12:57:47.483 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:151950K .......... .......... .......... .......... .......... 90% 9.16M 1s 2024-09-12T12:57:47.483 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:152000K .......... .......... .......... .......... .......... 90% 177M 1s 2024-09-12T12:57:47.483 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:152050K .......... .......... .......... .......... .......... 90% 175M 1s 2024-09-12T12:57:47.487 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:152100K .......... .......... .......... .......... .......... 90% 8.65M 1s 2024-09-12T12:57:47.489 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:152150K .......... .......... .......... .......... .......... 90% 21.1M 1s 2024-09-12T12:57:47.496 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:152200K .......... .......... .......... .......... .......... 90% 7.22M 1s 2024-09-12T12:57:47.503 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:152250K .......... .......... .......... .......... .......... 90% 9.42M 1s 2024-09-12T12:57:47.503 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:152300K .......... .......... .......... .......... .......... 90% 192M 1s 2024-09-12T12:57:47.503 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:152350K .......... .......... .......... .......... .......... 90% 167M 1s 2024-09-12T12:57:47.509 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:152400K .......... .......... .......... .......... .......... 90% 6.50M 1s 2024-09-12T12:57:47.517 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:152450K .......... .......... .......... .......... .......... 90% 7.12M 1s 2024-09-12T12:57:47.524 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:152500K .......... .......... .......... .......... .......... 90% 9.27M 1s 2024-09-12T12:57:47.524 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:152550K .......... .......... .......... .......... .......... 90% 148M 1s 2024-09-12T12:57:47.524 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:152600K .......... .......... .......... .......... .......... 90% 207M 1s 2024-09-12T12:57:47.524 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:152650K .......... .......... .......... .......... .......... 90% 183M 1s 2024-09-12T12:57:47.530 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:152700K .......... .......... .......... .......... .......... 90% 6.50M 1s 2024-09-12T12:57:47.537 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:152750K .......... .......... .......... .......... .......... 90% 7.00M 1s 2024-09-12T12:57:47.545 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:152800K .......... .......... .......... .......... .......... 90% 8.91M 1s 2024-09-12T12:57:47.545 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:152850K .......... .......... .......... .......... .......... 90% 203M 1s 2024-09-12T12:57:47.546 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:152900K .......... .......... .......... .......... .......... 90% 202M 1s 2024-09-12T12:57:47.548 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:152950K .......... .......... .......... .......... .......... 90% 9.23M 1s 2024-09-12T12:57:47.558 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:153000K .......... .......... .......... .......... .......... 90% 5.42M 1s 2024-09-12T12:57:47.558 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:153050K .......... .......... .......... .......... .......... 90% 115M 1s 2024-09-12T12:57:47.565 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:153100K .......... .......... .......... .......... .......... 90% 9.22M 1s 2024-09-12T12:57:47.565 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:153150K .......... .......... .......... .......... .......... 90% 168M 1s 2024-09-12T12:57:47.565 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:153200K .......... .......... .......... .......... .......... 90% 188M 1s 2024-09-12T12:57:47.570 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:153250K .......... .......... .......... .......... .......... 90% 7.07M 1s 2024-09-12T12:57:47.578 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:153300K .......... .......... .......... .......... .......... 90% 6.72M 1s 2024-09-12T12:57:47.585 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:153350K .......... .......... .......... .......... .......... 91% 9.44M 1s 2024-09-12T12:57:47.585 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:153400K .......... .......... .......... .......... .......... 91% 167M 1s 2024-09-12T12:57:47.585 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:153450K .......... .......... .......... .......... .......... 91% 190M 1s 2024-09-12T12:57:47.585 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:153500K .......... .......... .......... .......... .......... 91% 199M 1s 2024-09-12T12:57:47.598 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:153550K .......... .......... .......... .......... .......... 91% 3.44M 1s 2024-09-12T12:57:47.598 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:153600K .......... .......... .......... .......... .......... 91% 127M 1s 2024-09-12T12:57:47.605 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:153650K .......... .......... .......... .......... .......... 91% 9.77M 1s 2024-09-12T12:57:47.605 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:153700K .......... .......... .......... .......... .......... 91% 169M 1s 2024-09-12T12:57:47.605 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:153750K .......... .......... .......... .......... .......... 91% 179M 1s 2024-09-12T12:57:47.609 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:153800K .......... .......... .......... .......... .......... 91% 9.20M 1s 2024-09-12T12:57:47.618 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:153850K .......... .......... .......... .......... .......... 91% 5.32M 1s 2024-09-12T12:57:47.618 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:153900K .......... .......... .......... .......... .......... 91% 164M 1s 2024-09-12T12:57:47.625 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:153950K .......... .......... .......... .......... .......... 91% 9.05M 1s 2024-09-12T12:57:47.625 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:154000K .......... .......... .......... .......... .......... 91% 198M 1s 2024-09-12T12:57:47.625 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:154050K .......... .......... .......... .......... .......... 91% 176M 1s 2024-09-12T12:57:47.631 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:154100K .......... .......... .......... .......... .......... 91% 6.75M 1s 2024-09-12T12:57:47.639 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:154150K .......... .......... .......... .......... .......... 91% 6.99M 1s 2024-09-12T12:57:47.646 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:154200K .......... .......... .......... .......... .......... 91% 8.94M 1s 2024-09-12T12:57:47.646 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:154250K .......... .......... .......... .......... .......... 91% 179M 1s 2024-09-12T12:57:47.646 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:154300K .......... .......... .......... .......... .......... 91% 207M 1s 2024-09-12T12:57:47.649 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:154350K .......... .......... .......... .......... .......... 91% 9.48M 1s 2024-09-12T12:57:47.659 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:154400K .......... .......... .......... .......... .......... 91% 5.43M 1s 2024-09-12T12:57:47.659 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:154450K .......... .......... .......... .......... .......... 91% 169M 1s 2024-09-12T12:57:47.667 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:154500K .......... .......... .......... .......... .......... 91% 8.84M 1s 2024-09-12T12:57:47.667 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:154550K .......... .......... .......... .......... .......... 91% 178M 1s 2024-09-12T12:57:47.667 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:154600K .......... .......... .......... .......... .......... 91% 190M 1s 2024-09-12T12:57:47.670 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:154650K .......... .......... .......... .......... .......... 91% 9.82M 1s 2024-09-12T12:57:47.681 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:154700K .......... .......... .......... .......... .......... 91% 5.31M 1s 2024-09-12T12:57:47.687 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:154750K .......... .......... .......... .......... .......... 91% 9.32M 1s 2024-09-12T12:57:47.687 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:154800K .......... .......... .......... .......... .......... 91% 172M 1s 2024-09-12T12:57:47.687 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:154850K .......... .......... .......... .......... .......... 91% 190M 1s 2024-09-12T12:57:47.690 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:154900K .......... .......... .......... .......... .......... 91% 9.29M 1s 2024-09-12T12:57:47.692 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:154950K .......... .......... .......... .......... .......... 91% 21.9M 1s 2024-09-12T12:57:47.700 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:155000K .......... .......... .......... .......... .......... 91% 6.68M 1s 2024-09-12T12:57:47.708 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:155050K .......... .......... .......... .......... .......... 92% 9.75M 1s 2024-09-12T12:57:47.708 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:155100K .......... .......... .......... .......... .......... 92% 175M 1s 2024-09-12T12:57:47.708 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:155150K .......... .......... .......... .......... .......... 92% 158M 1s 2024-09-12T12:57:47.710 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:155200K .......... .......... .......... .......... .......... 92% 9.54M 1s 2024-09-12T12:57:47.720 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:155250K .......... .......... .......... .......... .......... 92% 5.25M 1s 2024-09-12T12:57:47.720 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:155300K .......... .......... .......... .......... .......... 92% 50.7M 1s 2024-09-12T12:57:47.726 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:155350K .......... .......... .......... .......... .......... 92% 11.2M 1s 2024-09-12T12:57:47.727 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:155400K .......... .......... .......... .......... .......... 92% 174M 1s 2024-09-12T12:57:47.727 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:155450K .......... .......... .......... .......... .......... 92% 200M 1s 2024-09-12T12:57:47.733 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:155500K .......... .......... .......... .......... .......... 92% 6.58M 1s 2024-09-12T12:57:47.741 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:155550K .......... .......... .......... .......... .......... 92% 6.79M 1s 2024-09-12T12:57:47.747 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:155600K .......... .......... .......... .......... .......... 92% 9.82M 1s 2024-09-12T12:57:47.747 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:155650K .......... .......... .......... .......... .......... 92% 155M 1s 2024-09-12T12:57:47.747 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:155700K .......... .......... .......... .......... .......... 92% 195M 1s 2024-09-12T12:57:47.751 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:155750K .......... .......... .......... .......... .......... 92% 9.58M 1s 2024-09-12T12:57:47.761 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:155800K .......... .......... .......... .......... .......... 92% 5.08M 1s 2024-09-12T12:57:47.761 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:155850K .......... .......... .......... .......... .......... 92% 181M 1s 2024-09-12T12:57:47.767 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:155900K .......... .......... .......... .......... .......... 92% 9.71M 1s 2024-09-12T12:57:47.767 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:155950K .......... .......... .......... .......... .......... 92% 164M 1s 2024-09-12T12:57:47.767 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:156000K .......... .......... .......... .......... .......... 92% 175M 1s 2024-09-12T12:57:47.773 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:156050K .......... .......... .......... .......... .......... 92% 6.72M 1s 2024-09-12T12:57:47.782 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:156100K .......... .......... .......... .......... .......... 92% 6.54M 1s 2024-09-12T12:57:47.782 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:156150K .......... .......... .......... .......... .......... 92% 86.5M 1s 2024-09-12T12:57:47.788 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:156200K .......... .......... .......... .......... .......... 92% 10.8M 1s 2024-09-12T12:57:47.788 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:156250K .......... .......... .......... .......... .......... 92% 191M 1s 2024-09-12T12:57:47.788 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:156300K .......... .......... .......... .......... .......... 92% 192M 1s 2024-09-12T12:57:47.802 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:156350K .......... .......... .......... .......... .......... 92% 3.35M 1s 2024-09-12T12:57:47.802 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:156400K .......... .......... .......... .......... .......... 92% 180M 1s 2024-09-12T12:57:47.808 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:156450K .......... .......... .......... .......... .......... 92% 10.4M 1s 2024-09-12T12:57:47.808 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:156500K .......... .......... .......... .......... .......... 92% 141M 1s 2024-09-12T12:57:47.808 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:156550K .......... .......... .......... .......... .......... 92% 182M 1s 2024-09-12T12:57:47.812 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:156600K .......... .......... .......... .......... .......... 92% 9.02M 1s 2024-09-12T12:57:47.822 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:156650K .......... .......... .......... .......... .......... 92% 5.26M 1s 2024-09-12T12:57:47.822 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:156700K .......... .......... .......... .......... .......... 92% 181M 1s 2024-09-12T12:57:47.828 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:156750K .......... .......... .......... .......... .......... 93% 10.1M 1s 2024-09-12T12:57:47.828 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:156800K .......... .......... .......... .......... .......... 93% 194M 1s 2024-09-12T12:57:47.828 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:156850K .......... .......... .......... .......... .......... 93% 158M 1s 2024-09-12T12:57:47.834 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:156900K .......... .......... .......... .......... .......... 93% 6.78M 1s 2024-09-12T12:57:47.842 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:156950K .......... .......... .......... .......... .......... 93% 6.38M 1s 2024-09-12T12:57:47.844 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:157000K .......... .......... .......... .......... .......... 93% 17.9M 1s 2024-09-12T12:57:47.849 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:157050K .......... .......... .......... .......... .......... 93% 19.0M 1s 2024-09-12T12:57:47.849 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:157100K .......... .......... .......... .......... .......... 93% 110M 1s 2024-09-12T12:57:47.852 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:157150K .......... .......... .......... .......... .......... 93% 10.3M 1s 2024-09-12T12:57:47.855 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:157200K .......... .......... .......... .......... .......... 93% 19.9M 1s 2024-09-12T12:57:47.863 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:157250K .......... .......... .......... .......... .......... 93% 6.67M 1s 2024-09-12T12:57:47.865 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:157300K .......... .......... .......... .......... .......... 93% 18.1M 1s 2024-09-12T12:57:47.869 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:157350K .......... .......... .......... .......... .......... 93% 18.2M 1s 2024-09-12T12:57:47.869 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:157400K .......... .......... .......... .......... .......... 93% 113M 1s 2024-09-12T12:57:47.873 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:157450K .......... .......... .......... .......... .......... 93% 9.55M 1s 2024-09-12T12:57:47.875 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:157500K .......... .......... .......... .......... .......... 93% 22.9M 1s 2024-09-12T12:57:47.883 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:157550K .......... .......... .......... .......... .......... 93% 6.75M 1s 2024-09-12T12:57:47.889 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:157600K .......... .......... .......... .......... .......... 93% 9.82M 1s 2024-09-12T12:57:47.889 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:157650K .......... .......... .......... .......... .......... 93% 117M 1s 2024-09-12T12:57:47.889 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:157700K .......... .......... .......... .......... .......... 93% 171M 1s 2024-09-12T12:57:47.895 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:157750K .......... .......... .......... .......... .......... 93% 7.03M 1s 2024-09-12T12:57:47.903 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:157800K .......... .......... .......... .......... .......... 93% 6.58M 1s 2024-09-12T12:57:47.903 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:157850K .......... .......... .......... .......... .......... 93% 119M 1s 2024-09-12T12:57:47.905 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:157900K .......... .......... .......... .......... .......... 93% 18.1M 1s 2024-09-12T12:57:47.909 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:157950K .......... .......... .......... .......... .......... 93% 19.3M 1s 2024-09-12T12:57:47.910 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:158000K .......... .......... .......... .......... .......... 93% 176M 1s 2024-09-12T12:57:47.915 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:158050K .......... .......... .......... .......... .......... 93% 7.14M 1s 2024-09-12T12:57:47.924 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:158100K .......... .......... .......... .......... .......... 93% 6.54M 1s 2024-09-12T12:57:47.924 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:158150K .......... .......... .......... .......... .......... 93% 145M 1s 2024-09-12T12:57:47.927 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:158200K .......... .......... .......... .......... .......... 93% 10.1M 1s 2024-09-12T12:57:47.930 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:158250K .......... .......... .......... .......... .......... 93% 58.9M 1s 2024-09-12T12:57:47.930 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:158300K .......... .......... .......... .......... .......... 93% 41.1M 1s 2024-09-12T12:57:47.935 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:158350K .......... .......... .......... .......... .......... 93% 8.26M 1s 2024-09-12T12:57:47.945 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:158400K .......... .......... .......... .......... .......... 93% 6.39M 1s 2024-09-12T12:57:47.945 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:158450K .......... .......... .......... .......... .......... 94% 103M 1s 2024-09-12T12:57:47.950 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:158500K .......... .......... .......... .......... .......... 94% 11.1M 1s 2024-09-12T12:57:47.950 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:158550K .......... .......... .......... .......... .......... 94% 86.1M 1s 2024-09-12T12:57:47.950 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:158600K .......... .......... .......... .......... .......... 94% 100M 1s 2024-09-12T12:57:47.956 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:158650K .......... .......... .......... .......... .......... 94% 6.75M 1s 2024-09-12T12:57:47.964 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:158700K .......... .......... .......... .......... .......... 94% 6.97M 1s 2024-09-12T12:57:47.964 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:158750K .......... .......... .......... .......... .......... 94% 64.8M 1s 2024-09-12T12:57:47.969 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:158800K .......... .......... .......... .......... .......... 94% 10.7M 1s 2024-09-12T12:57:47.970 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:158850K .......... .......... .......... .......... .......... 94% 78.2M 1s 2024-09-12T12:57:47.970 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:158900K .......... .......... .......... .......... .......... 94% 126M 1s 2024-09-12T12:57:47.977 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:158950K .......... .......... .......... .......... .......... 94% 6.76M 1s 2024-09-12T12:57:47.987 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:159000K .......... .......... .......... .......... .......... 94% 7.10M 1s 2024-09-12T12:57:47.987 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:159050K .......... .......... .......... .......... .......... 94% 169M 1s 2024-09-12T12:57:47.990 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:159100K .......... .......... .......... .......... .......... 94% 10.7M 0s 2024-09-12T12:57:47.990 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:159150K .......... .......... .......... .......... .......... 94% 65.6M 0s 2024-09-12T12:57:47.994 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:159200K .......... .......... .......... .......... .......... 94% 10.1M 0s 2024-09-12T12:57:48.007 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:159250K .......... .......... .......... .......... .......... 94% 5.17M 0s 2024-09-12T12:57:48.007 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:159300K .......... .......... .......... .......... .......... 94% 105M 0s 2024-09-12T12:57:48.008 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:159350K .......... .......... .......... .......... .......... 94% 95.2M 0s 2024-09-12T12:57:48.012 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:159400K .......... .......... .......... .......... .......... 94% 11.2M 0s 2024-09-12T12:57:48.012 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:159450K .......... .......... .......... .......... .......... 94% 78.9M 0s 2024-09-12T12:57:48.012 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:159500K .......... .......... .......... .......... .......... 94% 106M 0s 2024-09-12T12:57:48.026 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:159550K .......... .......... .......... .......... .......... 94% 3.51M 0s 2024-09-12T12:57:48.026 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:159600K .......... .......... .......... .......... .......... 94% 97.4M 0s 2024-09-12T12:57:48.026 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:159650K .......... .......... .......... .......... .......... 94% 107M 0s 2024-09-12T12:57:48.031 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:159700K .......... .......... .......... .......... .......... 94% 11.2M 0s 2024-09-12T12:57:48.031 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:159750K .......... .......... .......... .......... .......... 94% 89.8M 0s 2024-09-12T12:57:48.035 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:159800K .......... .......... .......... .......... .......... 94% 9.68M 0s 2024-09-12T12:57:48.047 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:159850K .......... .......... .......... .......... .......... 94% 5.07M 0s 2024-09-12T12:57:48.047 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:159900K .......... .......... .......... .......... .......... 94% 106M 0s 2024-09-12T12:57:48.047 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:159950K .......... .......... .......... .......... .......... 94% 36.6M 0s 2024-09-12T12:57:48.053 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:160000K .......... .......... .......... .......... .......... 94% 13.3M 0s 2024-09-12T12:57:48.053 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:160050K .......... .......... .......... .......... .......... 94% 111M 0s 2024-09-12T12:57:48.055 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:160100K .......... .......... .......... .......... .......... 95% 10.5M 0s 2024-09-12T12:57:48.066 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:160150K .......... .......... .......... .......... .......... 95% 5.05M 0s 2024-09-12T12:57:48.066 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:160200K .......... .......... .......... .......... .......... 95% 91.1M 0s 2024-09-12T12:57:48.067 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:160250K .......... .......... .......... .......... .......... 95% 105M 0s 2024-09-12T12:57:48.072 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:160300K .......... .......... .......... .......... .......... 95% 11.2M 0s 2024-09-12T12:57:48.072 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:160350K .......... .......... .......... .......... .......... 95% 89.3M 0s 2024-09-12T12:57:48.078 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:160400K .......... .......... .......... .......... .......... 95% 6.89M 0s 2024-09-12T12:57:48.089 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:160450K .......... .......... .......... .......... .......... 95% 6.49M 0s 2024-09-12T12:57:48.089 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:160500K .......... .......... .......... .......... .......... 95% 101M 0s 2024-09-12T12:57:48.089 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:160550K .......... .......... .......... .......... .......... 95% 42.9M 0s 2024-09-12T12:57:48.092 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:160600K .......... .......... .......... .......... .......... 95% 13.4M 0s 2024-09-12T12:57:48.092 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:160650K .......... .......... .......... .......... .......... 95% 109M 0s 2024-09-12T12:57:48.096 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:160700K .......... .......... .......... .......... .......... 95% 10.1M 0s 2024-09-12T12:57:48.108 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:160750K .......... .......... .......... .......... .......... 95% 4.93M 0s 2024-09-12T12:57:48.109 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:160800K .......... .......... .......... .......... .......... 95% 110M 0s 2024-09-12T12:57:48.109 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:160850K .......... .......... .......... .......... .......... 95% 56.7M 0s 2024-09-12T12:57:48.112 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:160900K .......... .......... .......... .......... .......... 95% 12.8M 0s 2024-09-12T12:57:48.112 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:160950K .......... .......... .......... .......... .......... 95% 95.8M 0s 2024-09-12T12:57:48.119 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:161000K .......... .......... .......... .......... .......... 95% 6.88M 0s 2024-09-12T12:57:48.128 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:161050K .......... .......... .......... .......... .......... 95% 6.45M 0s 2024-09-12T12:57:48.128 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:161100K .......... .......... .......... .......... .......... 95% 109M 0s 2024-09-12T12:57:48.129 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:161150K .......... .......... .......... .......... .......... 95% 22.0M 0s 2024-09-12T12:57:48.132 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:161200K .......... .......... .......... .......... .......... 95% 19.5M 0s 2024-09-12T12:57:48.132 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:161250K .......... .......... .......... .......... .......... 95% 111M 0s 2024-09-12T12:57:48.139 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:161300K .......... .......... .......... .......... .......... 95% 6.98M 0s 2024-09-12T12:57:48.150 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:161350K .......... .......... .......... .......... .......... 95% 6.34M 0s 2024-09-12T12:57:48.150 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:161400K .......... .......... .......... .......... .......... 95% 103M 0s 2024-09-12T12:57:48.150 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:161450K .......... .......... .......... .......... .......... 95% 22.6M 0s 2024-09-12T12:57:48.152 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:161500K .......... .......... .......... .......... .......... 95% 19.6M 0s 2024-09-12T12:57:48.152 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:161550K .......... .......... .......... .......... .......... 95% 87.8M 0s 2024-09-12T12:57:48.160 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:161600K .......... .......... .......... .......... .......... 95% 6.85M 0s 2024-09-12T12:57:48.168 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:161650K .......... .......... .......... .......... .......... 95% 6.53M 0s 2024-09-12T12:57:48.168 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:161700K .......... .......... .......... .......... .......... 95% 102M 0s 2024-09-12T12:57:48.169 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:161750K .......... .......... .......... .......... .......... 95% 20.5M 0s 2024-09-12T12:57:48.173 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:161800K .......... .......... .......... .......... .......... 96% 19.2M 0s 2024-09-12T12:57:48.174 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:161850K .......... .......... .......... .......... .......... 96% 115M 0s 2024-09-12T12:57:48.179 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:161900K .......... .......... .......... .......... .......... 96% 7.10M 0s 2024-09-12T12:57:48.189 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:161950K .......... .......... .......... .......... .......... 96% 6.20M 0s 2024-09-12T12:57:48.189 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:162000K .......... .......... .......... .......... .......... 96% 115M 0s 2024-09-12T12:57:48.190 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:162050K .......... .......... .......... .......... .......... 96% 24.1M 0s 2024-09-12T12:57:48.194 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:162100K .......... .......... .......... .......... .......... 96% 18.2M 0s 2024-09-12T12:57:48.194 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:162150K .......... .......... .......... .......... .......... 96% 91.9M 0s 2024-09-12T12:57:48.199 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:162200K .......... .......... .......... .......... .......... 96% 7.14M 0s 2024-09-12T12:57:48.209 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:162250K .......... .......... .......... .......... .......... 96% 6.00M 0s 2024-09-12T12:57:48.210 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:162300K .......... .......... .......... .......... .......... 96% 114M 0s 2024-09-12T12:57:48.215 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:162350K .......... .......... .......... .......... .......... 96% 11.5M 0s 2024-09-12T12:57:48.215 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:162400K .......... .......... .......... .......... .......... 96% 104M 0s 2024-09-12T12:57:48.215 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:162450K .......... .......... .......... .......... .......... 96% 116M 0s 2024-09-12T12:57:48.220 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:162500K .......... .......... .......... .......... .......... 96% 7.48M 0s 2024-09-12T12:57:48.231 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:162550K .......... .......... .......... .......... .......... 96% 6.09M 0s 2024-09-12T12:57:48.231 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:162600K .......... .......... .......... .......... .......... 96% 111M 0s 2024-09-12T12:57:48.238 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:162650K .......... .......... .......... .......... .......... 96% 10.4M 0s 2024-09-12T12:57:48.238 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:162700K .......... .......... .......... .......... .......... 96% 100M 0s 2024-09-12T12:57:48.238 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:162750K .......... .......... .......... .......... .......... 96% 90.2M 0s 2024-09-12T12:57:48.259 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:162800K .......... .......... .......... .......... .......... 96% 3.49M 0s 2024-09-12T12:57:48.260 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:162850K .......... .......... .......... .......... .......... 96% 112M 0s 2024-09-12T12:57:48.260 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:162900K .......... .......... .......... .......... .......... 96% 101M 0s 2024-09-12T12:57:48.260 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:162950K .......... .......... .......... .......... .......... 96% 11.7M 0s 2024-09-12T12:57:48.260 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:163000K .......... .......... .......... .......... .......... 96% 108M 0s 2024-09-12T12:57:48.260 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:163050K .......... .......... .......... .......... .......... 96% 104M 0s 2024-09-12T12:57:48.261 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:163100K .......... .......... .......... .......... .......... 96% 7.24M 0s 2024-09-12T12:57:48.271 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:163150K .......... .......... .......... .......... .......... 96% 5.91M 0s 2024-09-12T12:57:48.271 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:163200K .......... .......... .......... .......... .......... 96% 112M 0s 2024-09-12T12:57:48.276 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:163250K .......... .......... .......... .......... .......... 96% 12.4M 0s 2024-09-12T12:57:48.276 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:163300K .......... .......... .......... .......... .......... 96% 101M 0s 2024-09-12T12:57:48.276 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:163350K .......... .......... .......... .......... .......... 96% 93.6M 0s 2024-09-12T12:57:48.292 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:163400K .......... .......... .......... .......... .......... 96% 3.36M 0s 2024-09-12T12:57:48.292 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:163450K .......... .......... .......... .......... .......... 96% 110M 0s 2024-09-12T12:57:48.292 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:163500K .......... .......... .......... .......... .......... 97% 106M 0s 2024-09-12T12:57:48.295 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:163550K .......... .......... .......... .......... .......... 97% 12.5M 0s 2024-09-12T12:57:48.295 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:163600K .......... .......... .......... .......... .......... 97% 110M 0s 2024-09-12T12:57:48.296 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:163650K .......... .......... .......... .......... .......... 97% 116M 0s 2024-09-12T12:57:48.312 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:163700K .......... .......... .......... .......... .......... 97% 3.32M 0s 2024-09-12T12:57:48.312 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:163750K .......... .......... .......... .......... .......... 97% 91.7M 0s 2024-09-12T12:57:48.312 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:163800K .......... .......... .......... .......... .......... 97% 112M 0s 2024-09-12T12:57:48.317 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:163850K .......... .......... .......... .......... .......... 97% 12.8M 0s 2024-09-12T12:57:48.317 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:163900K .......... .......... .......... .......... .......... 97% 112M 0s 2024-09-12T12:57:48.320 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:163950K .......... .......... .......... .......... .......... 97% 10.4M 0s 2024-09-12T12:57:48.333 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:164000K .......... .......... .......... .......... .......... 97% 4.54M 0s 2024-09-12T12:57:48.334 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:164050K .......... .......... .......... .......... .......... 97% 105M 0s 2024-09-12T12:57:48.334 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:164100K .......... .......... .......... .......... .......... 97% 99.7M 0s 2024-09-12T12:57:48.334 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:164150K .......... .......... .......... .......... .......... 97% 66.9M 0s 2024-09-12T12:57:48.336 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:164200K .......... .......... .......... .......... .......... 97% 15.4M 0s 2024-09-12T12:57:48.336 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:164250K .......... .......... .......... .......... .......... 97% 193M 0s 2024-09-12T12:57:48.343 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:164300K .......... .......... .......... .......... .......... 97% 6.54M 0s 2024-09-12T12:57:48.352 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:164350K .......... .......... .......... .......... .......... 97% 5.95M 0s 2024-09-12T12:57:48.353 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:164400K .......... .......... .......... .......... .......... 97% 206M 0s 2024-09-12T12:57:48.353 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:164450K .......... .......... .......... .......... .......... 97% 45.8M 0s 2024-09-12T12:57:48.357 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:164500K .......... .......... .......... .......... .......... 97% 16.0M 0s 2024-09-12T12:57:48.357 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:164550K .......... .......... .......... .......... .......... 97% 169M 0s 2024-09-12T12:57:48.374 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:164600K .......... .......... .......... .......... .......... 97% 3.12M 0s 2024-09-12T12:57:48.374 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:164650K .......... .......... .......... .......... .......... 97% 194M 0s 2024-09-12T12:57:48.374 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:164700K .......... .......... .......... .......... .......... 97% 188M 0s 2024-09-12T12:57:48.374 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:164750K .......... .......... .......... .......... .......... 97% 151M 0s 2024-09-12T12:57:48.377 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:164800K .......... .......... .......... .......... .......... 97% 13.7M 0s 2024-09-12T12:57:48.377 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:164850K .......... .......... .......... .......... .......... 97% 194M 0s 2024-09-12T12:57:48.384 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:164900K .......... .......... .......... .......... .......... 97% 5.76M 0s 2024-09-12T12:57:48.398 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:164950K .......... .......... .......... .......... .......... 97% 6.60M 0s 2024-09-12T12:57:48.399 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:165000K .......... .......... .......... .......... .......... 97% 180M 0s 2024-09-12T12:57:48.399 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:165050K .......... .......... .......... .......... .......... 97% 191M 0s 2024-09-12T12:57:48.399 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:165100K .......... .......... .......... .......... .......... 97% 14.2M 0s 2024-09-12T12:57:48.399 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:165150K .......... .......... .......... .......... .......... 98% 130M 0s 2024-09-12T12:57:48.404 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:165200K .......... .......... .......... .......... .......... 98% 6.46M 0s 2024-09-12T12:57:48.415 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:165250K .......... .......... .......... .......... .......... 98% 5.72M 0s 2024-09-12T12:57:48.415 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:165300K .......... .......... .......... .......... .......... 98% 185M 0s 2024-09-12T12:57:48.415 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:165350K .......... .......... .......... .......... .......... 98% 176M 0s 2024-09-12T12:57:48.415 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:165400K .......... .......... .......... .......... .......... 98% 178M 0s 2024-09-12T12:57:48.417 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:165450K .......... .......... .......... .......... .......... 98% 15.3M 0s 2024-09-12T12:57:48.421 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:165500K .......... .......... .......... .......... .......... 98% 9.93M 0s 2024-09-12T12:57:48.435 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:165550K .......... .......... .......... .......... .......... 98% 4.27M 0s 2024-09-12T12:57:48.436 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:165600K .......... .......... .......... .......... .......... 98% 193M 0s 2024-09-12T12:57:48.436 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:165650K .......... .......... .......... .......... .......... 98% 186M 0s 2024-09-12T12:57:48.436 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:165700K .......... .......... .......... .......... .......... 98% 187M 0s 2024-09-12T12:57:48.438 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:165750K .......... .......... .......... .......... .......... 98% 14.9M 0s 2024-09-12T12:57:48.438 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:165800K .......... .......... .......... .......... .......... 98% 165M 0s 2024-09-12T12:57:48.456 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:165850K .......... .......... .......... .......... .......... 98% 3.06M 0s 2024-09-12T12:57:48.456 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:165900K .......... .......... .......... .......... .......... 98% 180M 0s 2024-09-12T12:57:48.456 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:165950K .......... .......... .......... .......... .......... 98% 167M 0s 2024-09-12T12:57:48.456 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:166000K .......... .......... .......... .......... .......... 98% 197M 0s 2024-09-12T12:57:48.458 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:166050K .......... .......... .......... .......... .......... 98% 16.6M 0s 2024-09-12T12:57:48.458 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:166100K .......... .......... .......... .......... .......... 98% 178M 0s 2024-09-12T12:57:48.476 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:166150K .......... .......... .......... .......... .......... 98% 2.99M 0s 2024-09-12T12:57:48.476 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:166200K .......... .......... .......... .......... .......... 98% 193M 0s 2024-09-12T12:57:48.476 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:166250K .......... .......... .......... .......... .......... 98% 90.1M 0s 2024-09-12T12:57:48.476 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:166300K .......... .......... .......... .......... .......... 98% 193M 0s 2024-09-12T12:57:48.478 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:166350K .......... .......... .......... .......... .......... 98% 18.6M 0s 2024-09-12T12:57:48.478 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:166400K .......... .......... .......... .......... .......... 98% 192M 0s 2024-09-12T12:57:48.485 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:166450K .......... .......... .......... .......... .......... 98% 6.13M 0s 2024-09-12T12:57:48.496 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:166500K .......... .......... .......... .......... .......... 98% 5.68M 0s 2024-09-12T12:57:48.496 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:166550K .......... .......... .......... .......... .......... 98% 160M 0s 2024-09-12T12:57:48.497 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:166600K .......... .......... .......... .......... .......... 98% 208M 0s 2024-09-12T12:57:48.497 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:166650K .......... .......... .......... .......... .......... 98% 195M 0s 2024-09-12T12:57:48.499 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:166700K .......... .......... .......... .......... .......... 98% 16.1M 0s 2024-09-12T12:57:48.506 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:166750K .......... .......... .......... .......... .......... 98% 6.08M 0s 2024-09-12T12:57:48.517 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:166800K .......... .......... .......... .......... .......... 98% 5.70M 0s 2024-09-12T12:57:48.517 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:166850K .......... .......... .......... .......... .......... 99% 195M 0s 2024-09-12T12:57:48.517 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:166900K .......... .......... .......... .......... .......... 99% 202M 0s 2024-09-12T12:57:48.517 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:166950K .......... .......... .......... .......... .......... 99% 175M 0s 2024-09-12T12:57:48.520 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:167000K .......... .......... .......... .......... .......... 99% 16.1M 0s 2024-09-12T12:57:48.520 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:167050K .......... .......... .......... .......... .......... 99% 189M 0s 2024-09-12T12:57:48.537 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:167100K .......... .......... .......... .......... .......... 99% 3.01M 0s 2024-09-12T12:57:48.537 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:167150K .......... .......... .......... .......... .......... 99% 161M 0s 2024-09-12T12:57:48.537 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:167200K .......... .......... .......... .......... .......... 99% 167M 0s 2024-09-12T12:57:48.537 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:167250K .......... .......... .......... .......... .......... 99% 202M 0s 2024-09-12T12:57:48.539 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:167300K .......... .......... .......... .......... .......... 99% 17.5M 0s 2024-09-12T12:57:48.539 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:167350K .......... .......... .......... .......... .......... 99% 176M 0s 2024-09-12T12:57:48.558 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:167400K .......... .......... .......... .......... .......... 99% 2.95M 0s 2024-09-12T12:57:48.558 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:167450K .......... .......... .......... .......... .......... 99% 166M 0s 2024-09-12T12:57:48.558 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:167500K .......... .......... .......... .......... .......... 99% 191M 0s 2024-09-12T12:57:48.558 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:167550K .......... .......... .......... .......... .......... 99% 184M 0s 2024-09-12T12:57:48.560 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:167600K .......... .......... .......... .......... .......... 99% 18.4M 0s 2024-09-12T12:57:48.560 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:167650K .......... .......... .......... .......... .......... 99% 187M 0s 2024-09-12T12:57:48.566 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:167700K .......... .......... .......... .......... .......... 99% 6.09M 0s 2024-09-12T12:57:48.580 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:167750K .......... .......... .......... .......... .......... 99% 5.64M 0s 2024-09-12T12:57:48.580 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:167800K .......... .......... .......... .......... .......... 99% 197M 0s 2024-09-12T12:57:48.580 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:167850K .......... .......... .......... .......... .......... 99% 201M 0s 2024-09-12T12:57:48.580 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:167900K .......... .......... .......... .......... .......... 99% 15.7M 0s 2024-09-12T12:57:48.580 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:167950K .......... .......... .......... .......... .......... 99% 169M 0s 2024-09-12T12:57:48.587 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:168000K .......... .......... .......... .......... .......... 99% 6.27M 0s 2024-09-12T12:57:48.598 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:168050K .......... .......... .......... .......... .......... 99% 5.59M 0s 2024-09-12T12:57:48.598 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:168100K .......... .......... .......... .......... .......... 99% 192M 0s 2024-09-12T12:57:48.598 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:168150K .......... .......... .......... .......... .......... 99% 171M 0s 2024-09-12T12:57:48.599 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:168200K .......... .......... .......... .......... .......... 99% 219M 0s 2024-09-12T12:57:48.600 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:168250K .......... .......... .......... .......... .......... 99% 16.5M 0s 2024-09-12T12:57:48.604 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:168300K .......... .......... .......... .......... .......... 99% 10.8M 0s 2024-09-12T12:57:48.620 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:168350K .......... .......... .......... .......... .......... 99% 4.05M 0s 2024-09-12T12:57:48.620 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:168400K .......... .......... .......... .......... .......... 99% 196M 0s 2024-09-12T12:57:48.621 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:168450K .......... .......... .......... .......... .......... 99% 195M 0s 2024-09-12T12:57:48.621 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:168500K .......... .......... .......... .......... .......... 99% 218M 0s 2024-09-12T12:57:48.621 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:168550K .......... .......... . 100% 193M=9.1s 2024-09-12T12:57:48.621 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 2024-09-12T12:57:48.621 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:2024-09-12 12:57:48 (18.2 MB/s) - ‘linux-5.4.tar.gz’ saved [172616875/172616875] 2024-09-12T12:57:48.621 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: 2024-09-12T12:57:48.621 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..10} 2024-09-12T12:57:48.621 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEVS=() 2024-09-12T12:57:48.621 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ SNAP_PIDS=() 2024-09-12T12:57:48.621 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ COMPARE_PIDS=() 2024-09-12T12:57:48.621 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ WORKLOAD_PIDS=() 2024-09-12T12:57:48.622 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ RET=0 2024-09-12T12:57:48.622 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T12:57:48.622 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ IMG=image-primary1 2024-09-12T12:57:48.622 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ MNTPT=test-primary1 2024-09-12T12:57:48.622 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ create_image_and_enable_mirror cluster1 mirror image-primary1 snapshot 10G 2024-09-12T12:57:48.622 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:57:48.622 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T12:57:48.622 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:57:48.622 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T12:57:48.622 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T12:57:48.622 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T12:57:48.622 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mode=snapshot 2024-09-12T12:57:48.623 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n snapshot ']' 2024-09-12T12:57:48.623 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T12:57:48.623 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ create_image cluster1 mirror image-primary1 10G 2024-09-12T12:57:48.623 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:57:48.623 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T12:57:48.623 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:57:48.623 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T12:57:48.623 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T12:57:48.623 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T12:57:48.623 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local size=128 2024-09-12T12:57:48.623 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n 10G ']' 2024-09-12T12:57:48.623 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ size=10G 2024-09-12T12:57:48.624 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T12:57:48.624 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 create --size 10G --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image-primary1 2024-09-12T12:57:48.692 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ is_pool_mirror_mode_image mirror 2024-09-12T12:57:48.692 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:57:48.692 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' image = image ']' 2024-09-12T12:57:48.692 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T12:57:48.692 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ enable_mirror cluster1 mirror image-primary1 snapshot 2024-09-12T12:57:48.692 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:57:48.693 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:57:48.693 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T12:57:48.693 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mode=snapshot 2024-09-12T12:57:48.693 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 mirror image enable mirror/image-primary1 snapshot 2024-09-12T12:57:48.835 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Mirroring enabled 2024-09-12T12:57:48.845 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 info mirror/image-primary1 2024-09-12T12:57:48.907 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:rbd image 'image-primary1': 2024-09-12T12:57:48.907 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: size 10 GiB in 2560 objects 2024-09-12T12:57:48.908 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: order 22 (4 MiB objects) 2024-09-12T12:57:48.908 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: snapshot_count: 1 2024-09-12T12:57:48.908 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: id: 1093720fe4df 2024-09-12T12:57:48.908 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: block_name_prefix: rbd_data.1093720fe4df 2024-09-12T12:57:48.908 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: format: 2 2024-09-12T12:57:48.908 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: features: layering, exclusive-lock, object-map, fast-diff 2024-09-12T12:57:48.908 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: op_features: 2024-09-12T12:57:48.908 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: flags: 2024-09-12T12:57:48.908 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: create_timestamp: Thu Sep 12 12:57:48 2024 2024-09-12T12:57:48.908 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: access_timestamp: Thu Sep 12 12:57:48 2024 2024-09-12T12:57:48.908 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: modify_timestamp: Thu Sep 12 12:57:48 2024 2024-09-12T12:57:48.908 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring state: enabled 2024-09-12T12:57:48.909 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring mode: snapshot 2024-09-12T12:57:48.909 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring global id: 5a3f10b5-2891-464a-a2f2-921068cdf6e7 2024-09-12T12:57:48.909 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring primary: true 2024-09-12T12:57:48.912 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \n\b\d ]] 2024-09-12T12:57:48.912 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \k\r\b\d ]] 2024-09-12T12:57:48.913 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo rbd --cluster cluster1 device map -t krbd mirror/image-primary1 2024-09-12T12:57:49.513 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEV=/dev/rbd0 2024-09-12T12:57:49.513 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEVS+=($DEV) 2024-09-12T12:57:49.513 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo mkfs.ext4 /dev/rbd0 2024-09-12T12:57:49.564 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:mke2fs 1.46.5 (30-Dec-2021) 2024-09-12T12:57:49.577 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Discarding device blocks: 0/2621440 done 2024-09-12T12:57:49.578 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Creating filesystem with 2621440 4k blocks and 655360 inodes 2024-09-12T12:57:49.578 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Filesystem UUID: e22e2e3e-6f5c-49f0-8cc3-1bbc771d0997 2024-09-12T12:57:49.578 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Superblock backups stored on blocks: 2024-09-12T12:57:49.578 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 2024-09-12T12:57:49.578 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 2024-09-12T12:57:49.578 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Allocating group tables: 0/80 done 2024-09-12T12:57:49.578 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Writing inode tables: 0/80 done 2024-09-12T12:57:49.582 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Creating journal (16384 blocks): done 2024-09-12T12:57:49.628 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Writing superblocks and filesystem accounting information: 0/80 done 2024-09-12T12:57:49.629 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 2024-09-12T12:57:49.631 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mkdir test-primary1 2024-09-12T12:57:49.634 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo mount /dev/rbd0 test-primary1 2024-09-12T12:57:49.689 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ whoami 2024-09-12T12:57:49.690 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo chown ubuntu test-primary1 2024-09-12T12:57:49.715 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ SNAP_PIDS+=($!) 2024-09-12T12:57:49.715 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ take_mirror_snapshots cluster1 mirror image-primary1 2024-09-12T12:57:49.715 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:57:49.715 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:57:49.715 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T12:57:49.715 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:57:49.715 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T12:57:49.715 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ WORKLOAD_PIDS+=($!) 2024-09-12T12:57:49.715 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:57:49.715 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:57:49.716 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T12:57:49.716 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T12:57:49.716 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T12:57:49.716 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ IMG=image-primary2 2024-09-12T12:57:49.716 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ MNTPT=test-primary2 2024-09-12T12:57:49.716 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ create_image_and_enable_mirror cluster1 mirror image-primary2 snapshot 10G 2024-09-12T12:57:49.716 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ slow_untar_workload test-primary1 2024-09-12T12:57:49.716 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:57:49.716 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T12:57:49.716 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:57:49.716 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T12:57:49.716 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T12:57:49.716 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T12:57:49.717 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mode=snapshot 2024-09-12T12:57:49.717 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n snapshot ']' 2024-09-12T12:57:49.717 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T12:57:49.717 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ create_image cluster1 mirror image-primary2 10G 2024-09-12T12:57:49.717 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:57:49.717 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T12:57:49.717 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mountpt=test-primary1 2024-09-12T12:57:49.717 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:57:49.717 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T12:57:49.717 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T12:57:49.717 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T12:57:49.717 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local size=128 2024-09-12T12:57:49.717 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n 10G ']' 2024-09-12T12:57:49.718 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ size=10G 2024-09-12T12:57:49.718 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ cp linux-5.4.tar.gz test-primary1 2024-09-12T12:57:49.718 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T12:57:49.718 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 create --size 10G --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image-primary2 2024-09-12T12:57:49.792 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ is_pool_mirror_mode_image mirror 2024-09-12T12:57:49.792 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:57:49.792 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' image = image ']' 2024-09-12T12:57:49.792 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T12:57:49.792 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ enable_mirror cluster1 mirror image-primary2 snapshot 2024-09-12T12:57:49.792 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:57:49.793 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:57:49.793 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T12:57:49.793 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mode=snapshot 2024-09-12T12:57:49.793 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 mirror image enable mirror/image-primary2 snapshot 2024-09-12T12:57:49.839 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 4 2024-09-12T12:57:49.853 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:57:50.013 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local ret=0 2024-09-12T12:57:50.013 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ timeout 5m bash -c 'zcat test-primary1/linux-5.4.tar.gz | pv -L 256K | tar xf - -C test-primary1' 2024-09-12T12:57:50.844 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Mirroring enabled 2024-09-12T12:57:50.856 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 info mirror/image-primary2 2024-09-12T12:57:50.919 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:rbd image 'image-primary2': 2024-09-12T12:57:50.919 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: size 10 GiB in 2560 objects 2024-09-12T12:57:50.919 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: order 22 (4 MiB objects) 2024-09-12T12:57:50.919 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: snapshot_count: 1 2024-09-12T12:57:50.919 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: id: 10a3467ec284 2024-09-12T12:57:50.919 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: block_name_prefix: rbd_data.10a3467ec284 2024-09-12T12:57:50.919 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: format: 2 2024-09-12T12:57:50.919 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: features: layering, exclusive-lock, object-map, fast-diff 2024-09-12T12:57:50.919 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: op_features: 2024-09-12T12:57:50.919 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: flags: 2024-09-12T12:57:50.919 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: create_timestamp: Thu Sep 12 12:57:49 2024 2024-09-12T12:57:50.920 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: access_timestamp: Thu Sep 12 12:57:49 2024 2024-09-12T12:57:50.920 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: modify_timestamp: Thu Sep 12 12:57:49 2024 2024-09-12T12:57:50.920 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring state: enabled 2024-09-12T12:57:50.920 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring mode: snapshot 2024-09-12T12:57:50.920 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring global id: b7ebe2f2-cab7-4e57-a03a-3091186a2999 2024-09-12T12:57:50.920 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring primary: true 2024-09-12T12:57:50.924 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \n\b\d ]] 2024-09-12T12:57:50.924 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \k\r\b\d ]] 2024-09-12T12:57:50.924 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo rbd --cluster cluster1 device map -t krbd mirror/image-primary2 2024-09-12T12:57:51.051 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEV=/dev/rbd1 2024-09-12T12:57:51.051 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEVS+=($DEV) 2024-09-12T12:57:51.051 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo mkfs.ext4 /dev/rbd1 2024-09-12T12:57:51.071 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:mke2fs 1.46.5 (30-Dec-2021) 2024-09-12T12:57:51.083 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Discarding device blocks: 0/2621440 done 2024-09-12T12:57:51.084 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Creating filesystem with 2621440 4k blocks and 655360 inodes 2024-09-12T12:57:51.084 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Filesystem UUID: 290f1d3a-a3fe-4671-a1d7-4d37243bb623 2024-09-12T12:57:51.084 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Superblock backups stored on blocks: 2024-09-12T12:57:51.084 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 2024-09-12T12:57:51.084 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 2024-09-12T12:57:51.084 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Allocating group tables: 0/80 done 2024-09-12T12:57:51.084 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Writing inode tables: 0/80 done 2024-09-12T12:57:51.089 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Creating journal (16384 blocks): done 2024-09-12T12:57:51.138 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Writing superblocks and filesystem accounting information: 0/80 done 2024-09-12T12:57:51.138 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 2024-09-12T12:57:51.141 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mkdir test-primary2 2024-09-12T12:57:51.143 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo mount /dev/rbd1 test-primary2 2024-09-12T12:57:51.198 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ whoami 2024-09-12T12:57:51.200 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo chown ubuntu test-primary2 2024-09-12T12:57:51.226 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ SNAP_PIDS+=($!) 2024-09-12T12:57:51.226 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ take_mirror_snapshots cluster1 mirror image-primary2 2024-09-12T12:57:51.226 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:57:51.226 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:57:51.226 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T12:57:51.226 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:57:51.226 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T12:57:51.226 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:57:51.226 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:57:51.226 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T12:57:51.226 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T12:57:51.227 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ WORKLOAD_PIDS+=($!) 2024-09-12T12:57:51.227 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T12:57:51.227 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ IMG=image-primary3 2024-09-12T12:57:51.227 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ MNTPT=test-primary3 2024-09-12T12:57:51.227 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ create_image_and_enable_mirror cluster1 mirror image-primary3 snapshot 10G 2024-09-12T12:57:51.227 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ slow_untar_workload test-primary2 2024-09-12T12:57:51.227 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:57:51.227 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T12:57:51.227 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:57:51.227 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T12:57:51.227 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T12:57:51.227 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T12:57:51.227 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mode=snapshot 2024-09-12T12:57:51.227 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n snapshot ']' 2024-09-12T12:57:51.228 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T12:57:51.228 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ create_image cluster1 mirror image-primary3 10G 2024-09-12T12:57:51.228 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mountpt=test-primary2 2024-09-12T12:57:51.228 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:57:51.228 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T12:57:51.228 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:57:51.228 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ cp linux-5.4.tar.gz test-primary2 2024-09-12T12:57:51.228 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T12:57:51.228 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T12:57:51.228 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T12:57:51.228 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local size=128 2024-09-12T12:57:51.228 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n 10G ']' 2024-09-12T12:57:51.228 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ size=10G 2024-09-12T12:57:51.228 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T12:57:51.229 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 create --size 10G --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image-primary3 2024-09-12T12:57:51.301 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ is_pool_mirror_mode_image mirror 2024-09-12T12:57:51.301 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:57:51.301 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' image = image ']' 2024-09-12T12:57:51.301 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T12:57:51.301 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ enable_mirror cluster1 mirror image-primary3 snapshot 2024-09-12T12:57:51.302 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:57:51.302 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:57:51.302 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T12:57:51.302 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mode=snapshot 2024-09-12T12:57:51.302 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 mirror image enable mirror/image-primary3 snapshot 2024-09-12T12:57:51.535 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local ret=0 2024-09-12T12:57:51.535 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ timeout 5m bash -c 'zcat test-primary2/linux-5.4.tar.gz | pv -L 256K | tar xf - -C test-primary2' 2024-09-12T12:57:51.848 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Mirroring enabled 2024-09-12T12:57:51.853 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 6 2024-09-12T12:57:51.862 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 info mirror/image-primary3 2024-09-12T12:57:51.863 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:57:51.918 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:rbd image 'image-primary3': 2024-09-12T12:57:51.918 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: size 10 GiB in 2560 objects 2024-09-12T12:57:51.918 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: order 22 (4 MiB objects) 2024-09-12T12:57:51.918 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: snapshot_count: 1 2024-09-12T12:57:51.918 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: id: 10b0222d06e3 2024-09-12T12:57:51.918 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: block_name_prefix: rbd_data.10b0222d06e3 2024-09-12T12:57:51.918 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: format: 2 2024-09-12T12:57:51.918 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: features: layering, exclusive-lock, object-map, fast-diff 2024-09-12T12:57:51.918 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: op_features: 2024-09-12T12:57:51.918 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: flags: 2024-09-12T12:57:51.919 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: create_timestamp: Thu Sep 12 12:57:51 2024 2024-09-12T12:57:51.919 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: access_timestamp: Thu Sep 12 12:57:51 2024 2024-09-12T12:57:51.919 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: modify_timestamp: Thu Sep 12 12:57:51 2024 2024-09-12T12:57:51.919 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring state: enabled 2024-09-12T12:57:51.919 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring mode: snapshot 2024-09-12T12:57:51.919 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring global id: c469479f-c19b-4534-a75c-ca68f8c28929 2024-09-12T12:57:51.919 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring primary: true 2024-09-12T12:57:51.923 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \n\b\d ]] 2024-09-12T12:57:51.923 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \k\r\b\d ]] 2024-09-12T12:57:51.924 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo rbd --cluster cluster1 device map -t krbd mirror/image-primary3 2024-09-12T12:57:52.045 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEV=/dev/rbd2 2024-09-12T12:57:52.045 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEVS+=($DEV) 2024-09-12T12:57:52.045 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo mkfs.ext4 /dev/rbd2 2024-09-12T12:57:52.065 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:mke2fs 1.46.5 (30-Dec-2021) 2024-09-12T12:57:52.079 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Discarding device blocks: 0/2621440 done 2024-09-12T12:57:52.079 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Creating filesystem with 2621440 4k blocks and 655360 inodes 2024-09-12T12:57:52.079 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Filesystem UUID: 9399ec10-ac61-44e6-b850-14daec41b0c8 2024-09-12T12:57:52.079 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Superblock backups stored on blocks: 2024-09-12T12:57:52.080 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 2024-09-12T12:57:52.080 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 2024-09-12T12:57:52.080 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Allocating group tables: 0/80 done 2024-09-12T12:57:52.080 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Writing inode tables: 0/80 done 2024-09-12T12:57:52.084 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Creating journal (16384 blocks): done 2024-09-12T12:57:52.130 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Writing superblocks and filesystem accounting information: 0/80 done 2024-09-12T12:57:52.130 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 2024-09-12T12:57:52.133 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mkdir test-primary3 2024-09-12T12:57:52.136 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo mount /dev/rbd2 test-primary3 2024-09-12T12:57:52.190 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ whoami 2024-09-12T12:57:52.191 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo chown ubuntu test-primary3 2024-09-12T12:57:52.216 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ SNAP_PIDS+=($!) 2024-09-12T12:57:52.216 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ take_mirror_snapshots cluster1 mirror image-primary3 2024-09-12T12:57:52.216 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:57:52.216 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:57:52.216 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T12:57:52.217 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:57:52.217 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T12:57:52.217 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:57:52.217 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:57:52.217 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T12:57:52.217 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ WORKLOAD_PIDS+=($!) 2024-09-12T12:57:52.217 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T12:57:52.217 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T12:57:52.217 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ IMG=image-primary4 2024-09-12T12:57:52.217 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ MNTPT=test-primary4 2024-09-12T12:57:52.217 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ create_image_and_enable_mirror cluster1 mirror image-primary4 snapshot 10G 2024-09-12T12:57:52.217 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ slow_untar_workload test-primary3 2024-09-12T12:57:52.217 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:57:52.218 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T12:57:52.219 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:57:52.219 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T12:57:52.219 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T12:57:52.219 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T12:57:52.219 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mode=snapshot 2024-09-12T12:57:52.219 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n snapshot ']' 2024-09-12T12:57:52.219 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T12:57:52.219 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ create_image cluster1 mirror image-primary4 10G 2024-09-12T12:57:52.219 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:57:52.219 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T12:57:52.219 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:57:52.220 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T12:57:52.220 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T12:57:52.220 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T12:57:52.220 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local size=128 2024-09-12T12:57:52.220 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n 10G ']' 2024-09-12T12:57:52.220 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ size=10G 2024-09-12T12:57:52.220 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mountpt=test-primary3 2024-09-12T12:57:52.220 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T12:57:52.220 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 create --size 10G --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image-primary4 2024-09-12T12:57:52.220 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ cp linux-5.4.tar.gz test-primary3 2024-09-12T12:57:52.284 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ is_pool_mirror_mode_image mirror 2024-09-12T12:57:52.284 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:57:52.284 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' image = image ']' 2024-09-12T12:57:52.284 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T12:57:52.285 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ enable_mirror cluster1 mirror image-primary4 snapshot 2024-09-12T12:57:52.285 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:57:52.285 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:57:52.285 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T12:57:52.285 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mode=snapshot 2024-09-12T12:57:52.285 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 mirror image enable mirror/image-primary4 snapshot 2024-09-12T12:57:52.511 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local ret=0 2024-09-12T12:57:52.511 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ timeout 5m bash -c 'zcat test-primary3/linux-5.4.tar.gz | pv -L 256K | tar xf - -C test-primary3' 2024-09-12T12:57:52.851 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Mirroring enabled 2024-09-12T12:57:52.856 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:57:52.856 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T12:57:52.856 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:57:52.856 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:57:52.856 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T12:57:52.856 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T12:57:52.858 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 8 2024-09-12T12:57:52.866 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 info mirror/image-primary4 2024-09-12T12:57:52.867 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:57:52.926 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:rbd image 'image-primary4': 2024-09-12T12:57:52.926 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: size 10 GiB in 2560 objects 2024-09-12T12:57:52.926 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: order 22 (4 MiB objects) 2024-09-12T12:57:52.926 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: snapshot_count: 1 2024-09-12T12:57:52.927 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: id: 10be765f1407 2024-09-12T12:57:52.927 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: block_name_prefix: rbd_data.10be765f1407 2024-09-12T12:57:52.927 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: format: 2 2024-09-12T12:57:52.927 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: features: layering, exclusive-lock, object-map, fast-diff 2024-09-12T12:57:52.927 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: op_features: 2024-09-12T12:57:52.927 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: flags: 2024-09-12T12:57:52.927 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: create_timestamp: Thu Sep 12 12:57:52 2024 2024-09-12T12:57:52.927 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: access_timestamp: Thu Sep 12 12:57:52 2024 2024-09-12T12:57:52.927 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: modify_timestamp: Thu Sep 12 12:57:52 2024 2024-09-12T12:57:52.927 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring state: enabled 2024-09-12T12:57:52.928 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring mode: snapshot 2024-09-12T12:57:52.928 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring global id: f5942d86-af5f-4ca1-8ebc-da4cbe6dc314 2024-09-12T12:57:52.928 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring primary: true 2024-09-12T12:57:52.932 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \n\b\d ]] 2024-09-12T12:57:52.932 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \k\r\b\d ]] 2024-09-12T12:57:52.933 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo rbd --cluster cluster1 device map -t krbd mirror/image-primary4 2024-09-12T12:57:53.054 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEV=/dev/rbd3 2024-09-12T12:57:53.054 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEVS+=($DEV) 2024-09-12T12:57:53.054 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo mkfs.ext4 /dev/rbd3 2024-09-12T12:57:53.074 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:mke2fs 1.46.5 (30-Dec-2021) 2024-09-12T12:57:53.086 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Discarding device blocks: 0/2621440 done 2024-09-12T12:57:53.086 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Creating filesystem with 2621440 4k blocks and 655360 inodes 2024-09-12T12:57:53.086 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Filesystem UUID: f02e5e69-afee-4e39-b931-6a3331416c7b 2024-09-12T12:57:53.087 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Superblock backups stored on blocks: 2024-09-12T12:57:53.087 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 2024-09-12T12:57:53.087 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 2024-09-12T12:57:53.087 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Allocating group tables: 0/80 done 2024-09-12T12:57:53.087 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Writing inode tables: 0/80 done 2024-09-12T12:57:53.091 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Creating journal (16384 blocks): done 2024-09-12T12:57:53.135 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Writing superblocks and filesystem accounting information: 0/80 done 2024-09-12T12:57:53.135 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 2024-09-12T12:57:53.138 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mkdir test-primary4 2024-09-12T12:57:53.140 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo mount /dev/rbd3 test-primary4 2024-09-12T12:57:53.196 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ whoami 2024-09-12T12:57:53.198 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo chown ubuntu test-primary4 2024-09-12T12:57:53.223 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ SNAP_PIDS+=($!) 2024-09-12T12:57:53.223 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ take_mirror_snapshots cluster1 mirror image-primary4 2024-09-12T12:57:53.224 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:57:53.224 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:57:53.224 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T12:57:53.224 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:57:53.224 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T12:57:53.224 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:57:53.224 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:57:53.224 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T12:57:53.224 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T12:57:53.224 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ WORKLOAD_PIDS+=($!) 2024-09-12T12:57:53.224 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T12:57:53.224 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ IMG=image-primary5 2024-09-12T12:57:53.224 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ MNTPT=test-primary5 2024-09-12T12:57:53.225 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ create_image_and_enable_mirror cluster1 mirror image-primary5 snapshot 10G 2024-09-12T12:57:53.225 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ slow_untar_workload test-primary4 2024-09-12T12:57:53.225 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:57:53.225 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T12:57:53.225 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:57:53.225 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T12:57:53.225 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T12:57:53.225 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T12:57:53.225 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mode=snapshot 2024-09-12T12:57:53.225 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n snapshot ']' 2024-09-12T12:57:53.226 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T12:57:53.226 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ create_image cluster1 mirror image-primary5 10G 2024-09-12T12:57:53.226 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:57:53.226 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mountpt=test-primary4 2024-09-12T12:57:53.226 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T12:57:53.226 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:57:53.226 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T12:57:53.226 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T12:57:53.226 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T12:57:53.226 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local size=128 2024-09-12T12:57:53.226 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ cp linux-5.4.tar.gz test-primary4 2024-09-12T12:57:53.226 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n 10G ']' 2024-09-12T12:57:53.226 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ size=10G 2024-09-12T12:57:53.226 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T12:57:53.227 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 create --size 10G --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image-primary5 2024-09-12T12:57:53.288 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ is_pool_mirror_mode_image mirror 2024-09-12T12:57:53.288 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:57:53.288 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' image = image ']' 2024-09-12T12:57:53.288 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T12:57:53.288 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ enable_mirror cluster1 mirror image-primary5 snapshot 2024-09-12T12:57:53.288 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:57:53.289 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:57:53.289 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T12:57:53.289 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mode=snapshot 2024-09-12T12:57:53.289 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 mirror image enable mirror/image-primary5 snapshot 2024-09-12T12:57:53.511 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local ret=0 2024-09-12T12:57:53.527 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ timeout 5m bash -c 'zcat test-primary4/linux-5.4.tar.gz | pv -L 256K | tar xf - -C test-primary4' 2024-09-12T12:57:53.854 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Mirroring enabled 2024-09-12T12:57:53.859 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 11 2024-09-12T12:57:53.861 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 10 2024-09-12T12:57:53.869 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:57:53.870 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 info mirror/image-primary5 2024-09-12T12:57:53.870 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:57:53.928 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:rbd image 'image-primary5': 2024-09-12T12:57:53.928 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: size 10 GiB in 2560 objects 2024-09-12T12:57:53.929 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: order 22 (4 MiB objects) 2024-09-12T12:57:53.929 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: snapshot_count: 1 2024-09-12T12:57:53.929 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: id: 10d0df85804b 2024-09-12T12:57:53.929 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: block_name_prefix: rbd_data.10d0df85804b 2024-09-12T12:57:53.929 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: format: 2 2024-09-12T12:57:53.929 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: features: layering, exclusive-lock, object-map, fast-diff 2024-09-12T12:57:53.929 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: op_features: 2024-09-12T12:57:53.929 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: flags: 2024-09-12T12:57:53.929 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: create_timestamp: Thu Sep 12 12:57:53 2024 2024-09-12T12:57:53.929 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: access_timestamp: Thu Sep 12 12:57:53 2024 2024-09-12T12:57:53.929 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: modify_timestamp: Thu Sep 12 12:57:53 2024 2024-09-12T12:57:53.929 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring state: enabled 2024-09-12T12:57:53.930 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring mode: snapshot 2024-09-12T12:57:53.930 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring global id: 8b95d658-5f20-45ab-b663-60abf1f860db 2024-09-12T12:57:53.930 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring primary: true 2024-09-12T12:57:53.933 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \n\b\d ]] 2024-09-12T12:57:53.933 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \k\r\b\d ]] 2024-09-12T12:57:53.934 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo rbd --cluster cluster1 device map -t krbd mirror/image-primary5 2024-09-12T12:57:54.050 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEV=/dev/rbd4 2024-09-12T12:57:54.050 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEVS+=($DEV) 2024-09-12T12:57:54.051 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo mkfs.ext4 /dev/rbd4 2024-09-12T12:57:54.070 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:mke2fs 1.46.5 (30-Dec-2021) 2024-09-12T12:57:54.083 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Discarding device blocks: 0/2621440 done 2024-09-12T12:57:54.083 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Creating filesystem with 2621440 4k blocks and 655360 inodes 2024-09-12T12:57:54.083 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Filesystem UUID: 08b623f3-c10e-43da-8cfa-5a8ac8989244 2024-09-12T12:57:54.083 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Superblock backups stored on blocks: 2024-09-12T12:57:54.084 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 2024-09-12T12:57:54.084 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 2024-09-12T12:57:54.084 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Allocating group tables: 0/80 done 2024-09-12T12:57:54.084 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Writing inode tables: 0/80 done 2024-09-12T12:57:54.087 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Creating journal (16384 blocks): done 2024-09-12T12:57:54.130 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Writing superblocks and filesystem accounting information: 0/80 done 2024-09-12T12:57:54.130 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 2024-09-12T12:57:54.132 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mkdir test-primary5 2024-09-12T12:57:54.135 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo mount /dev/rbd4 test-primary5 2024-09-12T12:57:54.189 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ whoami 2024-09-12T12:57:54.190 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo chown ubuntu test-primary5 2024-09-12T12:57:54.215 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ SNAP_PIDS+=($!) 2024-09-12T12:57:54.215 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ take_mirror_snapshots cluster1 mirror image-primary5 2024-09-12T12:57:54.215 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:57:54.215 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:57:54.215 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ WORKLOAD_PIDS+=($!) 2024-09-12T12:57:54.216 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T12:57:54.216 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:57:54.216 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T12:57:54.216 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T12:57:54.216 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ IMG=image-primary6 2024-09-12T12:57:54.216 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:57:54.216 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ MNTPT=test-primary6 2024-09-12T12:57:54.216 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:57:54.216 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T12:57:54.216 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ create_image_and_enable_mirror cluster1 mirror image-primary6 snapshot 10G 2024-09-12T12:57:54.216 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T12:57:54.216 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ slow_untar_workload test-primary5 2024-09-12T12:57:54.216 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:57:54.216 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T12:57:54.217 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:57:54.217 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T12:57:54.217 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T12:57:54.217 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T12:57:54.217 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mode=snapshot 2024-09-12T12:57:54.217 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n snapshot ']' 2024-09-12T12:57:54.217 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T12:57:54.217 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ create_image cluster1 mirror image-primary6 10G 2024-09-12T12:57:54.217 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:57:54.217 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T12:57:54.217 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:57:54.217 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T12:57:54.217 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T12:57:54.217 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T12:57:54.218 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local size=128 2024-09-12T12:57:54.218 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n 10G ']' 2024-09-12T12:57:54.218 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ size=10G 2024-09-12T12:57:54.218 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mountpt=test-primary5 2024-09-12T12:57:54.218 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T12:57:54.218 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 create --size 10G --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image-primary6 2024-09-12T12:57:54.218 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ cp linux-5.4.tar.gz test-primary5 2024-09-12T12:57:54.285 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ is_pool_mirror_mode_image mirror 2024-09-12T12:57:54.285 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:57:54.286 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' image = image ']' 2024-09-12T12:57:54.286 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T12:57:54.286 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ enable_mirror cluster1 mirror image-primary6 snapshot 2024-09-12T12:57:54.286 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:57:54.286 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:57:54.286 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T12:57:54.286 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mode=snapshot 2024-09-12T12:57:54.286 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 mirror image enable mirror/image-primary6 snapshot 2024-09-12T12:57:54.506 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local ret=0 2024-09-12T12:57:54.506 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ timeout 5m bash -c 'zcat test-primary5/linux-5.4.tar.gz | pv -L 256K | tar xf - -C test-primary5' 2024-09-12T12:57:54.856 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Mirroring enabled 2024-09-12T12:57:54.857 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 13 2024-09-12T12:57:54.865 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:57:54.866 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T12:57:54.866 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:57:54.866 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:57:54.866 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T12:57:54.866 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T12:57:54.867 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:57:54.870 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 info mirror/image-primary6 2024-09-12T12:57:54.929 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:rbd image 'image-primary6': 2024-09-12T12:57:54.929 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: size 10 GiB in 2560 objects 2024-09-12T12:57:54.929 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: order 22 (4 MiB objects) 2024-09-12T12:57:54.929 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: snapshot_count: 1 2024-09-12T12:57:54.929 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: id: 10df59cb99ff 2024-09-12T12:57:54.929 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: block_name_prefix: rbd_data.10df59cb99ff 2024-09-12T12:57:54.929 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: format: 2 2024-09-12T12:57:54.929 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: features: layering, exclusive-lock, object-map, fast-diff 2024-09-12T12:57:54.929 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: op_features: 2024-09-12T12:57:54.930 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: flags: 2024-09-12T12:57:54.930 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: create_timestamp: Thu Sep 12 12:57:54 2024 2024-09-12T12:57:54.930 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: access_timestamp: Thu Sep 12 12:57:54 2024 2024-09-12T12:57:54.930 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: modify_timestamp: Thu Sep 12 12:57:54 2024 2024-09-12T12:57:54.930 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring state: enabled 2024-09-12T12:57:54.930 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring mode: snapshot 2024-09-12T12:57:54.930 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring global id: a790a319-cd27-4a01-b342-b61db0aa2b94 2024-09-12T12:57:54.930 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring primary: true 2024-09-12T12:57:54.934 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \n\b\d ]] 2024-09-12T12:57:54.934 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \k\r\b\d ]] 2024-09-12T12:57:54.935 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo rbd --cluster cluster1 device map -t krbd mirror/image-primary6 2024-09-12T12:57:55.051 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEV=/dev/rbd5 2024-09-12T12:57:55.051 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEVS+=($DEV) 2024-09-12T12:57:55.051 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo mkfs.ext4 /dev/rbd5 2024-09-12T12:57:55.071 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:mke2fs 1.46.5 (30-Dec-2021) 2024-09-12T12:57:55.085 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Discarding device blocks: 0/2621440 done 2024-09-12T12:57:55.085 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Creating filesystem with 2621440 4k blocks and 655360 inodes 2024-09-12T12:57:55.085 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Filesystem UUID: f97e060b-2b23-4085-9872-1fd119c45498 2024-09-12T12:57:55.085 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Superblock backups stored on blocks: 2024-09-12T12:57:55.085 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 2024-09-12T12:57:55.086 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 2024-09-12T12:57:55.086 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Allocating group tables: 0/80 done 2024-09-12T12:57:55.086 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Writing inode tables: 0/80 done 2024-09-12T12:57:55.088 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Creating journal (16384 blocks): done 2024-09-12T12:57:55.123 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Writing superblocks and filesystem accounting information: 0/80 done 2024-09-12T12:57:55.124 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 2024-09-12T12:57:55.126 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mkdir test-primary6 2024-09-12T12:57:55.128 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo mount /dev/rbd5 test-primary6 2024-09-12T12:57:55.182 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ whoami 2024-09-12T12:57:55.184 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo chown ubuntu test-primary6 2024-09-12T12:57:55.208 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ SNAP_PIDS+=($!) 2024-09-12T12:57:55.209 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ take_mirror_snapshots cluster1 mirror image-primary6 2024-09-12T12:57:55.209 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:57:55.209 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:57:55.209 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T12:57:55.209 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:57:55.209 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T12:57:55.209 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:57:55.209 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:57:55.209 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T12:57:55.209 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T12:57:55.209 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ WORKLOAD_PIDS+=($!) 2024-09-12T12:57:55.209 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T12:57:55.210 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ IMG=image-primary7 2024-09-12T12:57:55.210 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ MNTPT=test-primary7 2024-09-12T12:57:55.210 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ create_image_and_enable_mirror cluster1 mirror image-primary7 snapshot 10G 2024-09-12T12:57:55.210 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:57:55.210 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T12:57:55.210 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:57:55.210 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T12:57:55.210 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T12:57:55.210 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T12:57:55.210 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mode=snapshot 2024-09-12T12:57:55.211 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n snapshot ']' 2024-09-12T12:57:55.211 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T12:57:55.211 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ create_image cluster1 mirror image-primary7 10G 2024-09-12T12:57:55.211 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:57:55.211 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T12:57:55.211 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:57:55.211 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T12:57:55.211 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T12:57:55.211 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T12:57:55.211 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local size=128 2024-09-12T12:57:55.212 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n 10G ']' 2024-09-12T12:57:55.212 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ size=10G 2024-09-12T12:57:55.212 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T12:57:55.212 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 create --size 10G --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image-primary7 2024-09-12T12:57:55.212 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ slow_untar_workload test-primary6 2024-09-12T12:57:55.212 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mountpt=test-primary6 2024-09-12T12:57:55.212 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ cp linux-5.4.tar.gz test-primary6 2024-09-12T12:57:55.290 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ is_pool_mirror_mode_image mirror 2024-09-12T12:57:55.290 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:57:55.290 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' image = image ']' 2024-09-12T12:57:55.290 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T12:57:55.290 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ enable_mirror cluster1 mirror image-primary7 snapshot 2024-09-12T12:57:55.290 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:57:55.290 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:57:55.291 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T12:57:55.291 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mode=snapshot 2024-09-12T12:57:55.291 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 mirror image enable mirror/image-primary7 snapshot 2024-09-12T12:57:55.498 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local ret=0 2024-09-12T12:57:55.498 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ timeout 5m bash -c 'zcat test-primary6/linux-5.4.tar.gz | pv -L 256K | tar xf - -C test-primary6' 2024-09-12T12:57:55.863 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Mirroring enabled 2024-09-12T12:57:55.866 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 16 2024-09-12T12:57:55.869 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 15 2024-09-12T12:57:55.871 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:57:55.871 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T12:57:55.871 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:57:55.872 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:57:55.872 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T12:57:55.872 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T12:57:55.879 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 info mirror/image-primary7 2024-09-12T12:57:55.881 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:57:55.882 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:57:55.938 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:rbd image 'image-primary7': 2024-09-12T12:57:55.938 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: size 10 GiB in 2560 objects 2024-09-12T12:57:55.938 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: order 22 (4 MiB objects) 2024-09-12T12:57:55.938 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: snapshot_count: 1 2024-09-12T12:57:55.938 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: id: 10f13e53326c 2024-09-12T12:57:55.938 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: block_name_prefix: rbd_data.10f13e53326c 2024-09-12T12:57:55.938 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: format: 2 2024-09-12T12:57:55.938 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: features: layering, exclusive-lock, object-map, fast-diff 2024-09-12T12:57:55.938 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: op_features: 2024-09-12T12:57:55.938 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: flags: 2024-09-12T12:57:55.939 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: create_timestamp: Thu Sep 12 12:57:55 2024 2024-09-12T12:57:55.939 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: access_timestamp: Thu Sep 12 12:57:55 2024 2024-09-12T12:57:55.939 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: modify_timestamp: Thu Sep 12 12:57:55 2024 2024-09-12T12:57:55.939 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring state: enabled 2024-09-12T12:57:55.939 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring mode: snapshot 2024-09-12T12:57:55.939 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring global id: 59cd13a4-5c88-4a91-a100-f622774b37f9 2024-09-12T12:57:55.939 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring primary: true 2024-09-12T12:57:55.943 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \n\b\d ]] 2024-09-12T12:57:55.943 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \k\r\b\d ]] 2024-09-12T12:57:55.944 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo rbd --cluster cluster1 device map -t krbd mirror/image-primary7 2024-09-12T12:57:56.060 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEV=/dev/rbd6 2024-09-12T12:57:56.060 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEVS+=($DEV) 2024-09-12T12:57:56.060 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo mkfs.ext4 /dev/rbd6 2024-09-12T12:57:56.081 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:mke2fs 1.46.5 (30-Dec-2021) 2024-09-12T12:57:56.094 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Discarding device blocks: 0/2621440 done 2024-09-12T12:57:56.094 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Creating filesystem with 2621440 4k blocks and 655360 inodes 2024-09-12T12:57:56.094 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Filesystem UUID: 899526ba-1d04-47b5-ab3c-46c32a352970 2024-09-12T12:57:56.094 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Superblock backups stored on blocks: 2024-09-12T12:57:56.094 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 2024-09-12T12:57:56.094 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 2024-09-12T12:57:56.094 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Allocating group tables: 0/80 done 2024-09-12T12:57:56.094 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Writing inode tables: 0/80 done 2024-09-12T12:57:56.099 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Creating journal (16384 blocks): done 2024-09-12T12:57:56.139 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Writing superblocks and filesystem accounting information: 0/80 done 2024-09-12T12:57:56.139 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 2024-09-12T12:57:56.141 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mkdir test-primary7 2024-09-12T12:57:56.144 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo mount /dev/rbd6 test-primary7 2024-09-12T12:57:56.199 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ whoami 2024-09-12T12:57:56.201 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo chown ubuntu test-primary7 2024-09-12T12:57:56.227 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ SNAP_PIDS+=($!) 2024-09-12T12:57:56.227 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ take_mirror_snapshots cluster1 mirror image-primary7 2024-09-12T12:57:56.227 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:57:56.227 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:57:56.227 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T12:57:56.228 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:57:56.228 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T12:57:56.228 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:57:56.228 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:57:56.228 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T12:57:56.228 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T12:57:56.228 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ WORKLOAD_PIDS+=($!) 2024-09-12T12:57:56.228 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T12:57:56.228 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ IMG=image-primary8 2024-09-12T12:57:56.228 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ MNTPT=test-primary8 2024-09-12T12:57:56.228 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ create_image_and_enable_mirror cluster1 mirror image-primary8 snapshot 10G 2024-09-12T12:57:56.229 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ slow_untar_workload test-primary7 2024-09-12T12:57:56.229 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:57:56.229 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T12:57:56.229 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:57:56.229 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T12:57:56.229 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T12:57:56.229 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T12:57:56.229 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mode=snapshot 2024-09-12T12:57:56.230 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mountpt=test-primary7 2024-09-12T12:57:56.230 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n snapshot ']' 2024-09-12T12:57:56.230 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T12:57:56.230 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ create_image cluster1 mirror image-primary8 10G 2024-09-12T12:57:56.230 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ cp linux-5.4.tar.gz test-primary7 2024-09-12T12:57:56.230 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:57:56.230 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T12:57:56.230 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:57:56.230 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T12:57:56.230 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T12:57:56.230 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T12:57:56.230 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local size=128 2024-09-12T12:57:56.231 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n 10G ']' 2024-09-12T12:57:56.231 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ size=10G 2024-09-12T12:57:56.231 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T12:57:56.231 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 create --size 10G --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image-primary8 2024-09-12T12:57:56.331 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ is_pool_mirror_mode_image mirror 2024-09-12T12:57:56.331 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:57:56.331 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' image = image ']' 2024-09-12T12:57:56.331 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T12:57:56.331 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ enable_mirror cluster1 mirror image-primary8 snapshot 2024-09-12T12:57:56.331 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:57:56.331 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:57:56.331 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T12:57:56.331 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mode=snapshot 2024-09-12T12:57:56.332 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 mirror image enable mirror/image-primary8 snapshot 2024-09-12T12:57:56.535 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local ret=0 2024-09-12T12:57:56.535 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ timeout 5m bash -c 'zcat test-primary7/linux-5.4.tar.gz | pv -L 256K | tar xf - -C test-primary7' 2024-09-12T12:57:56.869 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Mirroring enabled 2024-09-12T12:57:56.870 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 19 2024-09-12T12:57:56.871 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:57:56.871 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T12:57:56.872 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:57:56.872 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:57:56.872 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T12:57:56.872 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T12:57:56.873 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:57:56.873 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T12:57:56.873 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:57:56.873 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:57:56.873 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T12:57:56.873 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T12:57:56.874 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 18 2024-09-12T12:57:56.883 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:57:56.884 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 info mirror/image-primary8 2024-09-12T12:57:56.888 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:57:56.948 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:rbd image 'image-primary8': 2024-09-12T12:57:56.948 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: size 10 GiB in 2560 objects 2024-09-12T12:57:56.948 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: order 22 (4 MiB objects) 2024-09-12T12:57:56.948 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: snapshot_count: 1 2024-09-12T12:57:56.949 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: id: 11037e005f55 2024-09-12T12:57:56.949 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: block_name_prefix: rbd_data.11037e005f55 2024-09-12T12:57:56.949 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: format: 2 2024-09-12T12:57:56.949 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: features: layering, exclusive-lock, object-map, fast-diff 2024-09-12T12:57:56.949 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: op_features: 2024-09-12T12:57:56.949 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: flags: 2024-09-12T12:57:56.949 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: create_timestamp: Thu Sep 12 12:57:56 2024 2024-09-12T12:57:56.949 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: access_timestamp: Thu Sep 12 12:57:56 2024 2024-09-12T12:57:56.949 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: modify_timestamp: Thu Sep 12 12:57:56 2024 2024-09-12T12:57:56.949 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring state: enabled 2024-09-12T12:57:56.949 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring mode: snapshot 2024-09-12T12:57:56.949 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring global id: e5807591-ea6d-4541-b1c3-0785c5e797ba 2024-09-12T12:57:56.949 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring primary: true 2024-09-12T12:57:56.953 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \n\b\d ]] 2024-09-12T12:57:56.953 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \k\r\b\d ]] 2024-09-12T12:57:56.954 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo rbd --cluster cluster1 device map -t krbd mirror/image-primary8 2024-09-12T12:57:57.068 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEV=/dev/rbd7 2024-09-12T12:57:57.069 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEVS+=($DEV) 2024-09-12T12:57:57.069 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo mkfs.ext4 /dev/rbd7 2024-09-12T12:57:57.089 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:mke2fs 1.46.5 (30-Dec-2021) 2024-09-12T12:57:57.100 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Discarding device blocks: 0/2621440 done 2024-09-12T12:57:57.100 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Creating filesystem with 2621440 4k blocks and 655360 inodes 2024-09-12T12:57:57.100 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Filesystem UUID: be0462af-39a8-46d1-a53b-ef2bbb32e01f 2024-09-12T12:57:57.100 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Superblock backups stored on blocks: 2024-09-12T12:57:57.100 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 2024-09-12T12:57:57.100 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 2024-09-12T12:57:57.100 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Allocating group tables: 0/80 done 2024-09-12T12:57:57.101 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Writing inode tables: 0/80 done 2024-09-12T12:57:57.105 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Creating journal (16384 blocks): done 2024-09-12T12:57:57.145 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Writing superblocks and filesystem accounting information: 0/80 done 2024-09-12T12:57:57.145 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 2024-09-12T12:57:57.147 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mkdir test-primary8 2024-09-12T12:57:57.150 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo mount /dev/rbd7 test-primary8 2024-09-12T12:57:57.203 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ whoami 2024-09-12T12:57:57.205 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo chown ubuntu test-primary8 2024-09-12T12:57:57.230 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ SNAP_PIDS+=($!) 2024-09-12T12:57:57.230 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ take_mirror_snapshots cluster1 mirror image-primary8 2024-09-12T12:57:57.230 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:57:57.230 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:57:57.230 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T12:57:57.231 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:57:57.231 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T12:57:57.231 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:57:57.231 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:57:57.231 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T12:57:57.231 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T12:57:57.231 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ WORKLOAD_PIDS+=($!) 2024-09-12T12:57:57.231 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T12:57:57.231 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ slow_untar_workload test-primary8 2024-09-12T12:57:57.231 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ IMG=image-primary9 2024-09-12T12:57:57.231 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ MNTPT=test-primary9 2024-09-12T12:57:57.231 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ create_image_and_enable_mirror cluster1 mirror image-primary9 snapshot 10G 2024-09-12T12:57:57.231 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mountpt=test-primary8 2024-09-12T12:57:57.232 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:57:57.232 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ cp linux-5.4.tar.gz test-primary8 2024-09-12T12:57:57.232 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T12:57:57.232 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:57:57.232 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T12:57:57.232 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T12:57:57.232 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T12:57:57.232 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mode=snapshot 2024-09-12T12:57:57.232 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n snapshot ']' 2024-09-12T12:57:57.232 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T12:57:57.232 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ create_image cluster1 mirror image-primary9 10G 2024-09-12T12:57:57.232 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:57:57.232 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T12:57:57.232 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:57:57.233 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T12:57:57.233 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T12:57:57.233 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T12:57:57.233 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local size=128 2024-09-12T12:57:57.233 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n 10G ']' 2024-09-12T12:57:57.233 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ size=10G 2024-09-12T12:57:57.233 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T12:57:57.233 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 create --size 10G --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image-primary9 2024-09-12T12:57:57.365 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ is_pool_mirror_mode_image mirror 2024-09-12T12:57:57.365 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:57:57.365 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' image = image ']' 2024-09-12T12:57:57.365 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T12:57:57.365 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ enable_mirror cluster1 mirror image-primary9 snapshot 2024-09-12T12:57:57.365 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:57:57.365 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:57:57.365 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T12:57:57.365 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mode=snapshot 2024-09-12T12:57:57.366 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 mirror image enable mirror/image-primary9 snapshot 2024-09-12T12:57:57.536 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local ret=0 2024-09-12T12:57:57.537 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ timeout 5m bash -c 'zcat test-primary8/linux-5.4.tar.gz | pv -L 256K | tar xf - -C test-primary8' 2024-09-12T12:57:57.871 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:57:57.871 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T12:57:57.871 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:57:57.871 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:57:57.871 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T12:57:57.871 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T12:57:57.878 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Mirroring enabled 2024-09-12T12:57:57.881 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 22 2024-09-12T12:57:57.885 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 23 2024-09-12T12:57:57.891 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 21 2024-09-12T12:57:57.897 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 info mirror/image-primary9 2024-09-12T12:57:57.898 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:57:57.902 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:57:57.906 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:57:57.957 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:rbd image 'image-primary9': 2024-09-12T12:57:57.958 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: size 10 GiB in 2560 objects 2024-09-12T12:57:57.958 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: order 22 (4 MiB objects) 2024-09-12T12:57:57.958 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: snapshot_count: 1 2024-09-12T12:57:57.958 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: id: 1118c9970411 2024-09-12T12:57:57.958 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: block_name_prefix: rbd_data.1118c9970411 2024-09-12T12:57:57.958 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: format: 2 2024-09-12T12:57:57.958 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: features: layering, exclusive-lock, object-map, fast-diff 2024-09-12T12:57:57.958 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: op_features: 2024-09-12T12:57:57.958 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: flags: 2024-09-12T12:57:57.958 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: create_timestamp: Thu Sep 12 12:57:57 2024 2024-09-12T12:57:57.958 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: access_timestamp: Thu Sep 12 12:57:57 2024 2024-09-12T12:57:57.958 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: modify_timestamp: Thu Sep 12 12:57:57 2024 2024-09-12T12:57:57.959 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring state: enabled 2024-09-12T12:57:57.959 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring mode: snapshot 2024-09-12T12:57:57.959 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring global id: 7d07d66e-fe3f-46b2-83dc-67fa35b43f0e 2024-09-12T12:57:57.959 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring primary: true 2024-09-12T12:57:57.963 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \n\b\d ]] 2024-09-12T12:57:57.963 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \k\r\b\d ]] 2024-09-12T12:57:57.964 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo rbd --cluster cluster1 device map -t krbd mirror/image-primary9 2024-09-12T12:57:58.084 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEV=/dev/rbd8 2024-09-12T12:57:58.084 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEVS+=($DEV) 2024-09-12T12:57:58.084 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo mkfs.ext4 /dev/rbd8 2024-09-12T12:57:58.105 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:mke2fs 1.46.5 (30-Dec-2021) 2024-09-12T12:57:58.117 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Discarding device blocks: 0/2621440 done 2024-09-12T12:57:58.117 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Creating filesystem with 2621440 4k blocks and 655360 inodes 2024-09-12T12:57:58.117 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Filesystem UUID: e1c9ee08-baa7-42c6-a99c-37796f25d135 2024-09-12T12:57:58.117 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Superblock backups stored on blocks: 2024-09-12T12:57:58.117 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 2024-09-12T12:57:58.117 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 2024-09-12T12:57:58.117 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Allocating group tables: 0/80 done 2024-09-12T12:57:58.117 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Writing inode tables: 0/80 done 2024-09-12T12:57:58.122 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Creating journal (16384 blocks): done 2024-09-12T12:57:58.162 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Writing superblocks and filesystem accounting information: 0/80 done 2024-09-12T12:57:58.162 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 2024-09-12T12:57:58.163 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mkdir test-primary9 2024-09-12T12:57:58.166 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo mount /dev/rbd8 test-primary9 2024-09-12T12:57:58.220 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ whoami 2024-09-12T12:57:58.222 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo chown ubuntu test-primary9 2024-09-12T12:57:58.247 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ SNAP_PIDS+=($!) 2024-09-12T12:57:58.247 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ take_mirror_snapshots cluster1 mirror image-primary9 2024-09-12T12:57:58.247 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:57:58.247 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:57:58.247 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T12:57:58.247 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:57:58.248 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T12:57:58.248 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:57:58.248 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:57:58.248 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T12:57:58.248 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ WORKLOAD_PIDS+=($!) 2024-09-12T12:57:58.248 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T12:57:58.248 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T12:57:58.248 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ IMG=image-primary10 2024-09-12T12:57:58.248 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ MNTPT=test-primary10 2024-09-12T12:57:58.248 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ slow_untar_workload test-primary9 2024-09-12T12:57:58.248 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ create_image_and_enable_mirror cluster1 mirror image-primary10 snapshot 10G 2024-09-12T12:57:58.248 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:57:58.248 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T12:57:58.249 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:57:58.249 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T12:57:58.249 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T12:57:58.249 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mountpt=test-primary9 2024-09-12T12:57:58.249 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T12:57:58.249 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mode=snapshot 2024-09-12T12:57:58.249 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n snapshot ']' 2024-09-12T12:57:58.249 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T12:57:58.249 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ cp linux-5.4.tar.gz test-primary9 2024-09-12T12:57:58.249 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ create_image cluster1 mirror image-primary10 10G 2024-09-12T12:57:58.249 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:57:58.249 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T12:57:58.249 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:57:58.249 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T12:57:58.250 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T12:57:58.250 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T12:57:58.250 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local size=128 2024-09-12T12:57:58.250 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n 10G ']' 2024-09-12T12:57:58.250 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ size=10G 2024-09-12T12:57:58.250 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T12:57:58.250 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 create --size 10G --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image-primary10 2024-09-12T12:57:58.382 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ is_pool_mirror_mode_image mirror 2024-09-12T12:57:58.382 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:57:58.382 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' image = image ']' 2024-09-12T12:57:58.382 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T12:57:58.382 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ enable_mirror cluster1 mirror image-primary10 snapshot 2024-09-12T12:57:58.382 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:57:58.382 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:57:58.383 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T12:57:58.383 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mode=snapshot 2024-09-12T12:57:58.383 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 mirror image enable mirror/image-primary10 snapshot 2024-09-12T12:57:58.553 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local ret=0 2024-09-12T12:57:58.553 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ timeout 5m bash -c 'zcat test-primary9/linux-5.4.tar.gz | pv -L 256K | tar xf - -C test-primary9' 2024-09-12T12:57:58.885 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:57:58.886 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T12:57:58.886 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:57:58.886 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:57:58.886 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T12:57:58.886 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T12:57:58.890 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:57:58.890 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T12:57:58.890 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:57:58.890 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:57:58.890 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T12:57:58.890 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T12:57:58.914 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 26 2024-09-12T12:57:58.915 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Mirroring enabled 2024-09-12T12:57:58.934 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 25 2024-09-12T12:57:58.960 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 info mirror/image-primary10 2024-09-12T12:57:58.964 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:57:59.008 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:57:59.059 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:rbd image 'image-primary10': 2024-09-12T12:57:59.059 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: size 10 GiB in 2560 objects 2024-09-12T12:57:59.059 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: order 22 (4 MiB objects) 2024-09-12T12:57:59.059 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: snapshot_count: 1 2024-09-12T12:57:59.059 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: id: 11281f820e59 2024-09-12T12:57:59.059 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: block_name_prefix: rbd_data.11281f820e59 2024-09-12T12:57:59.059 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: format: 2 2024-09-12T12:57:59.059 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: features: layering, exclusive-lock, object-map, fast-diff 2024-09-12T12:57:59.059 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: op_features: 2024-09-12T12:57:59.059 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: flags: 2024-09-12T12:57:59.059 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: create_timestamp: Thu Sep 12 12:57:58 2024 2024-09-12T12:57:59.060 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: access_timestamp: Thu Sep 12 12:57:58 2024 2024-09-12T12:57:59.060 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: modify_timestamp: Thu Sep 12 12:57:58 2024 2024-09-12T12:57:59.060 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring state: enabled 2024-09-12T12:57:59.060 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring mode: snapshot 2024-09-12T12:57:59.060 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring global id: e8086552-9fee-4a93-b94f-d2d6dc6bff63 2024-09-12T12:57:59.060 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring primary: true 2024-09-12T12:57:59.065 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \n\b\d ]] 2024-09-12T12:57:59.065 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \k\r\b\d ]] 2024-09-12T12:57:59.066 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo rbd --cluster cluster1 device map -t krbd mirror/image-primary10 2024-09-12T12:57:59.184 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEV=/dev/rbd9 2024-09-12T12:57:59.185 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEVS+=($DEV) 2024-09-12T12:57:59.185 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo mkfs.ext4 /dev/rbd9 2024-09-12T12:57:59.202 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:mke2fs 1.46.5 (30-Dec-2021) 2024-09-12T12:57:59.214 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Discarding device blocks: 0/2621440 done 2024-09-12T12:57:59.214 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Creating filesystem with 2621440 4k blocks and 655360 inodes 2024-09-12T12:57:59.214 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Filesystem UUID: 22950c6f-6483-4b8c-869f-b85c6a6bb3b7 2024-09-12T12:57:59.214 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Superblock backups stored on blocks: 2024-09-12T12:57:59.214 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 2024-09-12T12:57:59.214 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 2024-09-12T12:57:59.214 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Allocating group tables: 0/80 done 2024-09-12T12:57:59.214 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Writing inode tables: 0/80 done 2024-09-12T12:57:59.219 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Creating journal (16384 blocks): done 2024-09-12T12:57:59.259 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Writing superblocks and filesystem accounting information: 0/80 done 2024-09-12T12:57:59.259 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 2024-09-12T12:57:59.262 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mkdir test-primary10 2024-09-12T12:57:59.264 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo mount /dev/rbd9 test-primary10 2024-09-12T12:57:59.407 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ whoami 2024-09-12T12:57:59.409 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo chown ubuntu test-primary10 2024-09-12T12:57:59.434 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ SNAP_PIDS+=($!) 2024-09-12T12:57:59.434 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ take_mirror_snapshots cluster1 mirror image-primary10 2024-09-12T12:57:59.434 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:57:59.434 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:57:59.434 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T12:57:59.434 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:57:59.434 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T12:57:59.434 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:57:59.435 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:57:59.435 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T12:57:59.435 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T12:57:59.435 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ WORKLOAD_PIDS+=($!) 2024-09-12T12:57:59.435 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${SNAP_PIDS[@]} 2024-09-12T12:57:59.435 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 35490 2024-09-12T12:57:59.435 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ slow_untar_workload test-primary10 2024-09-12T12:57:59.435 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mountpt=test-primary10 2024-09-12T12:57:59.435 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ cp linux-5.4.tar.gz test-primary10 2024-09-12T12:57:59.780 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local ret=0 2024-09-12T12:57:59.780 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ timeout 5m bash -c 'zcat test-primary10/linux-5.4.tar.gz | pv -L 256K | tar xf - -C test-primary10' 2024-09-12T12:57:59.886 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:57:59.886 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T12:57:59.886 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:57:59.886 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:57:59.886 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T12:57:59.886 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T12:57:59.891 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:57:59.891 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T12:57:59.891 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:57:59.892 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:57:59.892 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T12:57:59.892 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T12:57:59.914 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 29 2024-09-12T12:57:59.917 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 28 2024-09-12T12:57:59.919 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 30 2024-09-12T12:57:59.928 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:57:59.929 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:57:59.929 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:58:00.887 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 31 2024-09-12T12:58:00.894 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 32 2024-09-12T12:58:00.899 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:58:00.902 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:58:00.903 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:58:00.904 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T12:58:00.904 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:58:00.904 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:58:00.904 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T12:58:00.904 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T12:58:00.905 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:58:00.905 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T12:58:00.905 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:58:00.905 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:58:00.905 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T12:58:00.906 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T12:58:00.909 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:58:00.909 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T12:58:00.909 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:58:00.909 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:58:00.909 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T12:58:00.909 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T12:58:01.891 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 33 2024-09-12T12:58:01.894 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 35 2024-09-12T12:58:01.895 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 34 2024-09-12T12:58:01.903 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:58:01.905 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:58:01.906 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:58:01.970 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:58:01.970 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T12:58:01.970 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:58:01.970 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:58:01.970 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T12:58:01.970 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T12:58:02.014 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:58:02.014 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T12:58:02.014 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:58:02.014 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:58:02.014 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T12:58:02.014 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T12:58:02.899 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 37 2024-09-12T12:58:02.900 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 36 2024-09-12T12:58:02.911 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:58:02.911 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:58:02.931 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:58:02.932 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T12:58:02.932 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:58:02.932 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:58:02.932 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T12:58:02.932 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T12:58:02.933 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:58:02.933 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T12:58:02.933 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:58:02.933 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:58:02.934 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T12:58:02.934 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T12:58:02.934 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:58:02.934 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T12:58:02.934 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:58:02.934 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:58:02.934 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T12:58:02.934 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T12:58:03.898 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 40 2024-09-12T12:58:03.902 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:58:03.902 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T12:58:03.902 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:58:03.902 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:58:03.902 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T12:58:03.902 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T12:58:03.904 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 39 2024-09-12T12:58:03.905 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 38 2024-09-12T12:58:03.905 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:58:03.905 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T12:58:03.905 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:58:03.905 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:58:03.906 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T12:58:03.906 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T12:58:03.910 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:58:03.913 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:58:03.916 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:58:04.905 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 42 2024-09-12T12:58:04.905 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:58:04.906 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T12:58:04.906 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 41 2024-09-12T12:58:04.906 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:58:04.906 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:58:04.906 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T12:58:04.907 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T12:58:04.908 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:58:04.908 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T12:58:04.908 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:58:04.908 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:58:04.908 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T12:58:04.908 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T12:58:04.909 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:58:04.909 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T12:58:04.909 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:58:04.909 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:58:04.909 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T12:58:04.909 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T12:58:04.917 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:58:04.922 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:58:05.909 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 43 2024-09-12T12:58:05.911 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 45 2024-09-12T12:58:05.915 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:58:05.915 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T12:58:05.915 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:58:05.915 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:58:05.915 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T12:58:05.915 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T12:58:05.916 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 44 2024-09-12T12:58:05.916 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:58:05.916 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T12:58:05.916 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:58:05.917 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:58:05.917 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T12:58:05.917 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T12:58:05.921 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:58:05.927 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:58:05.931 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:58:06.905 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 47 2024-09-12T12:58:06.913 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:58:06.913 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T12:58:06.913 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:58:06.913 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:58:06.913 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T12:58:06.913 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T12:58:06.913 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 46 2024-09-12T12:58:06.916 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:58:06.916 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T12:58:06.916 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:58:06.916 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:58:06.916 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T12:58:06.916 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T12:58:06.918 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:58:06.919 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:58:06.919 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T12:58:06.919 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:58:06.919 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:58:06.919 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T12:58:06.919 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T12:58:06.922 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:58:07.915 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 50 2024-09-12T12:58:07.920 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 49 2024-09-12T12:58:07.920 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:58:07.921 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T12:58:07.921 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:58:07.921 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:58:07.921 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T12:58:07.921 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T12:58:07.924 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 48 2024-09-12T12:58:07.924 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:58:07.924 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T12:58:07.924 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:58:07.924 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:58:07.925 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T12:58:07.925 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T12:58:07.929 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:58:07.935 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:58:07.944 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:58:08.921 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 51 2024-09-12T12:58:08.925 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:58:08.925 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T12:58:08.925 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:58:08.925 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:58:08.925 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T12:58:08.925 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T12:58:08.930 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 52 2024-09-12T12:58:08.931 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:58:08.931 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T12:58:08.931 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:58:08.931 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:58:08.932 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T12:58:08.932 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T12:58:08.932 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:58:08.935 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:58:08.935 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T12:58:08.935 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:58:08.935 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:58:08.935 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T12:58:08.936 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T12:58:08.949 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:58:09.921 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 55 2024-09-12T12:58:09.922 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:58:09.922 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T12:58:09.922 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:58:09.922 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:58:09.922 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T12:58:09.922 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T12:58:09.925 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:58:09.925 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T12:58:09.925 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:58:09.926 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:58:09.926 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T12:58:09.926 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T12:58:09.928 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 54 2024-09-12T12:58:09.928 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:58:09.939 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:58:10.933 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:58:10.934 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T12:58:10.934 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:58:10.934 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:58:10.934 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T12:58:10.934 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T12:58:10.937 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 57 2024-09-12T12:58:10.939 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:58:10.939 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T12:58:10.939 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:58:10.939 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:58:10.939 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T12:58:10.939 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T12:58:10.940 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 56 2024-09-12T12:58:10.946 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:58:10.946 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T12:58:10.946 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:58:10.946 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:58:10.946 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T12:58:10.946 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T12:58:10.947 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:58:10.960 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:58:11.910 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 53 2024-09-12T12:58:11.930 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:58:11.933 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:58:11.933 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T12:58:11.933 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:58:11.933 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:58:11.934 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T12:58:11.934 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T12:58:11.947 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 58 2024-09-12T12:58:11.949 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 59 2024-09-12T12:58:11.953 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:58:11.953 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T12:58:11.953 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:58:11.953 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:58:11.953 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T12:58:11.953 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T12:58:11.955 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:58:11.965 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:58:12.931 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:58:12.931 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T12:58:12.931 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:58:12.931 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:58:12.931 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T12:58:12.932 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T12:58:12.941 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:58:12.941 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T12:58:12.942 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:58:12.942 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:58:12.942 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T12:58:12.942 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T12:58:12.954 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 61 2024-09-12T12:58:12.964 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:58:13.917 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 60 2024-09-12T12:58:13.950 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:58:13.951 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T12:58:13.951 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:58:13.951 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:58:13.951 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T12:58:13.951 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T12:58:13.955 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:58:13.963 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:58:13.964 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T12:58:13.964 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:58:13.964 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:58:13.964 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T12:58:13.965 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T12:58:13.982 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 64 2024-09-12T12:58:13.993 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:58:14.931 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 62 2024-09-12T12:58:14.932 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:58:14.933 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T12:58:14.933 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:58:14.933 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:58:14.933 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T12:58:14.933 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T12:58:14.958 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:58:14.958 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T12:58:14.958 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:58:14.958 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:58:14.958 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T12:58:14.958 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T12:58:14.958 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:58:14.968 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:58:14.968 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T12:58:14.969 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:58:14.969 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:58:14.969 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T12:58:14.969 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T12:58:14.983 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 65 2024-09-12T12:58:14.992 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:58:15.941 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 63 2024-09-12T12:58:15.967 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:58:15.967 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T12:58:15.967 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:58:15.967 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:58:15.967 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T12:58:15.967 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T12:58:15.968 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:58:15.993 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 68 2024-09-12T12:58:16.008 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 67 2024-09-12T12:58:16.018 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:58:16.019 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:58:16.927 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 66 2024-09-12T12:58:16.955 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:58:16.957 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:58:16.957 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T12:58:16.957 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:58:16.957 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:58:16.957 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T12:58:16.957 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T12:58:16.995 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:58:16.995 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T12:58:16.996 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:58:16.996 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:58:16.996 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T12:58:16.996 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T12:58:17.680 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 69 2024-09-12T12:58:17.703 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:58:17.711 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T12:58:17.711+0000 7f67dc600640 -1 rbd::mirror::image_replayer::snapshot::Replayer: 0x56055689a400 handle_unlink_peer: failed to unlink local peer from remote image: (22) Invalid argument 2024-09-12T12:58:17.726 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 71 2024-09-12T12:58:17.751 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:58:17.960 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:58:17.961 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T12:58:17.961 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:58:17.961 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:58:17.961 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T12:58:17.961 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T12:58:17.995 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:58:17.995 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T12:58:17.995 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:58:17.995 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:58:17.995 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T12:58:17.995 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T12:58:18.675 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 70 2024-09-12T12:58:18.717 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:58:18.755 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 73 2024-09-12T12:58:18.779 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T12:58:18.779+0000 7f67dc600640 -1 rbd::mirror::image_replayer::snapshot::Replayer: 0x5605563cd400 handle_unlink_peer: failed to unlink local peer from remote image: (22) Invalid argument 2024-09-12T12:58:18.791 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:58:18.972 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:58:18.972 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T12:58:18.972 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:58:18.972 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:58:18.972 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T12:58:18.972 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T12:58:19.021 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:58:19.021 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T12:58:19.021 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:58:19.021 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:58:19.021 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T12:58:19.022 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T12:58:19.022 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:58:19.022 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T12:58:19.022 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:58:19.022 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:58:19.023 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T12:58:19.023 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T12:58:19.687 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 72 2024-09-12T12:58:19.730 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:58:19.957 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:58:19.958 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T12:58:19.958 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:58:19.958 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:58:19.958 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T12:58:19.958 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T12:58:20.681 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 74 2024-09-12T12:58:20.707 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:58:20.707 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T12:58:20.707 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:58:20.708 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:58:20.708 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T12:58:20.708 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T12:58:20.747 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 75 2024-09-12T12:58:20.757 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:58:20.757 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:58:20.757 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T12:58:20.757 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:58:20.758 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:58:20.758 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T12:58:20.758 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T12:58:20.799 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:58:20.812 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 76 2024-09-12T12:58:20.853 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:58:21.709 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 77 2024-09-12T12:58:21.721 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:58:21.722 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T12:58:21.722 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:58:21.722 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:58:21.722 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T12:58:21.722 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T12:58:21.725 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:58:21.758 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 80 2024-09-12T12:58:21.776 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:58:21.796 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:58:21.796 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T12:58:21.796 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:58:21.796 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:58:21.796 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T12:58:21.796 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T12:58:22.734 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:58:22.735 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T12:58:22.735 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:58:22.735 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:58:22.735 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T12:58:22.735 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T12:58:22.736 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 79 2024-09-12T12:58:22.762 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:58:22.813 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 81 2024-09-12T12:58:22.865 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:58:23.719 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 78 2024-09-12T12:58:23.727 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 82 2024-09-12T12:58:23.743 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:58:23.750 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:58:23.759 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:58:23.759 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T12:58:23.759 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:58:23.759 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:58:23.760 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T12:58:23.760 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T12:58:23.780 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 83 2024-09-12T12:58:23.790 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:58:23.802 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:58:23.802 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T12:58:23.802 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:58:23.802 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:58:23.802 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T12:58:23.802 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T12:58:23.858 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:58:23.858 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T12:58:23.859 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:58:23.859 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:58:23.859 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T12:58:23.859 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T12:58:24.708 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 85 2024-09-12T12:58:24.718 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:58:24.728 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:58:24.728 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T12:58:24.728 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:58:24.728 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:58:24.728 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T12:58:24.729 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T12:58:24.779 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:58:24.779 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T12:58:24.779 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:58:24.780 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:58:24.780 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T12:58:24.780 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T12:58:25.756 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 86 2024-09-12T12:58:25.767 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 84 2024-09-12T12:58:25.769 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:58:25.769 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T12:58:25.769 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:58:25.769 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:58:25.769 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T12:58:25.769 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T12:58:25.784 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:58:25.801 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:58:25.871 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:58:25.872 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T12:58:25.872 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:58:25.872 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:58:25.872 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T12:58:25.872 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T12:58:26.723 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 87 2024-09-12T12:58:26.747 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:58:26.747 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T12:58:26.748 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:58:26.748 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:58:26.748 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T12:58:26.748 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T12:58:26.753 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:58:26.753 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T12:58:26.753 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:58:26.753 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:58:26.753 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T12:58:26.753 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T12:58:26.768 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 88 2024-09-12T12:58:26.794 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:58:26.794 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T12:58:26.794 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:58:26.794 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:58:26.794 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T12:58:26.794 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T12:58:26.800 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:58:26.845 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:58:27.704 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 90 2024-09-12T12:58:27.719 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 89 2024-09-12T12:58:27.720 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:58:27.721 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T12:58:27.721 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:58:27.721 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:58:27.722 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T12:58:27.722 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T12:58:27.734 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:58:27.742 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:58:27.779 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 93 2024-09-12T12:58:27.797 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:58:28.743 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 91 2024-09-12T12:58:28.752 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 92 2024-09-12T12:58:28.768 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:58:28.771 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:58:28.792 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:58:28.792 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T12:58:28.792 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:58:28.793 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:58:28.793 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T12:58:28.793 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T12:58:28.808 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:58:28.809 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T12:58:28.809 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:58:28.809 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:58:28.809 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T12:58:28.809 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T12:58:29.736 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 94 2024-09-12T12:58:29.753 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:58:29.804 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:58:29.804 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T12:58:29.804 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:58:29.804 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:58:29.804 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T12:58:29.804 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T12:58:29.849 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:58:29.849 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T12:58:29.849 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:58:29.849 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:58:29.850 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T12:58:29.850 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T12:58:30.731 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 96 2024-09-12T12:58:30.738 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 95 2024-09-12T12:58:30.740 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:58:30.740 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T12:58:30.740 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:58:30.740 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:58:30.740 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T12:58:30.740 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T12:58:30.747 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:58:30.747 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T12:58:30.747 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:58:30.747 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:58:30.747 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T12:58:30.747 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T12:58:30.758 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:58:30.760 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:58:30.802 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:58:30.802 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T12:58:30.802 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:58:30.802 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:58:30.802 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T12:58:30.802 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T12:58:31.744 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 98 2024-09-12T12:58:31.749 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 97 2024-09-12T12:58:31.761 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 101 2024-09-12T12:58:31.763 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:58:31.765 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:58:31.771 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:58:31.772 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T12:58:31.772 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:58:31.772 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:58:31.772 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T12:58:31.772 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T12:58:31.776 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:58:31.776 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T12:58:31.777 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:58:31.777 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:58:31.777 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T12:58:31.777 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T12:58:31.777 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:58:32.756 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:58:32.756 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T12:58:32.756 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:58:32.756 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:58:32.757 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T12:58:32.757 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T12:58:32.767 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 100 2024-09-12T12:58:32.780 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 99 2024-09-12T12:58:32.794 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:58:32.797 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:58:33.743 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 102 2024-09-12T12:58:33.746 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 103 2024-09-12T12:58:33.763 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:58:33.763 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T12:58:33.763 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:58:33.763 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:58:33.763 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T12:58:33.763 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T12:58:33.765 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:58:33.765 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T12:58:33.765 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:58:33.765 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:58:33.766 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T12:58:33.766 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T12:58:33.771 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:58:33.778 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:58:34.766 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:58:34.766 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T12:58:34.766 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:58:34.766 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:58:34.766 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T12:58:34.766 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T12:58:34.768 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:58:34.768 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T12:58:34.768 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:58:34.768 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:58:34.768 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T12:58:34.768 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T12:58:34.780 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:58:34.780 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T12:58:34.780 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:58:34.781 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:58:34.781 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T12:58:34.781 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T12:58:35.732 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 104 2024-09-12T12:58:35.762 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:58:35.788 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 109 2024-09-12T12:58:35.797 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:58:35.797 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T12:58:35.797 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:58:35.797 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:58:35.797 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T12:58:35.797 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T12:58:35.801 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:58:35.801 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T12:58:35.801 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:58:35.801 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:58:35.801 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T12:58:35.801 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T12:58:35.804 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:58:36.722 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 105 2024-09-12T12:58:36.740 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 106 2024-09-12T12:58:36.774 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:58:36.774 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T12:58:36.774 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:58:36.775 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:58:36.775 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T12:58:36.775 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T12:58:36.782 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:58:36.783 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:58:36.784 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:58:36.784 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T12:58:36.784 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:58:36.784 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:58:36.784 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T12:58:36.784 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T12:58:37.672 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 108 2024-09-12T12:58:37.693 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 107 2024-09-12T12:58:37.710 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:58:37.729 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:58:38.734 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 110 2024-09-12T12:58:38.748 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 111 2024-09-12T12:58:38.765 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:58:38.765 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T12:58:38.765 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:58:38.765 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:58:38.765 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T12:58:38.765 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T12:58:38.773 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:58:38.785 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:58:38.806 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:58:38.807 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T12:58:38.807 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:58:38.807 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:58:38.807 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T12:58:38.807 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T12:58:39.732 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 112 2024-09-12T12:58:39.745 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 113 2024-09-12T12:58:39.767 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:58:39.768 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:58:39.785 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:58:39.785 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T12:58:39.786 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 114 2024-09-12T12:58:39.786 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:58:39.787 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:58:39.787 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T12:58:39.787 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T12:58:39.787 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:58:39.787 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T12:58:39.787 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:58:39.787 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:58:39.787 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T12:58:39.787 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T12:58:39.797 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:58:40.713 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:58:40.714 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T12:58:40.714 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:58:40.714 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:58:40.714 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T12:58:40.714 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T12:58:40.726 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 115 2024-09-12T12:58:40.731 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:58:40.732 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T12:58:40.732 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:58:40.732 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:58:40.732 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T12:58:40.732 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T12:58:40.748 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:58:40.774 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 117 2024-09-12T12:58:40.776 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 116 2024-09-12T12:58:40.786 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:58:40.788 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:58:41.759 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 118 2024-09-12T12:58:41.761 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 119 2024-09-12T12:58:41.771 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:58:41.773 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:58:41.776 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:58:41.776 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T12:58:41.776 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:58:41.776 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:58:41.777 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T12:58:41.777 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T12:58:41.788 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:58:41.788 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T12:58:41.788 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:58:41.789 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:58:41.789 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T12:58:41.789 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T12:58:42.770 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:58:42.770 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T12:58:42.770 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:58:42.770 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:58:42.770 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T12:58:42.770 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T12:58:42.771 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:58:42.771 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T12:58:42.771 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:58:42.771 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:58:42.771 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T12:58:42.771 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T12:58:42.773 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 120 2024-09-12T12:58:42.775 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 121 2024-09-12T12:58:42.782 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:58:42.785 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:58:42.800 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:58:42.800 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T12:58:42.800 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:58:42.800 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:58:42.800 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T12:58:42.800 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T12:58:43.751 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:58:43.751 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T12:58:43.751 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:58:43.751 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:58:43.751 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T12:58:43.751 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T12:58:43.768 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 122 2024-09-12T12:58:43.785 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 123 2024-09-12T12:58:43.787 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:58:43.788 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:58:43.788 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T12:58:43.788 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:58:43.789 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:58:43.789 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T12:58:43.789 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T12:58:43.790 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:58:43.791 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:58:43.791 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T12:58:43.791 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:58:43.792 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:58:43.792 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T12:58:43.792 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T12:58:44.773 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:58:44.774 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T12:58:44.774 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:58:44.774 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:58:44.774 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T12:58:44.774 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T12:58:44.775 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:58:44.775 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T12:58:44.775 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:58:44.776 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:58:44.776 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T12:58:44.776 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T12:58:45.748 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 124 2024-09-12T12:58:45.780 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 125 2024-09-12T12:58:45.784 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:58:45.784 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T12:58:45.784 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:58:45.784 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:58:45.784 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T12:58:45.784 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T12:58:45.788 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:58:45.788 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T12:58:45.788 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:58:45.788 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:58:45.788 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T12:58:45.788 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T12:58:45.789 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:58:45.816 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:58:46.768 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 126 2024-09-12T12:58:46.777 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 127 2024-09-12T12:58:46.788 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:58:46.789 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T12:58:46.789 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:58:46.789 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:58:46.789 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T12:58:46.789 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T12:58:46.794 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:58:46.794 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T12:58:46.794 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:58:46.794 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:58:46.794 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T12:58:46.794 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T12:58:46.813 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:58:46.814 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:58:47.656 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 129 2024-09-12T12:58:47.676 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 128 2024-09-12T12:58:47.706 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:58:47.747 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:58:47.776 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 131 2024-09-12T12:58:47.787 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:58:48.759 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 130 2024-09-12T12:58:48.792 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:58:48.792 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T12:58:48.793 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:58:48.793 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:58:48.793 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T12:58:48.793 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T12:58:48.793 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:58:48.820 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:58:48.820 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T12:58:48.821 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:58:48.821 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:58:48.821 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T12:58:48.821 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T12:58:49.772 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 133 2024-09-12T12:58:49.781 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 132 2024-09-12T12:58:49.806 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:58:49.806 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:58:49.818 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:58:49.818 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T12:58:49.818 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:58:49.818 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:58:49.818 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T12:58:49.818 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T12:58:49.818 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:58:49.818 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T12:58:49.818 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:58:49.818 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:58:49.819 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T12:58:49.819 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T12:58:49.821 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 134 2024-09-12T12:58:49.832 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:58:50.709 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:58:50.709 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T12:58:50.709 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:58:50.709 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:58:50.709 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T12:58:50.710 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T12:58:50.751 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:58:50.751 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T12:58:50.751 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:58:50.751 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:58:50.751 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T12:58:50.751 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T12:58:50.790 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:58:50.790 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T12:58:50.790 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:58:50.791 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:58:50.791 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T12:58:50.791 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T12:58:50.792 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 135 2024-09-12T12:58:50.809 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:58:50.816 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 137 2024-09-12T12:58:50.819 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 136 2024-09-12T12:58:50.828 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:58:50.833 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:58:51.800 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:58:51.800 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T12:58:51.800 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:58:51.800 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:58:51.800 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T12:58:51.800 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T12:58:51.809 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 138 2024-09-12T12:58:51.816 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 139 2024-09-12T12:58:51.834 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:58:51.838 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:58:52.810 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:58:52.810 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T12:58:52.810 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:58:52.810 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:58:52.810 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T12:58:52.810 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T12:58:52.810 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:58:52.810 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T12:58:52.810 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:58:52.810 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:58:52.811 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T12:58:52.811 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T12:58:52.820 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 141 2024-09-12T12:58:52.833 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:58:52.835 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:58:52.836 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T12:58:52.836 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:58:52.836 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:58:52.836 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T12:58:52.837 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T12:58:53.780 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 140 2024-09-12T12:58:53.795 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:58:53.812 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 143 2024-09-12T12:58:53.813 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:58:53.813 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T12:58:53.813 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:58:53.814 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:58:53.814 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T12:58:53.814 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T12:58:53.815 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 142 2024-09-12T12:58:53.827 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:58:53.828 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:58:53.831 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:58:53.831 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T12:58:53.831 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:58:53.831 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:58:53.831 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T12:58:53.831 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T12:58:53.837 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:58:53.837 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T12:58:53.837 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:58:53.837 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:58:53.837 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T12:58:53.837 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T12:58:54.837 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:58:54.838 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T12:58:54.838 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:58:54.838 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:58:54.838 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T12:58:54.838 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T12:58:54.839 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:58:54.840 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T12:58:54.840 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:58:54.840 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:58:54.840 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T12:58:54.840 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T12:58:55.784 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 144 2024-09-12T12:58:55.834 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:58:55.837 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:58:55.837 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T12:58:55.837 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:58:55.837 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:58:55.837 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T12:58:55.837 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T12:58:56.785 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 145 2024-09-12T12:58:56.798 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:58:56.798 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T12:58:56.798 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:58:56.799 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:58:56.799 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T12:58:56.799 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T12:58:56.816 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 147 2024-09-12T12:58:56.831 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:58:56.831 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T12:58:56.831 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:58:56.831 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:58:56.831 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T12:58:56.831 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T12:58:56.831 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:58:56.831 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T12:58:56.831 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:58:56.831 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:58:56.831 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T12:58:56.831 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T12:58:56.832 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:58:56.854 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:58:57.677 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 148 2024-09-12T12:58:57.681 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 149 2024-09-12T12:58:57.726 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:58:57.727 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:58:57.783 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T12:58:57.783+0000 7f67dd000640 -1 rbd::mirror::image_replayer::snapshot::Replayer: 0x56055c2d1800 handle_unlink_peer: failed to unlink local peer from remote image: (22) Invalid argument 2024-09-12T12:58:57.785 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 151 2024-09-12T12:58:57.794 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:58:58.800 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 150 2024-09-12T12:58:58.821 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 146 2024-09-12T12:58:58.833 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:58:58.833 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:58:58.833 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T12:58:58.833 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:58:58.834 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:58:58.834 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T12:58:58.834 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T12:58:58.859 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:58:59.805 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 153 2024-09-12T12:58:59.817 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 152 2024-09-12T12:58:59.834 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:58:59.835 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T12:58:59.835 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:58:59.835 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:58:59.835 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T12:58:59.835 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T12:58:59.837 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:58:59.839 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:58:59.857 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:58:59.857 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T12:58:59.858 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:58:59.858 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:58:59.858 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T12:58:59.858 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T12:58:59.858 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 154 2024-09-12T12:58:59.868 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:59:00.730 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:59:00.730 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T12:59:00.730 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:59:00.730 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:59:00.730 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T12:59:00.731 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T12:59:00.731 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:59:00.731 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T12:59:00.731 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:59:00.732 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:59:00.732 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T12:59:00.732 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T12:59:00.797 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:59:00.797 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T12:59:00.798 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:59:00.798 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:59:00.798 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T12:59:00.798 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T12:59:00.831 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 156 2024-09-12T12:59:00.835 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 155 2024-09-12T12:59:00.844 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:59:00.846 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:59:01.827 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 158 2024-09-12T12:59:01.835 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:59:01.835 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T12:59:01.835 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:59:01.835 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:59:01.835 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T12:59:01.836 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T12:59:01.836 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 157 2024-09-12T12:59:01.841 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:59:01.843 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:59:01.861 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:59:01.861 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T12:59:01.861 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:59:01.862 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:59:01.862 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T12:59:01.862 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T12:59:02.701 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 159 2024-09-12T12:59:02.719 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 160 2024-09-12T12:59:02.724 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 161 2024-09-12T12:59:02.725 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:59:02.731 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:59:02.738 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:59:02.840 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:59:02.840 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T12:59:02.840 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:59:02.841 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:59:02.841 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T12:59:02.841 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T12:59:02.842 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:59:02.842 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T12:59:02.842 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:59:02.842 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:59:02.842 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T12:59:02.842 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T12:59:02.872 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:59:02.872 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T12:59:02.872 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:59:02.872 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:59:02.872 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T12:59:02.872 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T12:59:03.843 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 163 2024-09-12T12:59:03.847 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:59:03.847 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T12:59:03.847 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:59:03.847 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:59:03.847 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T12:59:03.847 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T12:59:03.848 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:59:03.848 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T12:59:03.849 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:59:03.849 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:59:03.849 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T12:59:03.849 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T12:59:03.860 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:59:03.863 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 162 2024-09-12T12:59:03.877 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:59:04.844 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:59:04.844 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T12:59:04.844 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:59:04.844 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:59:04.844 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T12:59:04.844 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T12:59:04.845 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:59:04.845 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T12:59:04.845 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:59:04.845 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:59:04.846 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T12:59:04.846 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T12:59:05.728 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:59:05.728 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T12:59:05.728 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:59:05.729 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:59:05.729 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T12:59:05.729 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T12:59:05.734 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:59:05.734 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T12:59:05.734 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:59:05.734 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:59:05.735 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T12:59:05.735 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T12:59:05.742 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:59:05.742 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T12:59:05.742 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:59:05.742 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:59:05.742 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T12:59:05.742 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T12:59:05.825 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 164 2024-09-12T12:59:05.858 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:59:06.830 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 166 2024-09-12T12:59:06.838 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 165 2024-09-12T12:59:06.863 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:59:06.864 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T12:59:06.864 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:59:06.864 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:59:06.864 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T12:59:06.864 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T12:59:06.864 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:59:06.878 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:59:06.880 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:59:06.880 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T12:59:06.880 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:59:06.880 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:59:06.880 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T12:59:06.880 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T12:59:06.905 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 171 2024-09-12T12:59:06.921 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:59:07.662 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 168 2024-09-12T12:59:07.673 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 167 2024-09-12T12:59:07.717 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:59:07.732 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:59:07.754 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 169 2024-09-12T12:59:07.764 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 170 2024-09-12T12:59:07.774 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:59:07.786 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:59:08.694 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 172 2024-09-12T12:59:08.708 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 173 2024-09-12T12:59:08.712 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:59:08.722 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:59:08.861 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:59:08.861 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T12:59:08.861 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:59:08.861 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:59:08.861 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T12:59:08.861 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T12:59:09.850 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 174 2024-09-12T12:59:09.857 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:59:09.866 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:59:09.866 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T12:59:09.866 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:59:09.866 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:59:09.867 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T12:59:09.867 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T12:59:09.880 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:59:09.880 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T12:59:09.880 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:59:09.880 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:59:09.880 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T12:59:09.881 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T12:59:09.923 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:59:09.923 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T12:59:09.923 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:59:09.923 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:59:09.923 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T12:59:09.923 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T12:59:10.721 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:59:10.721 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T12:59:10.721 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:59:10.721 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:59:10.721 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T12:59:10.721 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T12:59:10.736 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:59:10.736 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T12:59:10.737 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:59:10.737 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:59:10.737 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T12:59:10.737 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T12:59:10.781 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:59:10.781 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T12:59:10.781 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:59:10.781 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:59:10.781 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T12:59:10.781 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T12:59:10.790 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:59:10.790 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T12:59:10.790 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:59:10.790 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:59:10.790 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T12:59:10.790 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T12:59:10.860 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 176 2024-09-12T12:59:10.865 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 177 2024-09-12T12:59:10.866 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 175 2024-09-12T12:59:10.876 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:59:10.878 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:59:10.880 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:59:11.717 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:59:11.717 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T12:59:11.717 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:59:11.717 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:59:11.717 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T12:59:11.717 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T12:59:11.727 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:59:11.727 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T12:59:11.727 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:59:11.727 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:59:11.727 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T12:59:11.727 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T12:59:11.868 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 178 2024-09-12T12:59:11.877 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 179 2024-09-12T12:59:11.882 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:59:11.884 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:59:12.722 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 180 2024-09-12T12:59:12.723 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 181 2024-09-12T12:59:12.735 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:59:12.737 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:59:12.860 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:59:12.860 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T12:59:12.860 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:59:12.860 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:59:12.860 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T12:59:12.860 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T12:59:13.699 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 182 2024-09-12T12:59:13.706 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 183 2024-09-12T12:59:13.715 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:59:13.716 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:59:13.879 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:59:13.879 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T12:59:13.879 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:59:13.880 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:59:13.880 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T12:59:13.880 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T12:59:13.881 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:59:13.881 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T12:59:13.881 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:59:13.881 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:59:13.881 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T12:59:13.881 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T12:59:13.882 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:59:13.883 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T12:59:13.883 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:59:13.883 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:59:13.883 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T12:59:13.883 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T12:59:14.885 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:59:14.885 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T12:59:14.885 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:59:14.885 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:59:14.885 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T12:59:14.885 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T12:59:14.886 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:59:14.886 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T12:59:14.886 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:59:14.886 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:59:14.886 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T12:59:14.886 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T12:59:15.684 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 184 2024-09-12T12:59:15.697 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 187 2024-09-12T12:59:15.720 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:59:15.735 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:59:15.739 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:59:15.740 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T12:59:15.740 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:59:15.740 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:59:15.740 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T12:59:15.740 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T12:59:15.741 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:59:15.741 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T12:59:15.741 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:59:15.741 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:59:15.742 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T12:59:15.742 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T12:59:16.705 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 186 2024-09-12T12:59:16.748 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 185 2024-09-12T12:59:16.750 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:59:16.750 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T12:59:16.750 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:59:16.750 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:59:16.750 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T12:59:16.750 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T12:59:16.750 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:59:16.750 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T12:59:16.750 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:59:16.750 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:59:16.750 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T12:59:16.751 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T12:59:16.751 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:59:16.758 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:59:16.777 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T12:59:16.777+0000 7f67dd000640 -1 rbd::mirror::image_replayer::snapshot::Replayer: 0x56055655b400 handle_unlink_peer: failed to unlink local peer from remote image: (22) Invalid argument 2024-09-12T12:59:17.689 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 189 2024-09-12T12:59:17.695 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 188 2024-09-12T12:59:17.725 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:59:17.737 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:59:18.682 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 191 2024-09-12T12:59:18.688 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 190 2024-09-12T12:59:18.723 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:59:18.724 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T12:59:18.724 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:59:18.724 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:59:18.724 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T12:59:18.724 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T12:59:18.733 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:59:18.733 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:59:18.737 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:59:18.737 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T12:59:18.738 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:59:18.738 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:59:18.738 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T12:59:18.738 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T12:59:19.689 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 193 2024-09-12T12:59:19.698 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 192 2024-09-12T12:59:19.733 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:59:19.735 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:59:19.747 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:59:19.747 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T12:59:19.747 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:59:19.747 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:59:19.747 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T12:59:19.747 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T12:59:19.749 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 194 2024-09-12T12:59:19.756 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:59:19.761 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:59:19.762 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T12:59:19.762 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:59:19.762 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:59:19.762 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T12:59:19.762 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T12:59:20.719 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 195 2024-09-12T12:59:20.729 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:59:20.729 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T12:59:20.729 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:59:20.729 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:59:20.729 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T12:59:20.730 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T12:59:20.731 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 197 2024-09-12T12:59:20.732 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 196 2024-09-12T12:59:20.732 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:59:20.741 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:59:20.741 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T12:59:20.741 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:59:20.742 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:59:20.742 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T12:59:20.742 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T12:59:20.744 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:59:20.747 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:59:21.722 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 199 2024-09-12T12:59:21.729 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 198 2024-09-12T12:59:21.736 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:59:21.736 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:59:21.736 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T12:59:21.736 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T12:59:21.736 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:59:21.736 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:59:21.737 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:59:21.737 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T12:59:21.737 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:59:21.737 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T12:59:21.737 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T12:59:21.737 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T12:59:21.737 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:59:21.739 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:59:22.735 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:59:22.735 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T12:59:22.735 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:59:22.736 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:59:22.736 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T12:59:22.736 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T12:59:22.737 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:59:22.738 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T12:59:22.738 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:59:22.738 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:59:22.738 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T12:59:22.738 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T12:59:22.738 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 201 2024-09-12T12:59:22.742 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 200 2024-09-12T12:59:22.746 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:59:22.750 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:59:22.759 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:59:22.759 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T12:59:22.759 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:59:22.759 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:59:22.759 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T12:59:22.759 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T12:59:23.732 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 202 2024-09-12T12:59:23.733 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 203 2024-09-12T12:59:23.734 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:59:23.734 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T12:59:23.734 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:59:23.734 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:59:23.735 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T12:59:23.735 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T12:59:23.739 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:59:23.744 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:59:23.747 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:59:23.747 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T12:59:23.747 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:59:23.747 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:59:23.747 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T12:59:23.747 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T12:59:23.748 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:59:23.748 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T12:59:23.748 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:59:23.748 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:59:23.748 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T12:59:23.749 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T12:59:24.741 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:59:24.741 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T12:59:24.741 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:59:24.741 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:59:24.741 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T12:59:24.742 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T12:59:24.742 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:59:24.742 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T12:59:24.742 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:59:24.742 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:59:24.743 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T12:59:24.743 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T12:59:25.724 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 204 2024-09-12T12:59:25.749 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:59:25.750 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T12:59:25.750 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:59:25.750 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:59:25.750 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T12:59:25.750 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T12:59:25.753 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:59:25.753 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T12:59:25.753 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:59:25.753 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:59:25.753 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T12:59:25.753 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T12:59:25.773 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:59:26.741 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 207 2024-09-12T12:59:26.742 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:59:26.742 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T12:59:26.742 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:59:26.742 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:59:26.742 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T12:59:26.742 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T12:59:26.747 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:59:26.747 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T12:59:26.747 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:59:26.747 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:59:26.747 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T12:59:26.747 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T12:59:26.772 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 205 2024-09-12T12:59:26.807 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:59:26.812 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:59:27.675 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 208 2024-09-12T12:59:27.686 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 209 2024-09-12T12:59:27.732 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:59:27.733 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:59:27.774 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T12:59:27.774+0000 7f67dd000640 -1 rbd::mirror::image_replayer::snapshot::Replayer: 0x56055c2b3000 handle_unlink_peer: failed to unlink local peer from remote image: (22) Invalid argument 2024-09-12T12:59:28.719 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 211 2024-09-12T12:59:28.725 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 210 2024-09-12T12:59:28.734 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 206 2024-09-12T12:59:28.758 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:59:28.761 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:59:28.775 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:59:28.775 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T12:59:28.775 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:59:28.775 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:59:28.776 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T12:59:28.776 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T12:59:28.781 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:59:28.815 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T12:59:28.814+0000 7f67dd000640 -1 rbd::mirror::image_replayer::snapshot::Replayer: 0x5605566a8800 handle_unlink_peer: failed to unlink local peer from remote image: (22) Invalid argument 2024-09-12T12:59:29.738 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 212 2024-09-12T12:59:29.754 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 213 2024-09-12T12:59:29.764 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:59:29.772 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:59:29.785 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 214 2024-09-12T12:59:29.792 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:59:29.810 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:59:29.810 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T12:59:29.810 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:59:29.810 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:59:29.811 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T12:59:29.811 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T12:59:29.815 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:59:29.815 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T12:59:29.815 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:59:29.815 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:59:29.815 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T12:59:29.815 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T12:59:30.735 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:59:30.735 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T12:59:30.736 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:59:30.736 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:59:30.736 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T12:59:30.736 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T12:59:30.737 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:59:30.737 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T12:59:30.737 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:59:30.737 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:59:30.737 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T12:59:30.738 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T12:59:30.752 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 215 2024-09-12T12:59:30.757 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 216 2024-09-12T12:59:30.764 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:59:30.769 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:59:31.757 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 218 2024-09-12T12:59:31.761 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:59:31.761 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T12:59:31.761 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:59:31.761 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:59:31.761 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T12:59:31.761 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T12:59:31.763 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:59:31.763 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T12:59:31.763 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:59:31.763 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:59:31.764 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T12:59:31.764 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T12:59:31.767 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 217 2024-09-12T12:59:31.773 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:59:31.777 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:59:31.784 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:59:31.784 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T12:59:31.784 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:59:31.784 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:59:31.785 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T12:59:31.785 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T12:59:32.766 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 221 2024-09-12T12:59:32.767 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:59:32.767 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T12:59:32.767 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:59:32.767 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:59:32.767 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T12:59:32.768 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T12:59:32.773 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 220 2024-09-12T12:59:32.774 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:59:32.774 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T12:59:32.774 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:59:32.775 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:59:32.775 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T12:59:32.775 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T12:59:32.780 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 219 2024-09-12T12:59:32.782 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:59:32.782 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:59:32.790 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:59:32.794 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:59:32.795 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T12:59:32.795 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:59:32.795 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:59:32.795 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T12:59:32.795 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T12:59:33.767 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:59:33.767 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T12:59:33.767 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:59:33.768 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:59:33.768 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T12:59:33.768 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T12:59:33.772 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:59:33.772 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T12:59:33.772 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:59:33.772 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:59:33.772 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T12:59:33.772 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T12:59:33.773 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 223 2024-09-12T12:59:33.774 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 222 2024-09-12T12:59:33.787 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:59:33.788 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:59:34.776 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:59:34.776 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T12:59:34.776 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:59:34.776 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:59:34.776 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T12:59:34.777 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T12:59:34.779 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:59:34.780 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T12:59:34.780 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:59:34.780 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:59:34.780 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T12:59:34.780 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T12:59:35.764 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 224 2024-09-12T12:59:35.785 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:59:35.785 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:59:35.786 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T12:59:35.786 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T12:59:35.786 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:59:35.786 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:59:35.786 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T12:59:35.786 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:59:35.786 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:59:35.786 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T12:59:35.787 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T12:59:35.787 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T12:59:35.792 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:59:35.792 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T12:59:35.793 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:59:35.793 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:59:35.793 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T12:59:35.793 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T12:59:35.809 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:59:36.775 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 225 2024-09-12T12:59:36.790 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:59:36.790 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T12:59:36.790 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:59:36.790 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:59:36.790 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T12:59:36.790 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T12:59:36.791 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:59:36.791 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T12:59:36.791 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:59:36.791 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:59:36.791 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T12:59:36.791 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T12:59:36.796 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 226 2024-09-12T12:59:36.824 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:59:36.835 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T12:59:36.835+0000 7f67dd000640 -1 rbd::mirror::image_replayer::snapshot::Replayer: 0x56055c2f2800 handle_unlink_peer: failed to unlink local peer from remote image: (22) Invalid argument 2024-09-12T12:59:36.838 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:59:36.855 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 229 2024-09-12T12:59:36.868 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:59:37.686 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 228 2024-09-12T12:59:37.699 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 227 2024-09-12T12:59:37.725 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:59:37.727 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 231 2024-09-12T12:59:37.734 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:59:37.757 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 230 2024-09-12T12:59:37.769 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:59:37.777 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:59:37.779 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 232 2024-09-12T12:59:37.791 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:59:38.813 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:59:38.813 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T12:59:38.813 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:59:38.813 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:59:38.813 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T12:59:38.813 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T12:59:39.766 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 233 2024-09-12T12:59:39.781 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:59:39.799 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 234 2024-09-12T12:59:39.806 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:59:39.827 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:59:39.827 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T12:59:39.827 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:59:39.827 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:59:39.828 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T12:59:39.828 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T12:59:39.842 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:59:39.842 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T12:59:39.842 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:59:39.842 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:59:39.843 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T12:59:39.843 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T12:59:39.871 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:59:39.871 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T12:59:39.872 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:59:39.872 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:59:39.872 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T12:59:39.872 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T12:59:40.728 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:59:40.728 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T12:59:40.728 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:59:40.728 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:59:40.728 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T12:59:40.728 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T12:59:40.737 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:59:40.738 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T12:59:40.738 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:59:40.738 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:59:40.738 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T12:59:40.738 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T12:59:40.772 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:59:40.772 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T12:59:40.772 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:59:40.772 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:59:40.772 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T12:59:40.772 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T12:59:40.780 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:59:40.780 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T12:59:40.780 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:59:40.780 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:59:40.780 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T12:59:40.780 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T12:59:40.791 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 235 2024-09-12T12:59:40.793 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 236 2024-09-12T12:59:40.794 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:59:40.794 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T12:59:40.794 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:59:40.794 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:59:40.794 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T12:59:40.794 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T12:59:40.795 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 237 2024-09-12T12:59:40.809 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:59:40.810 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:59:40.821 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:59:41.811 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 239 2024-09-12T12:59:41.823 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 238 2024-09-12T12:59:41.825 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 242 2024-09-12T12:59:41.834 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:59:41.834 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:59:41.834 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:59:42.784 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:59:42.784 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T12:59:42.784 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:59:42.784 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:59:42.785 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T12:59:42.785 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T12:59:42.810 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:59:42.810 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T12:59:42.810 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:59:42.810 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:59:42.811 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T12:59:42.811 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T12:59:43.801 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 240 2024-09-12T12:59:43.805 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 241 2024-09-12T12:59:43.813 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:59:43.813 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T12:59:43.813 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:59:43.813 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:59:43.813 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T12:59:43.813 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T12:59:43.814 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:59:43.814 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T12:59:43.814 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:59:43.814 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:59:43.814 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T12:59:43.814 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T12:59:43.823 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:59:43.824 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T12:59:43.824 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:59:43.824 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:59:43.824 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T12:59:43.824 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T12:59:43.848 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:59:43.854 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:59:43.932 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 243 2024-09-12T12:59:43.939 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:59:44.834 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:59:44.834 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T12:59:44.834 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:59:44.834 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:59:44.834 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T12:59:44.835 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T12:59:44.836 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:59:44.836 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T12:59:44.836 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:59:44.836 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:59:44.836 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T12:59:44.836 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T12:59:44.836 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:59:44.836 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T12:59:44.836 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:59:44.836 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:59:44.836 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T12:59:44.837 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T12:59:45.801 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 244 2024-09-12T12:59:45.809 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 247 2024-09-12T12:59:45.832 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:59:45.851 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:59:45.875 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T12:59:45.875+0000 7f67dc600640 -1 rbd::mirror::image_replayer::snapshot::Replayer: 0x560556169800 handle_unlink_peer: failed to unlink local peer from remote image: (22) Invalid argument 2024-09-12T12:59:45.888 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 250 2024-09-12T12:59:45.907 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 248 2024-09-12T12:59:45.908 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:59:45.914 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:59:46.794 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 245 2024-09-12T12:59:46.801 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 246 2024-09-12T12:59:46.847 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:59:46.850 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:59:46.851 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:59:46.851 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T12:59:46.851 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:59:46.851 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:59:46.851 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T12:59:46.851 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T12:59:46.856 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:59:46.857 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T12:59:46.857 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:59:46.857 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:59:46.857 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T12:59:46.857 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T12:59:46.942 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:59:46.943 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T12:59:46.943 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:59:46.943 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:59:46.943 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T12:59:46.943 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T12:59:47.676 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 249 2024-09-12T12:59:47.716 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:59:47.730 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 252 2024-09-12T12:59:47.735 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 251 2024-09-12T12:59:47.748 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:59:47.751 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:59:48.696 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 253 2024-09-12T12:59:48.709 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:59:48.835 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:59:48.835 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T12:59:48.835 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:59:48.836 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:59:48.836 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T12:59:48.836 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T12:59:48.854 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:59:48.854 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T12:59:48.855 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:59:48.855 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:59:48.855 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T12:59:48.855 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T12:59:48.911 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:59:48.911 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T12:59:48.911 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:59:48.911 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:59:48.911 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T12:59:48.911 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T12:59:48.916 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:59:48.916 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T12:59:48.917 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:59:48.917 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:59:48.917 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T12:59:48.917 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T12:59:49.826 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 254 2024-09-12T12:59:49.843 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:59:49.850 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:59:49.850 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T12:59:49.850 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:59:49.850 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:59:49.850 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T12:59:49.850 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T12:59:49.853 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:59:49.853 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T12:59:49.853 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:59:49.853 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:59:49.853 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T12:59:49.853 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T12:59:50.720 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:59:50.720 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T12:59:50.720 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:59:50.720 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:59:50.720 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T12:59:50.720 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T12:59:50.753 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:59:50.753 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T12:59:50.753 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:59:50.753 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:59:50.753 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T12:59:50.753 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T12:59:50.754 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:59:50.755 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T12:59:50.755 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:59:50.755 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:59:50.755 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T12:59:50.755 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T12:59:50.898 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 255 2024-09-12T12:59:50.900 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 256 2024-09-12T12:59:50.914 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 257 2024-09-12T12:59:50.915 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:59:50.923 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 259 2024-09-12T12:59:50.923 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:59:50.926 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 258 2024-09-12T12:59:50.926 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:59:50.938 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:59:50.940 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:59:51.712 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:59:51.712 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T12:59:51.712 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:59:51.712 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:59:51.712 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T12:59:51.712 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T12:59:51.859 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 260 2024-09-12T12:59:51.871 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:59:52.718 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 262 2024-09-12T12:59:52.725 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 261 2024-09-12T12:59:52.733 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:59:52.736 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:59:52.847 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:59:52.848 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T12:59:52.848 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:59:52.848 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:59:52.848 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T12:59:52.848 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T12:59:53.919 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:59:53.919 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T12:59:53.920 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:59:53.920 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:59:53.920 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T12:59:53.920 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T12:59:53.927 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:59:53.927 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T12:59:53.927 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:59:53.927 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:59:53.927 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T12:59:53.927 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T12:59:53.930 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:59:53.930 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T12:59:53.930 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:59:53.931 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:59:53.931 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T12:59:53.931 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T12:59:53.941 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:59:53.941 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T12:59:53.941 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:59:53.941 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:59:53.941 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T12:59:53.941 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T12:59:53.943 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:59:53.943 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T12:59:53.943 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:59:53.943 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:59:53.943 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T12:59:53.943 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T12:59:54.705 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 263 2024-09-12T12:59:54.736 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:59:54.875 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:59:54.875 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T12:59:54.875 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:59:54.875 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:59:54.875 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T12:59:54.875 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T12:59:55.716 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 264 2024-09-12T12:59:55.737 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:59:55.737 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T12:59:55.737 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:59:55.737 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:59:55.737 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T12:59:55.737 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T12:59:55.738 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:59:55.738 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T12:59:55.738 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:59:55.738 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:59:55.738 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T12:59:55.738 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T12:59:55.782 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:59:56.728 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 269 2024-09-12T12:59:56.734 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 266 2024-09-12T12:59:56.788 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T12:59:56.788+0000 7f67dc600640 -1 rbd::mirror::image_replayer::snapshot::Replayer: 0x5605567f3400 handle_unlink_peer: failed to unlink local peer from remote image: (22) Invalid argument 2024-09-12T12:59:56.789 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:59:56.810 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:59:57.681 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 268 2024-09-12T12:59:57.696 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 270 2024-09-12T12:59:57.738 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:59:57.739 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T12:59:57.739 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:59:57.739 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:59:57.739 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T12:59:57.739 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T12:59:57.771 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:59:57.791 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:59:57.810 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T12:59:57.811+0000 7f67dc600640 -1 rbd::mirror::image_replayer::snapshot::Replayer: 0x56055690c000 handle_unlink_peer: failed to unlink local peer from remote image: (22) Invalid argument 2024-09-12T12:59:57.814 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T12:59:57.815+0000 7f67dd000640 -1 rbd::mirror::image_replayer::snapshot::Replayer: 0x5605562aa800 handle_unlink_peer: failed to unlink local peer from remote image: (22) Invalid argument 2024-09-12T12:59:58.691 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 272 2024-09-12T12:59:58.699 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 267 2024-09-12T12:59:58.702 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 271 2024-09-12T12:59:58.723 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 265 2024-09-12T12:59:58.741 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:59:58.752 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:59:58.761 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:59:58.766 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:59:58.784 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 273 2024-09-12T12:59:58.785 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:59:58.785 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T12:59:58.785 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:59:58.785 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:59:58.786 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T12:59:58.786 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T12:59:58.795 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:59:59.714 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 274 2024-09-12T12:59:59.726 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T12:59:59.792 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:59:59.792 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T12:59:59.792 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:59:59.792 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:59:59.793 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T12:59:59.793 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T12:59:59.813 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T12:59:59.813 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T12:59:59.813 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T12:59:59.813 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T12:59:59.813 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T12:59:59.813 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:00:00.726 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 276 2024-09-12T13:00:00.729 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 275 2024-09-12T13:00:00.737 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:00:00.737 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:00:00.774 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:00:00.774 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:00:00.774 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:00:00.774 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:00:00.774 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:00:00.774 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:00:00.794 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:00:00.794 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:00:00.794 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:00:00.794 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:00:00.795 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:00:00.795 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:00:01.739 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 277 2024-09-12T13:00:01.744 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:00:01.744 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:00:01.744 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:00:01.744 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:00:01.744 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:00:01.744 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:00:01.746 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 278 2024-09-12T13:00:01.756 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:00:01.756 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:00:01.756 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:00:01.756 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:00:01.757 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:00:01.757 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:00:01.757 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:00:01.762 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:00:01.763 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:00:01.764 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:00:01.764 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:00:01.764 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:00:01.764 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:00:01.764 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:00:01.768 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:00:01.768 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:00:01.768 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:00:01.768 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:00:01.768 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:00:01.769 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:00:01.799 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:00:01.799 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:00:01.800 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:00:01.800 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:00:01.800 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:00:01.800 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:00:02.728 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:00:02.729 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:00:02.729 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:00:02.729 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:00:02.729 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:00:02.729 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:00:02.751 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 281 2024-09-12T13:00:02.754 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 282 2024-09-12T13:00:02.763 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:00:02.765 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:00:03.740 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:00:03.740 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:00:03.740 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:00:03.740 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:00:03.741 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:00:03.741 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:00:03.741 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:00:03.741 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:00:03.741 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:00:03.741 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:00:03.741 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:00:03.741 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:00:03.769 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:00:03.770+0000 7f67dd000640 -1 rbd::mirror::image_replayer::snapshot::Replayer: 0x56055c2f9800 handle_unlink_peer: failed to unlink local peer from remote image: (22) Invalid argument 2024-09-12T13:00:03.782 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:00:03.783+0000 7f67dd000640 -1 rbd::mirror::image_replayer::snapshot::Replayer: 0x56055633ec00 handle_unlink_peer: failed to unlink local peer from remote image: (22) Invalid argument 2024-09-12T13:00:04.728 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 283 2024-09-12T13:00:04.738 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 279 2024-09-12T13:00:04.739 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 280 2024-09-12T13:00:04.759 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:00:04.759 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:00:04.759 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:00:04.759 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:00:04.759 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:00:04.759 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:00:04.765 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:00:04.765 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:00:04.765 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:00:04.765 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:00:04.765 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:00:04.765 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:00:04.767 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:00:04.786 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:00:04.787 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:00:05.721 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 284 2024-09-12T13:00:05.757 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:00:05.766 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:00:05.766 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:00:05.766 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:00:05.766 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:00:05.766 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:00:05.766 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:00:05.768 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:00:05.768 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:00:05.768 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:00:05.768 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:00:05.768 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:00:05.768 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:00:05.813 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 288 2024-09-12T13:00:05.826 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:00:06.726 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 286 2024-09-12T13:00:06.743 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 285 2024-09-12T13:00:06.769 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:00:06.783 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:00:06.796 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 290 2024-09-12T13:00:06.796 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 289 2024-09-12T13:00:06.810 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:00:06.811 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:00:07.675 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 287 2024-09-12T13:00:07.702 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:00:07.770 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:00:07.770 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:00:07.770 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:00:07.770 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:00:07.770 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:00:07.770 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:00:07.788 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:00:07.788 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:00:07.789 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:00:07.789 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:00:07.789 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:00:07.789 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:00:07.790 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:00:07.790 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:00:07.790 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:00:07.790 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:00:07.790 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:00:07.790 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:00:08.755 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 291 2024-09-12T13:00:08.756 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 292 2024-09-12T13:00:08.759 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 293 2024-09-12T13:00:08.759 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:00:08.759 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:00:08.759 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:00:08.760 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:00:08.760 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:00:08.760 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:00:08.768 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:00:08.771 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:00:08.774 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:00:08.829 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:00:08.829 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:00:08.829 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:00:08.829 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:00:08.829 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:00:08.829 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:00:09.750 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 295 2024-09-12T13:00:09.759 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 294 2024-09-12T13:00:09.770 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:00:09.771 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:00:09.771 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:00:09.772 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:00:09.772 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:00:09.772 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:00:09.772 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:00:09.772 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:00:09.785 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:00:09.785 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:00:09.785 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:00:09.785 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:00:09.786 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:00:09.786 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:00:09.813 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:00:09.813 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:00:09.814 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:00:09.814 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:00:09.814 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:00:09.814 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:00:09.814 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:00:09.814 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:00:09.814 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:00:09.814 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:00:09.814 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:00:09.814 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:00:10.705 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:00:10.706 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:00:10.706 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:00:10.706 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:00:10.706 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:00:10.706 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:00:10.766 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 296 2024-09-12T13:00:10.770 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 298 2024-09-12T13:00:10.773 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 297 2024-09-12T13:00:10.782 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 299 2024-09-12T13:00:10.786 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:00:10.787 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:00:10.793 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:00:10.793 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:00:11.763 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 300 2024-09-12T13:00:11.771 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:00:11.771 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:00:11.771 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:00:11.771 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:00:11.771 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:00:11.771 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:00:11.771 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:00:11.773 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:00:11.773 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:00:11.773 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:00:11.773 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:00:11.773 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:00:11.773 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:00:11.777 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:00:11.777 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:00:11.777 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:00:11.777 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:00:11.777 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:00:11.777 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:00:12.765 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 301 2024-09-12T13:00:12.767 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 302 2024-09-12T13:00:12.773 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:00:12.774 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:00:12.774 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:00:12.774 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:00:12.774 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:00:12.774 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:00:12.774 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:00:12.774 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:00:12.774 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:00:12.774 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:00:12.774 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:00:12.774 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:00:12.778 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:00:12.782 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:00:13.771 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:00:13.771+0000 7f67dc600640 -1 rbd::mirror::image_replayer::snapshot::Replayer: 0x56055c2fac00 handle_unlink_peer: failed to unlink local peer from remote image: (22) Invalid argument 2024-09-12T13:00:13.789 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 304 2024-09-12T13:00:13.790 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:00:13.790 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:00:13.790 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:00:13.790 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:00:13.790 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:00:13.790 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:00:13.790 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${SNAP_PIDS[@]} 2024-09-12T13:00:13.791 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 35630 2024-09-12T13:00:13.795 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:00:13.795 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:00:13.795 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:00:13.795 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:00:13.796 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:00:13.796 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:00:13.796 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${SNAP_PIDS[@]} 2024-09-12T13:00:13.796 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 35762 2024-09-12T13:00:13.800 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:00:14.765 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 303 2024-09-12T13:00:14.773 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:00:14.773 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:00:14.773 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:00:14.773 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:00:14.773 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:00:14.773 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:00:14.796 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:00:14.812 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 307 2024-09-12T13:00:14.825 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:00:15.752 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 305 2024-09-12T13:00:15.781 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:00:15.781 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:00:15.781 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:00:15.781 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:00:15.781 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:00:15.782 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:00:15.785 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:00:15.785 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${SNAP_PIDS[@]} 2024-09-12T13:00:15.785 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 35914 2024-09-12T13:00:16.750 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 306 2024-09-12T13:00:16.794 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:00:16.818 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 309 2024-09-12T13:00:16.825 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:00:17.682 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 308 2024-09-12T13:00:17.706 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:00:17.799 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:00:17.799 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:00:17.799 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:00:17.799 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:00:17.800 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:00:17.800 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:00:18.788 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:00:18.788 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:00:18.788 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:00:18.788 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:00:18.788 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:00:18.788 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:00:18.788 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 310 2024-09-12T13:00:18.805 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:00:19.786 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 311 2024-09-12T13:00:19.797 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${SNAP_PIDS[@]} 2024-09-12T13:00:19.797 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 36047 2024-09-12T13:00:19.798 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:00:19.828 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:00:19.828 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:00:19.828 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:00:19.828 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:00:19.828 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:00:19.829 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:00:20.709 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${SNAP_PIDS[@]} 2024-09-12T13:00:20.709 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 36199 2024-09-12T13:00:20.709 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${SNAP_PIDS[@]} 2024-09-12T13:00:20.710 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 36352 2024-09-12T13:00:20.710 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${SNAP_PIDS[@]} 2024-09-12T13:00:20.710 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 36525 2024-09-12T13:00:21.808 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${SNAP_PIDS[@]} 2024-09-12T13:00:21.809 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 36679 2024-09-12T13:00:22.780 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 312 2024-09-12T13:00:22.799 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:00:22.800 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${SNAP_PIDS[@]} 2024-09-12T13:00:22.800 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 36852 2024-09-12T13:00:25.802 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ (( RET != 0 )) 2024-09-12T13:00:25.802 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${WORKLOAD_PIDS[@]} 2024-09-12T13:00:25.803 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 35491 2024-09-12T13:02:50.018 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ ret=124 2024-09-12T13:02:50.019 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ (( ret != 124 )) 2024-09-12T13:02:50.019 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${WORKLOAD_PIDS[@]} 2024-09-12T13:02:50.019 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 35631 2024-09-12T13:02:51.538 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ ret=124 2024-09-12T13:02:51.538 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ (( ret != 124 )) 2024-09-12T13:02:51.538 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${WORKLOAD_PIDS[@]} 2024-09-12T13:02:51.539 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 35763 2024-09-12T13:02:52.514 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ ret=124 2024-09-12T13:02:52.515 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ (( ret != 124 )) 2024-09-12T13:02:52.515 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${WORKLOAD_PIDS[@]} 2024-09-12T13:02:52.515 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 35915 2024-09-12T13:02:53.515 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ ret=124 2024-09-12T13:02:53.515 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ (( ret != 124 )) 2024-09-12T13:02:53.515 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${WORKLOAD_PIDS[@]} 2024-09-12T13:02:53.515 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 36048 2024-09-12T13:02:54.509 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ ret=124 2024-09-12T13:02:54.510 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ (( ret != 124 )) 2024-09-12T13:02:54.510 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${WORKLOAD_PIDS[@]} 2024-09-12T13:02:54.510 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 36200 2024-09-12T13:02:55.501 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ ret=124 2024-09-12T13:02:55.501 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ (( ret != 124 )) 2024-09-12T13:02:55.502 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${WORKLOAD_PIDS[@]} 2024-09-12T13:02:55.502 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 36353 2024-09-12T13:02:56.538 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ ret=124 2024-09-12T13:02:56.538 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ (( ret != 124 )) 2024-09-12T13:02:56.538 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${WORKLOAD_PIDS[@]} 2024-09-12T13:02:56.538 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 36526 2024-09-12T13:02:57.540 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ ret=124 2024-09-12T13:02:57.540 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ (( ret != 124 )) 2024-09-12T13:02:57.540 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${WORKLOAD_PIDS[@]} 2024-09-12T13:02:57.540 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 36680 2024-09-12T13:02:58.557 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ ret=124 2024-09-12T13:02:58.557 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ (( ret != 124 )) 2024-09-12T13:02:58.557 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${WORKLOAD_PIDS[@]} 2024-09-12T13:02:58.557 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 36853 2024-09-12T13:02:59.783 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ ret=124 2024-09-12T13:02:59.783 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ (( ret != 124 )) 2024-09-12T13:02:59.784 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ (( RET != 0 )) 2024-09-12T13:02:59.784 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:02:59.784 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ COMPARE_PIDS+=($!) 2024-09-12T13:02:59.784 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:02:59.784 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ compare_demoted_promoted_image 1 2024-09-12T13:02:59.788 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local dev=/dev/rbd0 2024-09-12T13:02:59.788 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local img=image-primary1 2024-09-12T13:02:59.788 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mntpt=test-primary1 2024-09-12T13:02:59.788 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local demote_md5 promote_md5 2024-09-12T13:02:59.788 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo umount test-primary1 2024-09-12T13:02:59.788 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ COMPARE_PIDS+=($!) 2024-09-12T13:02:59.788 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:02:59.788 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ compare_demoted_promoted_image 2 2024-09-12T13:02:59.788 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local dev=/dev/rbd1 2024-09-12T13:02:59.788 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local img=image-primary2 2024-09-12T13:02:59.788 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mntpt=test-primary2 2024-09-12T13:02:59.789 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local demote_md5 promote_md5 2024-09-12T13:02:59.789 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo umount test-primary2 2024-09-12T13:02:59.789 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ COMPARE_PIDS+=($!) 2024-09-12T13:02:59.789 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:02:59.789 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ compare_demoted_promoted_image 3 2024-09-12T13:02:59.789 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local dev=/dev/rbd2 2024-09-12T13:02:59.789 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local img=image-primary3 2024-09-12T13:02:59.789 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mntpt=test-primary3 2024-09-12T13:02:59.789 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local demote_md5 promote_md5 2024-09-12T13:02:59.789 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo umount test-primary3 2024-09-12T13:02:59.789 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ COMPARE_PIDS+=($!) 2024-09-12T13:02:59.789 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:02:59.789 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ compare_demoted_promoted_image 4 2024-09-12T13:02:59.790 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local dev=/dev/rbd3 2024-09-12T13:02:59.790 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local img=image-primary4 2024-09-12T13:02:59.790 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mntpt=test-primary4 2024-09-12T13:02:59.790 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local demote_md5 promote_md5 2024-09-12T13:02:59.790 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo umount test-primary4 2024-09-12T13:02:59.790 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ COMPARE_PIDS+=($!) 2024-09-12T13:02:59.790 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:02:59.790 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ compare_demoted_promoted_image 5 2024-09-12T13:02:59.790 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local dev=/dev/rbd4 2024-09-12T13:02:59.790 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local img=image-primary5 2024-09-12T13:02:59.790 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mntpt=test-primary5 2024-09-12T13:02:59.790 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local demote_md5 promote_md5 2024-09-12T13:02:59.790 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo umount test-primary5 2024-09-12T13:02:59.791 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ COMPARE_PIDS+=($!) 2024-09-12T13:02:59.791 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:02:59.792 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ compare_demoted_promoted_image 6 2024-09-12T13:02:59.793 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ COMPARE_PIDS+=($!) 2024-09-12T13:02:59.793 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:02:59.793 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ compare_demoted_promoted_image 7 2024-09-12T13:02:59.793 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local dev=/dev/rbd5 2024-09-12T13:02:59.793 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local img=image-primary6 2024-09-12T13:02:59.793 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mntpt=test-primary6 2024-09-12T13:02:59.793 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local demote_md5 promote_md5 2024-09-12T13:02:59.793 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo umount test-primary6 2024-09-12T13:02:59.794 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local dev=/dev/rbd6 2024-09-12T13:02:59.794 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local img=image-primary7 2024-09-12T13:02:59.794 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mntpt=test-primary7 2024-09-12T13:02:59.794 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local demote_md5 promote_md5 2024-09-12T13:02:59.794 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo umount test-primary7 2024-09-12T13:02:59.794 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ COMPARE_PIDS+=($!) 2024-09-12T13:02:59.794 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:02:59.794 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ compare_demoted_promoted_image 8 2024-09-12T13:02:59.794 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ COMPARE_PIDS+=($!) 2024-09-12T13:02:59.794 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:02:59.795 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ COMPARE_PIDS+=($!) 2024-09-12T13:02:59.795 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local dev=/dev/rbd7 2024-09-12T13:02:59.795 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${COMPARE_PIDS[@]} 2024-09-12T13:02:59.795 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 42837 2024-09-12T13:02:59.796 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ compare_demoted_promoted_image 9 2024-09-12T13:02:59.796 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local dev=/dev/rbd8 2024-09-12T13:02:59.796 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local img=image-primary9 2024-09-12T13:02:59.796 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mntpt=test-primary9 2024-09-12T13:02:59.796 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local demote_md5 promote_md5 2024-09-12T13:02:59.796 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo umount test-primary9 2024-09-12T13:02:59.796 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ compare_demoted_promoted_image 10 2024-09-12T13:02:59.796 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local dev=/dev/rbd9 2024-09-12T13:02:59.796 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local img=image-primary10 2024-09-12T13:02:59.796 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mntpt=test-primary10 2024-09-12T13:02:59.796 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local demote_md5 promote_md5 2024-09-12T13:02:59.797 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo umount test-primary10 2024-09-12T13:02:59.797 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local img=image-primary8 2024-09-12T13:02:59.797 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mntpt=test-primary8 2024-09-12T13:02:59.797 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local demote_md5 promote_md5 2024-09-12T13:02:59.797 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo umount test-primary8 2024-09-12T13:03:00.433 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ awk '{print $1}' 2024-09-12T13:03:00.434 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo md5sum /dev/rbd4 2024-09-12T13:03:00.439 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo md5sum /dev/rbd5 2024-09-12T13:03:00.446 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ awk '{print $1}' 2024-09-12T13:03:00.530 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ awk '{print $1}' 2024-09-12T13:03:00.530 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo md5sum /dev/rbd7 2024-09-12T13:03:00.553 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo md5sum /dev/rbd6 2024-09-12T13:03:00.553 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ awk '{print $1}' 2024-09-12T13:03:00.567 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo md5sum /dev/rbd2 2024-09-12T13:03:00.568 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ awk '{print $1}' 2024-09-12T13:03:00.575 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo md5sum /dev/rbd1 2024-09-12T13:03:00.576 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ awk '{print $1}' 2024-09-12T13:03:00.658 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo md5sum /dev/rbd0 2024-09-12T13:03:00.659 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ awk '{print $1}' 2024-09-12T13:03:00.714 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo md5sum /dev/rbd3 2024-09-12T13:03:00.715 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ awk '{print $1}' 2024-09-12T13:03:00.775 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo md5sum /dev/rbd9 2024-09-12T13:03:00.775 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ awk '{print $1}' 2024-09-12T13:03:00.776 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ awk '{print $1}' 2024-09-12T13:03:00.776 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo md5sum /dev/rbd8 2024-09-12T13:03:56.687 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ demote_md5=8973418b9c4ac572630aa950c6432bf0 2024-09-12T13:03:56.687 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo rbd --cluster cluster1 device unmap -t krbd mirror/image-primary2 2024-09-12T13:03:57.329 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ demote_image cluster1 mirror image-primary2 2024-09-12T13:03:57.332 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:03:57.332 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:03:57.332 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:03:57.332 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 mirror image demote mirror/image-primary2 2024-09-12T13:04:00.946 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Image demoted to non-primary 2024-09-12T13:04:00.978 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary2 up+unknown 2024-09-12T13:04:00.978 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:04:00.978 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:04:00.978 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:04:00.978 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:04:00.978 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:04:00.978 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:04:00.978 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:04:00.978 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:04:02.028 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary2 up+unknown '' '' 2024-09-12T13:04:02.028 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:04:02.028 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:04:02.028 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:04:02.028 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:04:02.029 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:04:02.029 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:04:02.029 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:04:02.031 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:04:02.031 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary2 2024-09-12T13:04:02.251 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary2: 2024-09-12T13:04:02.252 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: b7ebe2f2-cab7-4e57-a03a-3091186a2999 2024-09-12T13:04:02.252 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:04:02.252 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:04:02.252 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:04:02.252 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:03:47' 2024-09-12T13:04:02.252 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:04:02.262 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:04:02.262 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:04:02.266 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 2 2024-09-12T13:04:03.061 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ demote_md5=556b8bb44ef8d2d6fce0910eee93534d 2024-09-12T13:04:03.061 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo rbd --cluster cluster1 device unmap -t krbd mirror/image-primary7 2024-09-12T13:04:03.380 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ demote_image cluster1 mirror image-primary7 2024-09-12T13:04:03.380 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:04:03.380 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:04:03.380 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:04:03.381 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 mirror image demote mirror/image-primary7 2024-09-12T13:04:03.477 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ demote_md5=1c46232c0659be87c589995110ffb9af 2024-09-12T13:04:03.477 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo rbd --cluster cluster1 device unmap -t krbd mirror/image-primary8 2024-09-12T13:04:03.783 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ demote_image cluster1 mirror image-primary8 2024-09-12T13:04:03.783 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:04:03.783 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:04:03.783 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:04:03.783 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 mirror image demote mirror/image-primary8 2024-09-12T13:04:04.276 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary2 up+unknown '' '' 2024-09-12T13:04:04.276 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:04:04.276 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:04:04.276 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:04:04.276 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:04:04.276 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:04:04.276 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:04:04.276 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:04:04.279 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:04:04.279 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary2 2024-09-12T13:04:04.538 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary2: 2024-09-12T13:04:04.538 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: b7ebe2f2-cab7-4e57-a03a-3091186a2999 2024-09-12T13:04:04.538 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:04:04.538 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:04:04.538 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:04:04.539 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:03:47' 2024-09-12T13:04:04.539 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:04:04.550 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:04:04.550 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:04:04.550 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 4 2024-09-12T13:04:04.626 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ demote_md5=6f74976b8ee4ca5d9633fddc227d3915 2024-09-12T13:04:04.626 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo rbd --cluster cluster1 device unmap -t krbd mirror/image-primary9 2024-09-12T13:04:04.944 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ demote_image cluster1 mirror image-primary9 2024-09-12T13:04:05.013 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:04:05.013 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:04:05.013 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:04:05.013 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 mirror image demote mirror/image-primary9 2024-09-12T13:04:05.490 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ demote_md5=ed95043a70577e18d5c8cd422e6d8d86 2024-09-12T13:04:05.490 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo rbd --cluster cluster1 device unmap -t krbd mirror/image-primary1 2024-09-12T13:04:05.643 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ demote_image cluster1 mirror image-primary1 2024-09-12T13:04:05.644 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:04:05.644 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:04:05.645 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:04:05.645 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 mirror image demote mirror/image-primary1 2024-09-12T13:04:05.673 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ demote_md5=02c7b7be3bb22b94ac7599660ca16db7 2024-09-12T13:04:05.673 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo rbd --cluster cluster1 device unmap -t krbd mirror/image-primary3 2024-09-12T13:04:05.699 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ demote_md5=60fdbc55a27e7b769003b837ab23f416 2024-09-12T13:04:05.699 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo rbd --cluster cluster1 device unmap -t krbd mirror/image-primary4 2024-09-12T13:04:05.854 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ demote_md5=5fa22ff55b5cbbd7ac3ddea465d7059c 2024-09-12T13:04:05.854 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo rbd --cluster cluster1 device unmap -t krbd mirror/image-primary10 2024-09-12T13:04:05.898 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ demote_image cluster1 mirror image-primary3 2024-09-12T13:04:05.898 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:04:05.898 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:04:05.898 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:04:05.898 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 mirror image demote mirror/image-primary3 2024-09-12T13:04:05.904 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ demote_image cluster1 mirror image-primary4 2024-09-12T13:04:05.904 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:04:05.904 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:04:05.905 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:04:05.905 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 mirror image demote mirror/image-primary4 2024-09-12T13:04:05.990 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Image demoted to non-primary 2024-09-12T13:04:06.017 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ demote_md5=7069169804ded02d8cc7feac58e1c5fd 2024-09-12T13:04:06.018 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo rbd --cluster cluster1 device unmap -t krbd mirror/image-primary5 2024-09-12T13:04:06.021 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary8 up+unknown 2024-09-12T13:04:06.022 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:04:06.023 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:04:06.023 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:04:06.023 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:04:06.023 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:04:06.023 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:04:06.023 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:04:06.023 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:04:06.036 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ demote_image cluster1 mirror image-primary10 2024-09-12T13:04:06.036 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:04:06.036 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:04:06.036 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:04:06.037 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 mirror image demote mirror/image-primary10 2024-09-12T13:04:06.180 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ demote_image cluster1 mirror image-primary5 2024-09-12T13:04:06.180 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:04:06.180 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:04:06.181 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:04:06.181 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 mirror image demote mirror/image-primary5 2024-09-12T13:04:06.334 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ demote_md5=b91f8d6f5e9db3c178acac32a0fbaddd 2024-09-12T13:04:06.334 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo rbd --cluster cluster1 device unmap -t krbd mirror/image-primary6 2024-09-12T13:04:06.424 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ demote_image cluster1 mirror image-primary6 2024-09-12T13:04:06.424 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:04:06.424 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:04:06.424 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:04:06.424 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 mirror image demote mirror/image-primary6 2024-09-12T13:04:06.984 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Image demoted to non-primary 2024-09-12T13:04:06.986 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Image demoted to non-primary 2024-09-12T13:04:07.002 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary1 up+unknown 2024-09-12T13:04:07.002 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:04:07.002 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:04:07.002 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:04:07.003 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:04:07.003 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:04:07.003 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:04:07.003 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:04:07.003 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:04:07.007 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary9 up+unknown 2024-09-12T13:04:07.007 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:04:07.008 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:04:07.008 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:04:07.008 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:04:07.008 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:04:07.008 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:04:07.008 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:04:07.008 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:04:07.020 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Image demoted to non-primary 2024-09-12T13:04:07.029 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary8 up+unknown '' '' 2024-09-12T13:04:07.029 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:04:07.029 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:04:07.029 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:04:07.030 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:04:07.030 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:04:07.030 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:04:07.030 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:04:07.030 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:04:07.030 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary8 2024-09-12T13:04:07.038 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary4 up+unknown 2024-09-12T13:04:07.038 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:04:07.038 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:04:07.039 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:04:07.039 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:04:07.039 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:04:07.039 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:04:07.039 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:04:07.039 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:04:07.046 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Image demoted to non-primary 2024-09-12T13:04:07.060 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary5 up+unknown 2024-09-12T13:04:07.060 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:04:07.060 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:04:07.060 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:04:07.060 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:04:07.060 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:04:07.061 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:04:07.061 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:04:07.061 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:04:07.122 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary8: 2024-09-12T13:04:07.122 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: e5807591-ea6d-4541-b1c3-0785c5e797ba 2024-09-12T13:04:07.122 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:04:07.122 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:04:07.122 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:04:07.122 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:03:47' 2024-09-12T13:04:07.122 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:04:07.125 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:04:07.126 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:04:07.126 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 2 2024-09-12T13:04:07.768 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Image demoted to non-primary 2024-09-12T13:04:07.813 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary7 up+unknown 2024-09-12T13:04:07.819 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:04:07.819 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:04:07.819 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:04:07.819 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:04:07.819 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:04:07.819 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:04:07.819 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:04:07.819 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:04:08.011 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary9 up+unknown '' '' 2024-09-12T13:04:08.011 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:04:08.011 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:04:08.011 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:04:08.011 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:04:08.011 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:04:08.012 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:04:08.012 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:04:08.012 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary1 up+unknown '' '' 2024-09-12T13:04:08.012 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:04:08.012 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:04:08.012 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:04:08.012 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:04:08.012 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:04:08.012 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:04:08.012 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:04:08.014 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:04:08.014 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary9 2024-09-12T13:04:08.014 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:04:08.014 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary1 2024-09-12T13:04:08.042 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary4 up+unknown '' '' 2024-09-12T13:04:08.042 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:04:08.042 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:04:08.042 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:04:08.042 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:04:08.042 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:04:08.042 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:04:08.042 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:04:08.043 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:04:08.043 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary4 2024-09-12T13:04:08.071 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary5 up+unknown '' '' 2024-09-12T13:04:08.072 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:04:08.072 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:04:08.072 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:04:08.072 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:04:08.072 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:04:08.072 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:04:08.072 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:04:08.076 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:04:08.076 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary5 2024-09-12T13:04:08.239 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary1: 2024-09-12T13:04:08.239 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 5a3f10b5-2891-464a-a2f2-921068cdf6e7 2024-09-12T13:04:08.239 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:04:08.240 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:04:08.240 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:04:08.240 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:03:47' 2024-09-12T13:04:08.240 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:04:08.269 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:04:08.269 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:04:08.269 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 2 2024-09-12T13:04:08.308 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary9: 2024-09-12T13:04:08.309 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 7d07d66e-fe3f-46b2-83dc-67fa35b43f0e 2024-09-12T13:04:08.309 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:04:08.309 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:04:08.309 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:04:08.309 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:03:47' 2024-09-12T13:04:08.309 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:04:08.321 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:04:08.321 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:04:08.321 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 2 2024-09-12T13:04:08.355 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary5: 2024-09-12T13:04:08.355 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 8b95d658-5f20-45ab-b663-60abf1f860db 2024-09-12T13:04:08.355 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:04:08.355 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:04:08.355 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:04:08.357 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:03:47' 2024-09-12T13:04:08.358 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:04:08.363 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:04:08.363 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:04:08.363 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 2 2024-09-12T13:04:08.411 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary4: 2024-09-12T13:04:08.411 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: f5942d86-af5f-4ca1-8ebc-da4cbe6dc314 2024-09-12T13:04:08.411 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:04:08.412 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:04:08.412 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:04:08.412 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:03:47' 2024-09-12T13:04:08.412 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:04:08.415 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:04:08.415 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:04:08.415 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 2 2024-09-12T13:04:08.561 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary2 up+unknown '' '' 2024-09-12T13:04:08.561 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:04:08.561 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:04:08.561 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:04:08.561 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:04:08.561 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:04:08.561 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:04:08.561 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:04:08.562 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:04:08.562 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary2 2024-09-12T13:04:08.651 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary2: 2024-09-12T13:04:08.651 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: b7ebe2f2-cab7-4e57-a03a-3091186a2999 2024-09-12T13:04:08.651 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:04:08.651 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:04:08.651 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:04:08.651 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:03:47' 2024-09-12T13:04:08.652 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:04:08.654 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:04:08.654 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:04:08.654 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 8 2024-09-12T13:04:08.749 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Image demoted to non-primary 2024-09-12T13:04:08.756 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Image demoted to non-primary 2024-09-12T13:04:08.770 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary3 up+unknown 2024-09-12T13:04:08.771 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:04:08.771 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:04:08.771 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:04:08.771 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:04:08.771 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:04:08.771 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:04:08.772 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:04:08.772 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:04:08.775 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary10 up+unknown 2024-09-12T13:04:08.775 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:04:08.775 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:04:08.775 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:04:08.775 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:04:08.776 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:04:08.776 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:04:08.776 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:04:08.776 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:04:08.822 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary7 up+unknown '' '' 2024-09-12T13:04:08.822 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:04:08.823 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:04:08.823 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:04:08.823 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:04:08.823 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:04:08.823 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:04:08.823 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:04:08.823 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:04:08.823 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary7 2024-09-12T13:04:08.905 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary7: 2024-09-12T13:04:08.905 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 59cd13a4-5c88-4a91-a100-f622774b37f9 2024-09-12T13:04:08.905 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:04:08.905 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:04:08.905 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:04:08.905 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:03:47' 2024-09-12T13:04:08.906 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:04:08.908 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:04:08.908 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:04:08.908 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 2 2024-09-12T13:04:09.128 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary8 up+unknown '' '' 2024-09-12T13:04:09.129 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:04:09.129 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:04:09.129 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:04:09.129 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:04:09.129 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:04:09.129 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:04:09.129 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:04:09.130 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:04:09.130 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary8 2024-09-12T13:04:09.198 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary8: 2024-09-12T13:04:09.198 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: e5807591-ea6d-4541-b1c3-0785c5e797ba 2024-09-12T13:04:09.198 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:04:09.198 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:04:09.198 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:04:09.198 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:03:47' 2024-09-12T13:04:09.199 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:04:09.201 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:04:09.201 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:04:09.201 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 4 2024-09-12T13:04:09.774 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary3 up+unknown '' '' 2024-09-12T13:04:09.774 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:04:09.775 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:04:09.775 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:04:09.775 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:04:09.775 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:04:09.775 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:04:09.775 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:04:09.775 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:04:09.775 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary3 2024-09-12T13:04:09.779 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary10 up+unknown '' '' 2024-09-12T13:04:09.779 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:04:09.779 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:04:09.779 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:04:09.780 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:04:09.780 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:04:09.780 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:04:09.780 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:04:09.780 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:04:09.780 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary10 2024-09-12T13:04:09.848 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary3: 2024-09-12T13:04:09.848 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: c469479f-c19b-4534-a75c-ca68f8c28929 2024-09-12T13:04:09.848 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:04:09.848 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:04:09.848 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:04:09.848 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:03:47' 2024-09-12T13:04:09.848 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:04:09.850 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary10: 2024-09-12T13:04:09.850 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: e8086552-9fee-4a93-b94f-d2d6dc6bff63 2024-09-12T13:04:09.850 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:04:09.850 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:04:09.850 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:04:09.850 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:03:47' 2024-09-12T13:04:09.850 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:04:09.851 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:04:09.851 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:04:09.851 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 2 2024-09-12T13:04:09.853 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:04:09.853 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:04:09.853 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 2 2024-09-12T13:04:10.278 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary1 up+unknown '' '' 2024-09-12T13:04:10.279 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:04:10.279 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:04:10.279 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:04:10.279 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:04:10.279 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:04:10.279 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:04:10.279 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:04:10.279 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:04:10.280 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary1 2024-09-12T13:04:10.324 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary9 up+unknown '' '' 2024-09-12T13:04:10.325 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:04:10.325 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:04:10.325 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:04:10.325 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:04:10.325 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:04:10.325 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:04:10.326 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:04:10.326 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:04:10.326 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary9 2024-09-12T13:04:10.354 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary1: 2024-09-12T13:04:10.355 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 5a3f10b5-2891-464a-a2f2-921068cdf6e7 2024-09-12T13:04:10.355 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:04:10.355 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:04:10.355 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:04:10.355 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:03:47' 2024-09-12T13:04:10.355 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:04:10.357 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:04:10.357 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:04:10.357 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 4 2024-09-12T13:04:10.368 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary5 up+unknown '' '' 2024-09-12T13:04:10.369 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:04:10.369 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:04:10.369 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:04:10.369 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:04:10.370 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:04:10.370 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:04:10.370 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:04:10.370 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:04:10.370 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary5 2024-09-12T13:04:10.396 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary9: 2024-09-12T13:04:10.396 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 7d07d66e-fe3f-46b2-83dc-67fa35b43f0e 2024-09-12T13:04:10.396 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:04:10.396 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:04:10.397 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:04:10.397 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:03:47' 2024-09-12T13:04:10.397 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:04:10.398 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:04:10.399 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:04:10.399 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 4 2024-09-12T13:04:10.419 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary4 up+unknown '' '' 2024-09-12T13:04:10.419 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:04:10.419 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:04:10.419 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:04:10.419 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:04:10.419 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:04:10.419 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:04:10.420 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:04:10.420 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:04:10.420 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary4 2024-09-12T13:04:10.438 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary5: 2024-09-12T13:04:10.438 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 8b95d658-5f20-45ab-b663-60abf1f860db 2024-09-12T13:04:10.439 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:04:10.439 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:04:10.439 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:04:10.439 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:03:47' 2024-09-12T13:04:10.439 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:04:10.441 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:04:10.441 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:04:10.441 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 4 2024-09-12T13:04:10.491 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary4: 2024-09-12T13:04:10.492 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: f5942d86-af5f-4ca1-8ebc-da4cbe6dc314 2024-09-12T13:04:10.492 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:04:10.492 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:04:10.492 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:04:10.492 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:03:47' 2024-09-12T13:04:10.492 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:04:10.494 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:04:10.494 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:04:10.494 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 4 2024-09-12T13:04:10.743 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Image demoted to non-primary 2024-09-12T13:04:10.753 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary6 up+unknown 2024-09-12T13:04:10.753 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:04:10.753 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:04:10.753 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:04:10.753 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:04:10.753 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:04:10.753 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:04:10.754 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:04:10.754 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:04:10.911 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary7 up+unknown '' '' 2024-09-12T13:04:10.911 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:04:10.911 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:04:10.911 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:04:10.911 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:04:10.911 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:04:10.911 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:04:10.911 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:04:10.911 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:04:10.912 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary7 2024-09-12T13:04:10.987 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary7: 2024-09-12T13:04:10.987 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 59cd13a4-5c88-4a91-a100-f622774b37f9 2024-09-12T13:04:10.987 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:04:10.987 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:04:10.987 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:04:10.987 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:03:47' 2024-09-12T13:04:10.987 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:04:10.992 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:04:10.992 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:04:10.992 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 4 2024-09-12T13:04:11.756 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary6 up+unknown '' '' 2024-09-12T13:04:11.756 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:04:11.756 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:04:11.756 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:04:11.756 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:04:11.756 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:04:11.756 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:04:11.756 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:04:11.757 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:04:11.757 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary6 2024-09-12T13:04:11.830 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary6: 2024-09-12T13:04:11.830 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: a790a319-cd27-4a01-b342-b61db0aa2b94 2024-09-12T13:04:11.830 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:04:11.830 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:04:11.830 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:04:11.830 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:03:47' 2024-09-12T13:04:11.830 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:04:11.832 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:04:11.833 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:04:11.833 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 2 2024-09-12T13:04:11.853 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary3 up+unknown '' '' 2024-09-12T13:04:11.853 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:04:11.854 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:04:11.854 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:04:11.854 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:04:11.854 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:04:11.854 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:04:11.854 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:04:11.854 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:04:11.854 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary3 2024-09-12T13:04:11.855 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary10 up+unknown '' '' 2024-09-12T13:04:11.855 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:04:11.855 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:04:11.855 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:04:11.855 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:04:11.855 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:04:11.856 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:04:11.856 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:04:11.856 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:04:11.856 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary10 2024-09-12T13:04:11.928 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary3: 2024-09-12T13:04:11.928 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: c469479f-c19b-4534-a75c-ca68f8c28929 2024-09-12T13:04:11.928 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:04:11.928 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:04:11.928 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:04:11.928 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:03:47' 2024-09-12T13:04:11.928 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:04:11.929 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary10: 2024-09-12T13:04:11.929 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: e8086552-9fee-4a93-b94f-d2d6dc6bff63 2024-09-12T13:04:11.929 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:04:11.929 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:04:11.929 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:04:11.929 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:03:47' 2024-09-12T13:04:11.929 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:04:11.930 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:04:11.930 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:04:11.930 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 4 2024-09-12T13:04:11.931 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:04:11.931 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:04:11.931 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 4 2024-09-12T13:04:13.203 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary8 up+unknown '' '' 2024-09-12T13:04:13.203 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:04:13.204 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:04:13.204 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:04:13.204 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:04:13.204 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:04:13.204 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:04:13.204 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:04:13.204 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:04:13.204 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary8 2024-09-12T13:04:13.292 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary8: 2024-09-12T13:04:13.292 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: e5807591-ea6d-4541-b1c3-0785c5e797ba 2024-09-12T13:04:13.292 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:04:13.292 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:04:13.292 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:04:13.292 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:03:47' 2024-09-12T13:04:13.292 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:04:13.294 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:04:13.295 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:04:13.295 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 8 2024-09-12T13:04:13.835 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary6 up+unknown '' '' 2024-09-12T13:04:13.835 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:04:13.835 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:04:13.835 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:04:13.835 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:04:13.835 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:04:13.836 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:04:13.836 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:04:13.836 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:04:13.836 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary6 2024-09-12T13:04:13.915 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary6: 2024-09-12T13:04:13.916 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: a790a319-cd27-4a01-b342-b61db0aa2b94 2024-09-12T13:04:13.916 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:04:13.916 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:04:13.916 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:04:13.916 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:03:47' 2024-09-12T13:04:13.916 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:04:13.918 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:04:13.918 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:04:13.919 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 4 2024-09-12T13:04:14.360 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary1 up+unknown '' '' 2024-09-12T13:04:14.360 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:04:14.360 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:04:14.360 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:04:14.360 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:04:14.361 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:04:14.361 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:04:14.361 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:04:14.361 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:04:14.361 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary1 2024-09-12T13:04:14.401 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary9 up+unknown '' '' 2024-09-12T13:04:14.401 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:04:14.401 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:04:14.402 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:04:14.402 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:04:14.402 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:04:14.402 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:04:14.402 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:04:14.402 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:04:14.402 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary9 2024-09-12T13:04:14.440 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary1: 2024-09-12T13:04:14.440 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 5a3f10b5-2891-464a-a2f2-921068cdf6e7 2024-09-12T13:04:14.440 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:04:14.440 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:04:14.440 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:04:14.441 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:03:47' 2024-09-12T13:04:14.441 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:04:14.443 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:04:14.443 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:04:14.443 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 8 2024-09-12T13:04:14.443 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary5 up+unknown '' '' 2024-09-12T13:04:14.444 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:04:14.444 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:04:14.444 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:04:14.444 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:04:14.444 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:04:14.444 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:04:14.444 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:04:14.444 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:04:14.444 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary5 2024-09-12T13:04:14.483 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary9: 2024-09-12T13:04:14.483 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 7d07d66e-fe3f-46b2-83dc-67fa35b43f0e 2024-09-12T13:04:14.483 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:04:14.483 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:04:14.483 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:04:14.483 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:03:47' 2024-09-12T13:04:14.483 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:04:14.485 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:04:14.485 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:04:14.485 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 8 2024-09-12T13:04:14.497 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary4 up+unknown '' '' 2024-09-12T13:04:14.497 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:04:14.497 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:04:14.497 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:04:14.497 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:04:14.497 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:04:14.497 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:04:14.497 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:04:14.498 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:04:14.498 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary4 2024-09-12T13:04:14.521 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary5: 2024-09-12T13:04:14.521 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 8b95d658-5f20-45ab-b663-60abf1f860db 2024-09-12T13:04:14.521 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:04:14.521 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:04:14.522 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:04:14.522 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:03:47' 2024-09-12T13:04:14.522 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:04:14.524 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:04:14.524 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:04:14.525 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 8 2024-09-12T13:04:14.570 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary4: 2024-09-12T13:04:14.570 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: f5942d86-af5f-4ca1-8ebc-da4cbe6dc314 2024-09-12T13:04:14.571 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:04:14.571 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:04:14.571 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:04:14.571 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:03:47' 2024-09-12T13:04:14.571 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:04:14.573 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:04:14.573 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:04:14.573 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 8 2024-09-12T13:04:14.992 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary7 up+unknown '' '' 2024-09-12T13:04:14.992 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:04:14.993 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:04:14.993 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:04:14.993 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:04:14.993 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:04:14.993 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:04:14.993 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:04:14.993 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:04:14.993 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary7 2024-09-12T13:04:15.066 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary7: 2024-09-12T13:04:15.067 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 59cd13a4-5c88-4a91-a100-f622774b37f9 2024-09-12T13:04:15.067 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:04:15.067 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:04:15.067 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:04:15.067 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:03:47' 2024-09-12T13:04:15.067 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:04:15.069 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:04:15.069 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:04:15.069 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 8 2024-09-12T13:04:15.933 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary3 up+unknown '' '' 2024-09-12T13:04:15.933 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:04:15.933 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:04:15.934 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:04:15.934 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:04:15.934 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:04:15.934 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:04:15.934 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:04:15.935 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary10 up+unknown '' '' 2024-09-12T13:04:15.935 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:04:15.935 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:04:15.935 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:04:15.935 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:04:15.935 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:04:15.935 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:04:15.935 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:04:15.935 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:04:15.935 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary3 2024-09-12T13:04:15.936 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:04:15.936 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary10 2024-09-12T13:04:16.008 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary3: 2024-09-12T13:04:16.008 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: c469479f-c19b-4534-a75c-ca68f8c28929 2024-09-12T13:04:16.008 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:04:16.008 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:04:16.009 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:04:16.009 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:03:47' 2024-09-12T13:04:16.009 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:04:16.009 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary10: 2024-09-12T13:04:16.009 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: e8086552-9fee-4a93-b94f-d2d6dc6bff63 2024-09-12T13:04:16.010 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:04:16.010 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:04:16.010 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:04:16.010 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:03:47' 2024-09-12T13:04:16.010 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:04:16.011 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:04:16.011 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:04:16.011 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 8 2024-09-12T13:04:16.011 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:04:16.011 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:04:16.012 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 8 2024-09-12T13:04:16.656 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary2 up+unknown '' '' 2024-09-12T13:04:16.657 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:04:16.657 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:04:16.657 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:04:16.657 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:04:16.657 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:04:16.657 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:04:16.657 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:04:16.657 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:04:16.657 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary2 2024-09-12T13:04:16.738 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary2: 2024-09-12T13:04:16.738 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: b7ebe2f2-cab7-4e57-a03a-3091186a2999 2024-09-12T13:04:16.738 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:04:16.738 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:04:16.738 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:04:16.738 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:03:47' 2024-09-12T13:04:16.738 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:04:16.741 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:04:16.741 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:04:16.741 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 8 2024-09-12T13:04:17.921 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary6 up+unknown '' '' 2024-09-12T13:04:17.921 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:04:17.921 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:04:17.922 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:04:17.922 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:04:17.922 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:04:17.922 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:04:17.922 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:04:17.922 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:04:17.922 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary6 2024-09-12T13:04:17.999 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary6: 2024-09-12T13:04:17.999 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: a790a319-cd27-4a01-b342-b61db0aa2b94 2024-09-12T13:04:17.999 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+unknown 2024-09-12T13:04:17.999 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: remote image is not primary 2024-09-12T13:04:17.999 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:04:17.999 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:04:17' 2024-09-12T13:04:18.000 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:04:18.001 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+unknown 2024-09-12T13:04:18.002 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:04:18.004 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: remote image is not primary 2024-09-12T13:04:18.005 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:04:18.005 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:04:18.005 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:04:18.007 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:04:18.008 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:04:18.010 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:04:18.011 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:04:18.011 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:04:18.015 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:04:17' 2024-09-12T13:04:18.015 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary6 up+unknown '2024-09-12 13:04:17' 2024-09-12T13:04:18.016 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:04:18.016 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:04:18.016 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:04:18.016 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:04:18.016 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:04:17' 2024-09-12T13:04:18.016 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:04:18.016 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:04:18.016 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-09-12T13:04:18.117 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041934232mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary15a3f10b5-2891-464a-a2f2-921068cdf6e7up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:04:17image-primary10e8086552-9fee-4a93-b94f-d2d6dc6bff63up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:04:17image-primary2b7ebe2f2-cab7-4e57-a03a-3091186a2999up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:04:17image-primary3c469479f-c19b-4534-a75c-ca68f8c28929up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:04:17image-primary4f5942d86-af5f-4ca1-8ebc-da4cbe6dc314up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:04:17image-primary58b95d658-5f20-45ab-b663-60abf1f860dbup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:04:17image-primary6a790a319-cd27-4a01-b342-b61db0aa2b94up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:04:17image-primary759cd13a4-5c88-4a91-a100-f622774b37f9up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:04:17image-primary8e5807591-ea6d-4541-b1c3-0785c5e797baup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:04:17image-primary97d07d66e-fe3f-46b2-83dc-67fa35b43f0eup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:04:17' 2024-09-12T13:04:18.117 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:04:18.118 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary6'\'']/last_update' 2024-09-12T13:04:18.166 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:04:17' 2024-09-12T13:04:18.167 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary6'\'']/state' 2024-09-12T13:04:18.172 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+unknown 2024-09-12T13:04:18.172 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:04:18.172 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+unknown 2024-09-12T13:04:18.173 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:04:18.174 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:04:18.174 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:04:18.174 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster2 mirror image-primary6 up+unknown 2024-09-12T13:04:18.174 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:04:18.175 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:04:18.175 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:04:18.175 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:04:18.175 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:04:18.175 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:04:18.175 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:04:18.175 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:04:19.177 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster2 mirror image-primary6 up+unknown '' '' 2024-09-12T13:04:19.177 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:04:19.177 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:04:19.177 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:04:19.177 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:04:19.177 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:04:19.177 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:04:19.177 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:04:19.178 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:04:19.178 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster2 mirror image status mirror/image-primary6 2024-09-12T13:04:19.245 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary6: 2024-09-12T13:04:19.245 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: a790a319-cd27-4a01-b342-b61db0aa2b94 2024-09-12T13:04:19.246 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+unknown 2024-09-12T13:04:19.246 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: remote image is not primary 2024-09-12T13:04:19.246 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:04:19.246 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:04:17' 2024-09-12T13:04:19.246 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:04:19.248 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+unknown 2024-09-12T13:04:19.248 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:04:19.250 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: remote image is not primary 2024-09-12T13:04:19.251 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:04:19.251 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:04:19.252 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:04:19.254 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:04:19.254 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:04:19.257 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:04:19.257 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:04:19.258 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:04:19.262 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:04:17' 2024-09-12T13:04:19.262 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster2 mirror image-primary6 up+unknown '2024-09-12 13:04:17' 2024-09-12T13:04:19.262 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:04:19.262 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:04:19.262 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:04:19.262 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:04:19.262 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:04:17' 2024-09-12T13:04:19.262 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:04:19.263 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:04:19.263 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2024-09-12T13:04:19.367 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041924231mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary15a3f10b5-2891-464a-a2f2-921068cdf6e7up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:04:17image-primary10e8086552-9fee-4a93-b94f-d2d6dc6bff63up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:04:17image-primary2b7ebe2f2-cab7-4e57-a03a-3091186a2999up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:04:17image-primary3c469479f-c19b-4534-a75c-ca68f8c28929up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:04:17image-primary4f5942d86-af5f-4ca1-8ebc-da4cbe6dc314up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:04:17image-primary58b95d658-5f20-45ab-b663-60abf1f860dbup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:04:17image-primary6a790a319-cd27-4a01-b342-b61db0aa2b94up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:04:17image-primary759cd13a4-5c88-4a91-a100-f622774b37f9up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:04:17image-primary8e5807591-ea6d-4541-b1c3-0785c5e797baup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:04:17image-primary97d07d66e-fe3f-46b2-83dc-67fa35b43f0eup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:04:17' 2024-09-12T13:04:19.367 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:04:19.367 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary6'\'']/last_update' 2024-09-12T13:04:19.371 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:04:17' 2024-09-12T13:04:19.372 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary6'\'']/state' 2024-09-12T13:04:19.376 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+unknown 2024-09-12T13:04:19.376 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:04:19.377 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+unknown 2024-09-12T13:04:19.379 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:04:19.379 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:04:19.380 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:04:19.380 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ promote_image cluster2 mirror image-primary6 2024-09-12T13:04:19.380 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:04:19.380 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:04:19.380 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:04:19.380 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local force= 2024-09-12T13:04:19.380 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 mirror image promote mirror/image-primary6 2024-09-12T13:04:19.742 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Image promoted to primary 2024-09-12T13:04:19.754 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \n\b\d ]] 2024-09-12T13:04:19.754 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \k\r\b\d ]] 2024-09-12T13:04:19.755 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo rbd --cluster cluster2 device map -t krbd mirror/image-primary6 2024-09-12T13:04:19.886 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ dev=/dev/rbd0 2024-09-12T13:04:19.887 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo md5sum /dev/rbd0 2024-09-12T13:04:19.888 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ awk '{print $1}' 2024-09-12T13:04:21.297 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary8 up+unknown '' '' 2024-09-12T13:04:21.297 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:04:21.297 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:04:21.297 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:04:21.298 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:04:21.298 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:04:21.298 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:04:21.298 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:04:21.298 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:04:21.298 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary8 2024-09-12T13:04:21.368 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary8: 2024-09-12T13:04:21.368 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: e5807591-ea6d-4541-b1c3-0785c5e797ba 2024-09-12T13:04:21.368 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+unknown 2024-09-12T13:04:21.369 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: remote image is not primary 2024-09-12T13:04:21.369 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:04:21.369 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:04:17' 2024-09-12T13:04:21.369 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:04:21.370 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+unknown 2024-09-12T13:04:21.371 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:04:21.373 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: remote image is not primary 2024-09-12T13:04:21.373 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:04:21.374 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:04:21.374 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:04:21.377 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:04:21.377 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:04:21.379 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:04:21.379 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:04:21.380 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:04:21.385 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:04:17' 2024-09-12T13:04:21.385 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary8 up+unknown '2024-09-12 13:04:17' 2024-09-12T13:04:21.385 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:04:21.385 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:04:21.385 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:04:21.385 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:04:21.385 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:04:17' 2024-09-12T13:04:21.385 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:04:21.386 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:04:21.386 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-09-12T13:04:21.488 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041934232mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary15a3f10b5-2891-464a-a2f2-921068cdf6e7up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:04:17image-primary10e8086552-9fee-4a93-b94f-d2d6dc6bff63up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:04:17image-primary2b7ebe2f2-cab7-4e57-a03a-3091186a2999up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:04:17image-primary3c469479f-c19b-4534-a75c-ca68f8c28929up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:04:17image-primary4f5942d86-af5f-4ca1-8ebc-da4cbe6dc314up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:04:17image-primary58b95d658-5f20-45ab-b663-60abf1f860dbup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:04:17image-primary6a790a319-cd27-4a01-b342-b61db0aa2b94up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:04:17image-primary759cd13a4-5c88-4a91-a100-f622774b37f9up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:04:17image-primary8e5807591-ea6d-4541-b1c3-0785c5e797baup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:04:17image-primary97d07d66e-fe3f-46b2-83dc-67fa35b43f0eup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:04:17' 2024-09-12T13:04:21.488 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:04:21.489 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary8'\'']/last_update' 2024-09-12T13:04:21.493 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:04:17' 2024-09-12T13:04:21.494 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary8'\'']/state' 2024-09-12T13:04:21.498 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+unknown 2024-09-12T13:04:21.498 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:04:21.499 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+unknown 2024-09-12T13:04:21.500 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:04:21.501 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:04:21.501 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:04:21.501 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster2 mirror image-primary8 up+unknown 2024-09-12T13:04:21.501 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:04:21.501 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:04:21.502 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:04:21.502 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:04:21.502 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:04:21.502 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:04:21.502 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:04:21.502 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:04:22.446 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary1 up+unknown '' '' 2024-09-12T13:04:22.446 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:04:22.446 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:04:22.446 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:04:22.446 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:04:22.446 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:04:22.446 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:04:22.447 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:04:22.447 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:04:22.447 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary1 2024-09-12T13:04:22.488 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary9 up+unknown '' '' 2024-09-12T13:04:22.488 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:04:22.488 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:04:22.489 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:04:22.489 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:04:22.489 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:04:22.489 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:04:22.489 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:04:22.489 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:04:22.489 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary9 2024-09-12T13:04:22.503 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster2 mirror image-primary8 up+unknown '' '' 2024-09-12T13:04:22.504 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:04:22.504 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:04:22.504 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:04:22.504 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:04:22.504 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:04:22.504 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:04:22.504 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:04:22.504 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:04:22.505 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster2 mirror image status mirror/image-primary8 2024-09-12T13:04:22.522 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary1: 2024-09-12T13:04:22.523 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 5a3f10b5-2891-464a-a2f2-921068cdf6e7 2024-09-12T13:04:22.523 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+unknown 2024-09-12T13:04:22.523 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: remote image is not primary 2024-09-12T13:04:22.523 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:04:22.523 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:04:17' 2024-09-12T13:04:22.523 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:04:22.525 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+unknown 2024-09-12T13:04:22.526 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:04:22.526 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary5 up+unknown '' '' 2024-09-12T13:04:22.526 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:04:22.526 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:04:22.526 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:04:22.526 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:04:22.526 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:04:22.527 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:04:22.527 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:04:22.527 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:04:22.528 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary5 2024-09-12T13:04:22.529 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: remote image is not primary 2024-09-12T13:04:22.529 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:04:22.529 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:04:22.530 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:04:22.532 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:04:22.533 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:04:22.536 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:04:22.537 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:04:22.538 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:04:22.545 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:04:17' 2024-09-12T13:04:22.545 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary1 up+unknown '2024-09-12 13:04:17' 2024-09-12T13:04:22.545 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:04:22.545 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:04:22.545 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:04:22.545 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:04:22.545 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:04:17' 2024-09-12T13:04:22.546 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:04:22.546 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:04:22.546 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-09-12T13:04:22.570 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary9: 2024-09-12T13:04:22.570 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 7d07d66e-fe3f-46b2-83dc-67fa35b43f0e 2024-09-12T13:04:22.570 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+unknown 2024-09-12T13:04:22.570 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: remote image is not primary 2024-09-12T13:04:22.570 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:04:22.570 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:04:17' 2024-09-12T13:04:22.571 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:04:22.573 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+unknown 2024-09-12T13:04:22.573 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:04:22.575 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary4 up+unknown '' '' 2024-09-12T13:04:22.575 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:04:22.575 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:04:22.575 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:04:22.575 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:04:22.575 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:04:22.575 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:04:22.575 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:04:22.577 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:04:22.577 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary4 2024-09-12T13:04:22.577 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: remote image is not primary 2024-09-12T13:04:22.578 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:04:22.578 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:04:22.579 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:04:22.581 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:04:22.582 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:04:22.587 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:04:22.587 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:04:22.588 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:04:22.592 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:04:17' 2024-09-12T13:04:22.592 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary9 up+unknown '2024-09-12 13:04:17' 2024-09-12T13:04:22.593 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:04:22.593 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:04:22.593 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:04:22.593 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:04:22.593 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:04:17' 2024-09-12T13:04:22.593 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:04:22.593 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:04:22.594 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-09-12T13:04:22.595 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary8: 2024-09-12T13:04:22.595 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: e5807591-ea6d-4541-b1c3-0785c5e797ba 2024-09-12T13:04:22.596 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+unknown 2024-09-12T13:04:22.596 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: remote image is not primary 2024-09-12T13:04:22.596 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:04:22.596 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:04:17' 2024-09-12T13:04:22.596 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:04:22.599 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+unknown 2024-09-12T13:04:22.599 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:04:22.602 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: remote image is not primary 2024-09-12T13:04:22.603 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:04:22.603 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:04:22.604 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:04:22.607 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:04:22.607 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:04:22.611 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:04:22.612 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:04:22.613 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:04:22.614 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary5: 2024-09-12T13:04:22.614 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 8b95d658-5f20-45ab-b663-60abf1f860db 2024-09-12T13:04:22.614 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+unknown 2024-09-12T13:04:22.614 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: remote image is not primary 2024-09-12T13:04:22.615 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:04:22.615 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:04:17' 2024-09-12T13:04:22.615 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:04:22.617 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+unknown 2024-09-12T13:04:22.617 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:04:22.619 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:04:17' 2024-09-12T13:04:22.619 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster2 mirror image-primary8 up+unknown '2024-09-12 13:04:17' 2024-09-12T13:04:22.619 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:04:22.620 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:04:22.620 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:04:22.620 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:04:22.620 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:04:17' 2024-09-12T13:04:22.620 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:04:22.620 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:04:22.620 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2024-09-12T13:04:22.621 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: remote image is not primary 2024-09-12T13:04:22.621 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:04:22.622 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:04:22.622 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:04:22.625 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:04:22.626 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:04:22.630 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:04:22.630 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:04:22.631 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:04:22.638 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:04:17' 2024-09-12T13:04:22.638 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary5 up+unknown '2024-09-12 13:04:17' 2024-09-12T13:04:22.638 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:04:22.638 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:04:22.638 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:04:22.638 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:04:22.638 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:04:17' 2024-09-12T13:04:22.638 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:04:22.638 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:04:22.639 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-09-12T13:04:22.676 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041934232mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary15a3f10b5-2891-464a-a2f2-921068cdf6e7up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:04:17image-primary10e8086552-9fee-4a93-b94f-d2d6dc6bff63up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:04:17image-primary2b7ebe2f2-cab7-4e57-a03a-3091186a2999up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:04:17image-primary3c469479f-c19b-4534-a75c-ca68f8c28929up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:04:17image-primary4f5942d86-af5f-4ca1-8ebc-da4cbe6dc314up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:04:17image-primary58b95d658-5f20-45ab-b663-60abf1f860dbup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:04:17image-primary6a790a319-cd27-4a01-b342-b61db0aa2b94up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:04:17image-primary759cd13a4-5c88-4a91-a100-f622774b37f9up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:04:17image-primary8e5807591-ea6d-4541-b1c3-0785c5e797baup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:04:17image-primary97d07d66e-fe3f-46b2-83dc-67fa35b43f0eup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:04:17' 2024-09-12T13:04:22.676 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:04:22.678 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary1'\'']/last_update' 2024-09-12T13:04:22.680 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary4: 2024-09-12T13:04:22.680 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: f5942d86-af5f-4ca1-8ebc-da4cbe6dc314 2024-09-12T13:04:22.680 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+unknown 2024-09-12T13:04:22.681 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: remote image is not primary 2024-09-12T13:04:22.681 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:04:22.681 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:04:17' 2024-09-12T13:04:22.681 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:04:22.685 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+unknown 2024-09-12T13:04:22.686 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:04:17' 2024-09-12T13:04:22.686 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:04:22.686 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary1'\'']/state' 2024-09-12T13:04:22.689 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: remote image is not primary 2024-09-12T13:04:22.691 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:04:22.692 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:04:22.693 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:04:22.694 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+unknown 2024-09-12T13:04:22.694 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:04:22.695 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+unknown 2024-09-12T13:04:22.697 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:04:22.698 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:04:22.699 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:04:22.699 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:04:22.700 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:04:22.700 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster2 mirror image-primary1 up+unknown 2024-09-12T13:04:22.700 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:04:22.700 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:04:22.701 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:04:22.701 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:04:22.701 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:04:22.701 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:04:22.701 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:04:22.701 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:04:22.702 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:04:22.702 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:04:22.703 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:04:22.707 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:04:17' 2024-09-12T13:04:22.708 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary4 up+unknown '2024-09-12 13:04:17' 2024-09-12T13:04:22.708 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:04:22.708 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:04:22.708 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:04:22.708 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:04:22.708 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:04:17' 2024-09-12T13:04:22.708 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:04:22.708 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:04:22.708 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-09-12T13:04:22.759 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041934232mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary15a3f10b5-2891-464a-a2f2-921068cdf6e7up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:04:17image-primary10e8086552-9fee-4a93-b94f-d2d6dc6bff63up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:04:17image-primary2b7ebe2f2-cab7-4e57-a03a-3091186a2999up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:04:17image-primary3c469479f-c19b-4534-a75c-ca68f8c28929up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:04:17image-primary4f5942d86-af5f-4ca1-8ebc-da4cbe6dc314up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:04:17image-primary58b95d658-5f20-45ab-b663-60abf1f860dbup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:04:17image-primary6a790a319-cd27-4a01-b342-b61db0aa2b94up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:04:17image-primary759cd13a4-5c88-4a91-a100-f622774b37f9up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:04:17image-primary8e5807591-ea6d-4541-b1c3-0785c5e797baup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:04:17image-primary97d07d66e-fe3f-46b2-83dc-67fa35b43f0eup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:04:17' 2024-09-12T13:04:22.759 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:04:22.763 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary9'\'']/last_update' 2024-09-12T13:04:22.771 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:04:17' 2024-09-12T13:04:22.772 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary9'\'']/state' 2024-09-12T13:04:22.778 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+unknown 2024-09-12T13:04:22.778 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:04:22.779 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+unknown 2024-09-12T13:04:22.781 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:04:22.782 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:04:22.783 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:04:22.783 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster2 mirror image-primary9 up+unknown 2024-09-12T13:04:22.783 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:04:22.783 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:04:22.783 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:04:22.783 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:04:22.783 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:04:22.783 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:04:22.783 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:04:22.783 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:04:22.790 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041924231mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary15a3f10b5-2891-464a-a2f2-921068cdf6e7up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:04:17image-primary10e8086552-9fee-4a93-b94f-d2d6dc6bff63up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:04:17image-primary2b7ebe2f2-cab7-4e57-a03a-3091186a2999up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:04:17image-primary3c469479f-c19b-4534-a75c-ca68f8c28929up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:04:17image-primary4f5942d86-af5f-4ca1-8ebc-da4cbe6dc314up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:04:17image-primary58b95d658-5f20-45ab-b663-60abf1f860dbup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:04:17image-primary6a790a319-cd27-4a01-b342-b61db0aa2b94up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:04:17image-primary759cd13a4-5c88-4a91-a100-f622774b37f9up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:04:17image-primary8e5807591-ea6d-4541-b1c3-0785c5e797baup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:04:17image-primary97d07d66e-fe3f-46b2-83dc-67fa35b43f0eup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:04:17' 2024-09-12T13:04:22.790 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:04:22.791 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary8'\'']/last_update' 2024-09-12T13:04:22.796 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:04:17' 2024-09-12T13:04:22.796 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary8'\'']/state' 2024-09-12T13:04:22.801 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+unknown 2024-09-12T13:04:22.802 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:04:22.803 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041934232mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary15a3f10b5-2891-464a-a2f2-921068cdf6e7up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:04:17image-primary10e8086552-9fee-4a93-b94f-d2d6dc6bff63up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:04:17image-primary2b7ebe2f2-cab7-4e57-a03a-3091186a2999up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:04:17image-primary3c469479f-c19b-4534-a75c-ca68f8c28929up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:04:17image-primary4f5942d86-af5f-4ca1-8ebc-da4cbe6dc314up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:04:17image-primary58b95d658-5f20-45ab-b663-60abf1f860dbup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:04:17image-primary6a790a319-cd27-4a01-b342-b61db0aa2b94up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:04:17image-primary759cd13a4-5c88-4a91-a100-f622774b37f9up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:04:17image-primary8e5807591-ea6d-4541-b1c3-0785c5e797baup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:04:17image-primary97d07d66e-fe3f-46b2-83dc-67fa35b43f0eup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:04:17' 2024-09-12T13:04:22.803 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+unknown 2024-09-12T13:04:22.803 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:04:22.804 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary5'\'']/last_update' 2024-09-12T13:04:22.805 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:04:22.806 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:04:22.806 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:04:22.806 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ promote_image cluster2 mirror image-primary8 2024-09-12T13:04:22.807 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:04:22.807 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:04:22.807 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:04:22.807 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local force= 2024-09-12T13:04:22.808 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 mirror image promote mirror/image-primary8 2024-09-12T13:04:22.810 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:04:17' 2024-09-12T13:04:22.811 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary5'\'']/state' 2024-09-12T13:04:22.817 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+unknown 2024-09-12T13:04:22.817 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:04:22.818 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+unknown 2024-09-12T13:04:22.818 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:04:22.819 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:04:22.819 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:04:22.819 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster2 mirror image-primary5 up+unknown 2024-09-12T13:04:22.819 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:04:22.820 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:04:22.820 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:04:22.820 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:04:22.820 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:04:22.820 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:04:22.820 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:04:22.821 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:04:22.837 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041934232mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary15a3f10b5-2891-464a-a2f2-921068cdf6e7up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:04:17image-primary10e8086552-9fee-4a93-b94f-d2d6dc6bff63up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:04:17image-primary2b7ebe2f2-cab7-4e57-a03a-3091186a2999up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:04:17image-primary3c469479f-c19b-4534-a75c-ca68f8c28929up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:04:17image-primary4f5942d86-af5f-4ca1-8ebc-da4cbe6dc314up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:04:17image-primary58b95d658-5f20-45ab-b663-60abf1f860dbup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:04:17image-primary6a790a319-cd27-4a01-b342-b61db0aa2b94up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:04:17image-primary759cd13a4-5c88-4a91-a100-f622774b37f9up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:04:17image-primary8e5807591-ea6d-4541-b1c3-0785c5e797baup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:04:17image-primary97d07d66e-fe3f-46b2-83dc-67fa35b43f0eup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:04:17' 2024-09-12T13:04:22.837 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:04:22.838 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary4'\'']/last_update' 2024-09-12T13:04:22.842 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:04:17' 2024-09-12T13:04:22.843 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary4'\'']/state' 2024-09-12T13:04:22.847 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+unknown 2024-09-12T13:04:22.847 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:04:22.848 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+unknown 2024-09-12T13:04:22.850 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:04:22.852 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:04:22.852 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:04:22.852 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster2 mirror image-primary4 up+unknown 2024-09-12T13:04:22.852 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:04:22.852 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:04:22.853 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:04:22.853 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:04:22.853 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:04:22.853 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:04:22.853 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:04:22.853 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:04:23.072 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary7 up+unknown '' '' 2024-09-12T13:04:23.072 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:04:23.072 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:04:23.072 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:04:23.072 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:04:23.072 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:04:23.072 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:04:23.072 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:04:23.073 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:04:23.073 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary7 2024-09-12T13:04:23.142 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary7: 2024-09-12T13:04:23.142 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 59cd13a4-5c88-4a91-a100-f622774b37f9 2024-09-12T13:04:23.142 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+unknown 2024-09-12T13:04:23.142 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: remote image is not primary 2024-09-12T13:04:23.143 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:04:23.143 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:04:17' 2024-09-12T13:04:23.143 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:04:23.145 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+unknown 2024-09-12T13:04:23.145 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:04:23.148 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: remote image is not primary 2024-09-12T13:04:23.148 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:04:23.148 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:04:23.149 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:04:23.151 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:04:23.151 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:04:23.154 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:04:23.154 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:04:23.155 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:04:23.159 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:04:17' 2024-09-12T13:04:23.159 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary7 up+unknown '2024-09-12 13:04:17' 2024-09-12T13:04:23.159 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:04:23.159 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:04:23.159 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:04:23.159 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:04:23.160 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:04:17' 2024-09-12T13:04:23.160 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:04:23.160 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:04:23.160 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-09-12T13:04:23.256 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041934232mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary15a3f10b5-2891-464a-a2f2-921068cdf6e7up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:04:17image-primary10e8086552-9fee-4a93-b94f-d2d6dc6bff63up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:04:17image-primary2b7ebe2f2-cab7-4e57-a03a-3091186a2999up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:04:17image-primary3c469479f-c19b-4534-a75c-ca68f8c28929up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:04:17image-primary4f5942d86-af5f-4ca1-8ebc-da4cbe6dc314up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:04:17image-primary58b95d658-5f20-45ab-b663-60abf1f860dbup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:04:17image-primary6a790a319-cd27-4a01-b342-b61db0aa2b94up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:04:17image-primary759cd13a4-5c88-4a91-a100-f622774b37f9up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:04:17image-primary8e5807591-ea6d-4541-b1c3-0785c5e797baup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:04:17image-primary97d07d66e-fe3f-46b2-83dc-67fa35b43f0eup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:04:17' 2024-09-12T13:04:23.256 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:04:23.256 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary7'\'']/last_update' 2024-09-12T13:04:23.260 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:04:17' 2024-09-12T13:04:23.261 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary7'\'']/state' 2024-09-12T13:04:23.265 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+unknown 2024-09-12T13:04:23.265 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:04:23.266 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+unknown 2024-09-12T13:04:23.267 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:04:23.268 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:04:23.268 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:04:23.268 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster2 mirror image-primary7 up+unknown 2024-09-12T13:04:23.269 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:04:23.269 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:04:23.269 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:04:23.269 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:04:23.269 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:04:23.269 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:04:23.269 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:04:23.269 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:04:23.703 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster2 mirror image-primary1 up+unknown '' '' 2024-09-12T13:04:23.703 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:04:23.703 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:04:23.703 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:04:23.703 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:04:23.703 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:04:23.703 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:04:23.703 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:04:23.704 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:04:23.704 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster2 mirror image status mirror/image-primary1 2024-09-12T13:04:23.747 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Image promoted to primary 2024-09-12T13:04:23.762 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \n\b\d ]] 2024-09-12T13:04:23.762 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \k\r\b\d ]] 2024-09-12T13:04:23.762 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo rbd --cluster cluster2 device map -t krbd mirror/image-primary8 2024-09-12T13:04:23.781 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary1: 2024-09-12T13:04:23.782 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 5a3f10b5-2891-464a-a2f2-921068cdf6e7 2024-09-12T13:04:23.782 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+unknown 2024-09-12T13:04:23.782 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: remote image is not primary 2024-09-12T13:04:23.782 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:04:23.782 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:04:17' 2024-09-12T13:04:23.782 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:04:23.784 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+unknown 2024-09-12T13:04:23.784 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:04:23.785 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster2 mirror image-primary9 up+unknown '' '' 2024-09-12T13:04:23.785 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:04:23.785 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:04:23.785 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:04:23.785 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:04:23.786 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:04:23.786 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:04:23.786 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:04:23.786 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:04:23.786 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster2 mirror image status mirror/image-primary9 2024-09-12T13:04:23.787 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: remote image is not primary 2024-09-12T13:04:23.787 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:04:23.788 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:04:23.788 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:04:23.791 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:04:23.791 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:04:23.793 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:04:23.794 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:04:23.795 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:04:23.798 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:04:17' 2024-09-12T13:04:23.798 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster2 mirror image-primary1 up+unknown '2024-09-12 13:04:17' 2024-09-12T13:04:23.799 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:04:23.799 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:04:23.799 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:04:23.799 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:04:23.799 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:04:17' 2024-09-12T13:04:23.799 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:04:23.799 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:04:23.800 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2024-09-12T13:04:23.822 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster2 mirror image-primary5 up+unknown '' '' 2024-09-12T13:04:23.822 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:04:23.822 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:04:23.822 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:04:23.822 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:04:23.822 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:04:23.823 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:04:23.823 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:04:23.823 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:04:23.824 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster2 mirror image status mirror/image-primary5 2024-09-12T13:04:23.854 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster2 mirror image-primary4 up+unknown '' '' 2024-09-12T13:04:23.854 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:04:23.854 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:04:23.854 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:04:23.854 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:04:23.855 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:04:23.855 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:04:23.855 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:04:23.855 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:04:23.855 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster2 mirror image status mirror/image-primary4 2024-09-12T13:04:23.884 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary9: 2024-09-12T13:04:23.884 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 7d07d66e-fe3f-46b2-83dc-67fa35b43f0e 2024-09-12T13:04:23.884 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+unknown 2024-09-12T13:04:23.884 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: remote image is not primary 2024-09-12T13:04:23.884 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:04:23.885 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:04:17' 2024-09-12T13:04:23.885 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:04:23.887 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+unknown 2024-09-12T13:04:23.888 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:04:23.893 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: remote image is not primary 2024-09-12T13:04:23.895 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:04:23.895 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:04:23.896 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:04:23.899 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:04:23.900 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:04:23.904 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:04:23.904 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:04:23.906 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:04:23.912 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:04:17' 2024-09-12T13:04:23.913 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster2 mirror image-primary9 up+unknown '2024-09-12 13:04:17' 2024-09-12T13:04:23.913 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:04:23.913 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:04:23.913 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:04:23.913 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:04:23.913 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:04:17' 2024-09-12T13:04:23.913 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:04:23.913 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:04:23.914 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2024-09-12T13:04:23.922 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ dev=/dev/rbd1 2024-09-12T13:04:23.925 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo md5sum /dev/rbd1 2024-09-12T13:04:23.925 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ awk '{print $1}' 2024-09-12T13:04:23.975 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary5: 2024-09-12T13:04:23.976 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 8b95d658-5f20-45ab-b663-60abf1f860db 2024-09-12T13:04:23.976 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+unknown 2024-09-12T13:04:23.976 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: remote image is not primary 2024-09-12T13:04:23.976 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:04:23.976 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:04:17' 2024-09-12T13:04:23.976 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:04:23.979 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+unknown 2024-09-12T13:04:23.979 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:04:23.984 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: remote image is not primary 2024-09-12T13:04:23.984 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:04:23.986 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:04:23.986 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:04:23.987 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary4: 2024-09-12T13:04:23.987 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: f5942d86-af5f-4ca1-8ebc-da4cbe6dc314 2024-09-12T13:04:23.987 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+unknown 2024-09-12T13:04:23.987 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: remote image is not primary 2024-09-12T13:04:23.987 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:04:23.988 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:04:17' 2024-09-12T13:04:23.988 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:04:23.988 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041924231mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary15a3f10b5-2891-464a-a2f2-921068cdf6e7up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:04:17image-primary10e8086552-9fee-4a93-b94f-d2d6dc6bff63up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:04:17image-primary2b7ebe2f2-cab7-4e57-a03a-3091186a2999up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:04:17image-primary3c469479f-c19b-4534-a75c-ca68f8c28929up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:04:17image-primary4f5942d86-af5f-4ca1-8ebc-da4cbe6dc314up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:04:17image-primary58b95d658-5f20-45ab-b663-60abf1f860dbup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:04:17image-primary6a790a319-cd27-4a01-b342-b61db0aa2b94up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:04:17image-primary759cd13a4-5c88-4a91-a100-f622774b37f9up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:04:17image-primary8e5807591-ea6d-4541-b1c3-0785c5e797baup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:04:17image-primary97d07d66e-fe3f-46b2-83dc-67fa35b43f0eup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:04:17' 2024-09-12T13:04:23.988 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:04:23.989 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary1'\'']/last_update' 2024-09-12T13:04:23.990 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:04:23.990 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+unknown 2024-09-12T13:04:23.990 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:04:23.990 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:04:23.993 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:04:23.993 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:04:23.994 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: remote image is not primary 2024-09-12T13:04:23.995 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:04:17' 2024-09-12T13:04:23.995 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:04:23.995 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:04:23.996 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary1'\'']/state' 2024-09-12T13:04:23.996 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:04:23.997 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:04:24.000 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:04:24.000 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:04:24.001 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+unknown 2024-09-12T13:04:24.002 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:04:17' 2024-09-12T13:04:24.002 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster2 mirror image-primary5 up+unknown '2024-09-12 13:04:17' 2024-09-12T13:04:24.002 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:04:24.003 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:04:24.003 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:04:24.003 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:04:24.003 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:04:17' 2024-09-12T13:04:24.003 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:04:24.003 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:04:24.003 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+unknown 2024-09-12T13:04:24.004 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:04:24.004 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2024-09-12T13:04:24.004 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:04:24.004 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:04:24.005 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:04:24.006 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:04:24.007 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:04:24.007 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:04:24.007 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ promote_image cluster2 mirror image-primary1 2024-09-12T13:04:24.007 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:04:24.007 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:04:24.008 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:04:24.008 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local force= 2024-09-12T13:04:24.008 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 mirror image promote mirror/image-primary1 2024-09-12T13:04:24.013 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary3 up+unknown '' '' 2024-09-12T13:04:24.013 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:04:24.013 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:04:24.013 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:04:24.013 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:04:24.013 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:04:24.013 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:04:24.013 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:04:24.014 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:04:17' 2024-09-12T13:04:24.014 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster2 mirror image-primary4 up+unknown '2024-09-12 13:04:17' 2024-09-12T13:04:24.014 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:04:24.014 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:04:24.014 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:04:24.014 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:04:24.014 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:04:17' 2024-09-12T13:04:24.014 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:04:24.014 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:04:24.015 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2024-09-12T13:04:24.015 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary10 up+unknown '' '' 2024-09-12T13:04:24.015 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:04:24.015 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:04:24.015 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:04:24.016 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:04:24.016 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:04:24.016 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:04:24.016 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:04:24.016 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:04:24.016 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary3 2024-09-12T13:04:24.016 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:04:24.016 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary10 2024-09-12T13:04:24.183 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041924231mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary15a3f10b5-2891-464a-a2f2-921068cdf6e7up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:04:17image-primary10e8086552-9fee-4a93-b94f-d2d6dc6bff63up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:04:17image-primary2b7ebe2f2-cab7-4e57-a03a-3091186a2999up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:04:17image-primary3c469479f-c19b-4534-a75c-ca68f8c28929up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:04:17image-primary4f5942d86-af5f-4ca1-8ebc-da4cbe6dc314up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:04:17image-primary58b95d658-5f20-45ab-b663-60abf1f860dbup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:04:17image-primary6a790a319-cd27-4a01-b342-b61db0aa2b94up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:04:17image-primary759cd13a4-5c88-4a91-a100-f622774b37f9up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:04:17image-primary8e5807591-ea6d-4541-b1c3-0785c5e797baup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:04:17image-primary97d07d66e-fe3f-46b2-83dc-67fa35b43f0eup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:04:17' 2024-09-12T13:04:24.183 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:04:24.183 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary9'\'']/last_update' 2024-09-12T13:04:24.185 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary3: 2024-09-12T13:04:24.185 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: c469479f-c19b-4534-a75c-ca68f8c28929 2024-09-12T13:04:24.185 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+unknown 2024-09-12T13:04:24.185 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: remote image is not primary 2024-09-12T13:04:24.185 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:04:24.186 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:04:17' 2024-09-12T13:04:24.186 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:04:24.190 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+unknown 2024-09-12T13:04:24.191 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:04:24.191 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:04:17' 2024-09-12T13:04:24.193 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary9'\'']/state' 2024-09-12T13:04:24.195 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: remote image is not primary 2024-09-12T13:04:24.197 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:04:24.198 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:04:24.199 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:04:24.203 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary10: 2024-09-12T13:04:24.203 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: e8086552-9fee-4a93-b94f-d2d6dc6bff63 2024-09-12T13:04:24.203 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+unknown 2024-09-12T13:04:24.203 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: remote image is not primary 2024-09-12T13:04:24.203 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:04:24.203 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:04:17' 2024-09-12T13:04:24.203 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:04:24.206 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:04:24.207 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+unknown 2024-09-12T13:04:24.207 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:04:24.207 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:04:24.208 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+unknown 2024-09-12T13:04:24.215 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:04:24.215 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:04:24.215 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+unknown 2024-09-12T13:04:24.216 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:04:24.216 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:04:24.216 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:04:24.216 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:04:24.220 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ promote_image cluster2 mirror image-primary9 2024-09-12T13:04:24.220 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:04:24.220 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:04:24.220 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:04:24.220 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local force= 2024-09-12T13:04:24.221 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 mirror image promote mirror/image-primary9 2024-09-12T13:04:24.221 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:04:24.225 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: remote image is not primary 2024-09-12T13:04:24.226 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:04:24.227 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:04:24.227 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:04:24.229 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:04:17' 2024-09-12T13:04:24.230 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary3 up+unknown '2024-09-12 13:04:17' 2024-09-12T13:04:24.230 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:04:24.230 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:04:24.230 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:04:24.230 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:04:24.230 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:04:17' 2024-09-12T13:04:24.230 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:04:24.234 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:04:24.234 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-09-12T13:04:24.235 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:04:24.236 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:04:24.240 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:04:24.241 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:04:24.242 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:04:24.247 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:04:17' 2024-09-12T13:04:24.247 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary10 up+unknown '2024-09-12 13:04:17' 2024-09-12T13:04:24.247 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:04:24.247 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:04:24.247 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:04:24.247 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:04:24.247 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:04:17' 2024-09-12T13:04:24.247 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:04:24.248 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:04:24.248 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-09-12T13:04:24.255 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041924231mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary15a3f10b5-2891-464a-a2f2-921068cdf6e7up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:04:17image-primary10e8086552-9fee-4a93-b94f-d2d6dc6bff63up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:04:17image-primary2b7ebe2f2-cab7-4e57-a03a-3091186a2999up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:04:17image-primary3c469479f-c19b-4534-a75c-ca68f8c28929up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:04:17image-primary4f5942d86-af5f-4ca1-8ebc-da4cbe6dc314up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:04:17image-primary58b95d658-5f20-45ab-b663-60abf1f860dbup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:04:17image-primary6a790a319-cd27-4a01-b342-b61db0aa2b94up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:04:17image-primary759cd13a4-5c88-4a91-a100-f622774b37f9up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:04:17image-primary8e5807591-ea6d-4541-b1c3-0785c5e797baup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:04:17image-primary97d07d66e-fe3f-46b2-83dc-67fa35b43f0eup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:04:17' 2024-09-12T13:04:24.256 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:04:24.256 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary4'\'']/last_update' 2024-09-12T13:04:24.259 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041924231mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary15a3f10b5-2891-464a-a2f2-921068cdf6e7up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:04:17image-primary10e8086552-9fee-4a93-b94f-d2d6dc6bff63up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:04:17image-primary2b7ebe2f2-cab7-4e57-a03a-3091186a2999up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:04:17image-primary3c469479f-c19b-4534-a75c-ca68f8c28929up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:04:17image-primary4f5942d86-af5f-4ca1-8ebc-da4cbe6dc314up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:04:17image-primary58b95d658-5f20-45ab-b663-60abf1f860dbup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:04:17image-primary6a790a319-cd27-4a01-b342-b61db0aa2b94up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:04:17image-primary759cd13a4-5c88-4a91-a100-f622774b37f9up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:04:17image-primary8e5807591-ea6d-4541-b1c3-0785c5e797baup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:04:17image-primary97d07d66e-fe3f-46b2-83dc-67fa35b43f0eup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:04:17' 2024-09-12T13:04:24.259 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:04:24.260 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary5'\'']/last_update' 2024-09-12T13:04:24.261 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:04:17' 2024-09-12T13:04:24.261 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary4'\'']/state' 2024-09-12T13:04:24.267 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+unknown 2024-09-12T13:04:24.267 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:04:24.268 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:04:17' 2024-09-12T13:04:24.268 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+unknown 2024-09-12T13:04:24.268 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary5'\'']/state' 2024-09-12T13:04:24.271 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster2 mirror image-primary7 up+unknown '' '' 2024-09-12T13:04:24.271 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:04:24.271 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:04:24.271 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:04:24.271 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:04:24.271 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:04:24.271 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:04:24.271 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:04:24.272 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:04:24.272 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:04:24.272 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster2 mirror image status mirror/image-primary7 2024-09-12T13:04:24.273 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:04:24.273 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:04:24.273 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ promote_image cluster2 mirror image-primary4 2024-09-12T13:04:24.273 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:04:24.273 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:04:24.274 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:04:24.274 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local force= 2024-09-12T13:04:24.274 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 mirror image promote mirror/image-primary4 2024-09-12T13:04:24.277 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+unknown 2024-09-12T13:04:24.277 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:04:24.281 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+unknown 2024-09-12T13:04:24.281 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:04:24.282 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:04:24.283 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:04:24.283 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ promote_image cluster2 mirror image-primary5 2024-09-12T13:04:24.283 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:04:24.283 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:04:24.283 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:04:24.283 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local force= 2024-09-12T13:04:24.283 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 mirror image promote mirror/image-primary5 2024-09-12T13:04:24.411 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary7: 2024-09-12T13:04:24.411 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 59cd13a4-5c88-4a91-a100-f622774b37f9 2024-09-12T13:04:24.412 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+unknown 2024-09-12T13:04:24.412 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: remote image is not primary 2024-09-12T13:04:24.412 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:04:24.412 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:04:17' 2024-09-12T13:04:24.412 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:04:24.415 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+unknown 2024-09-12T13:04:24.416 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:04:24.421 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: remote image is not primary 2024-09-12T13:04:24.421 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:04:24.423 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:04:24.423 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:04:24.426 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041934232mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary15a3f10b5-2891-464a-a2f2-921068cdf6e7up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:04:17image-primary10e8086552-9fee-4a93-b94f-d2d6dc6bff63up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:04:17image-primary2b7ebe2f2-cab7-4e57-a03a-3091186a2999up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:04:17image-primary3c469479f-c19b-4534-a75c-ca68f8c28929up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:04:17image-primary4f5942d86-af5f-4ca1-8ebc-da4cbe6dc314up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:04:17image-primary58b95d658-5f20-45ab-b663-60abf1f860dbup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:04:17image-primary6a790a319-cd27-4a01-b342-b61db0aa2b94up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:04:17image-primary759cd13a4-5c88-4a91-a100-f622774b37f9up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:04:17image-primary8e5807591-ea6d-4541-b1c3-0785c5e797baup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:04:17image-primary97d07d66e-fe3f-46b2-83dc-67fa35b43f0eup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:04:17' 2024-09-12T13:04:24.427 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:04:24.427 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:04:24.427 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:04:24.427 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary3'\'']/last_update' 2024-09-12T13:04:24.430 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:04:24.431 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:04:24.432 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:04:24.433 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:04:17' 2024-09-12T13:04:24.434 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary3'\'']/state' 2024-09-12T13:04:24.436 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:04:17' 2024-09-12T13:04:24.436 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster2 mirror image-primary7 up+unknown '2024-09-12 13:04:17' 2024-09-12T13:04:24.436 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:04:24.436 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:04:24.436 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:04:24.437 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:04:24.437 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:04:17' 2024-09-12T13:04:24.437 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:04:24.437 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:04:24.437 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2024-09-12T13:04:24.439 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+unknown 2024-09-12T13:04:24.439 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:04:24.440 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+unknown 2024-09-12T13:04:24.442 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:04:24.443 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:04:24.443 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:04:24.443 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster2 mirror image-primary3 up+unknown 2024-09-12T13:04:24.443 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:04:24.444 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:04:24.444 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:04:24.444 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:04:24.444 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:04:24.444 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:04:24.444 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:04:24.444 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:04:24.445 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041934232mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary15a3f10b5-2891-464a-a2f2-921068cdf6e7up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:04:17image-primary10e8086552-9fee-4a93-b94f-d2d6dc6bff63up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:04:17image-primary2b7ebe2f2-cab7-4e57-a03a-3091186a2999up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:04:17image-primary3c469479f-c19b-4534-a75c-ca68f8c28929up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:04:17image-primary4f5942d86-af5f-4ca1-8ebc-da4cbe6dc314up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:04:17image-primary58b95d658-5f20-45ab-b663-60abf1f860dbup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:04:17image-primary6a790a319-cd27-4a01-b342-b61db0aa2b94up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:04:17image-primary759cd13a4-5c88-4a91-a100-f622774b37f9up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:04:17image-primary8e5807591-ea6d-4541-b1c3-0785c5e797baup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:04:17image-primary97d07d66e-fe3f-46b2-83dc-67fa35b43f0eup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:04:17' 2024-09-12T13:04:24.445 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:04:24.446 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary10'\'']/last_update' 2024-09-12T13:04:24.450 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:04:17' 2024-09-12T13:04:24.451 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary10'\'']/state' 2024-09-12T13:04:24.455 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+unknown 2024-09-12T13:04:24.456 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:04:24.456 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+unknown 2024-09-12T13:04:24.458 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:04:24.460 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:04:24.460 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:04:24.460 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster2 mirror image-primary10 up+unknown 2024-09-12T13:04:24.460 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:04:24.460 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:04:24.460 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:04:24.460 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:04:24.460 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:04:24.460 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:04:24.460 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:04:24.460 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:04:24.559 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041924231mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary15a3f10b5-2891-464a-a2f2-921068cdf6e7up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:04:17image-primary10e8086552-9fee-4a93-b94f-d2d6dc6bff63up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:04:17image-primary2b7ebe2f2-cab7-4e57-a03a-3091186a2999up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:04:17image-primary3c469479f-c19b-4534-a75c-ca68f8c28929up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:04:17image-primary4f5942d86-af5f-4ca1-8ebc-da4cbe6dc314up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:04:17image-primary58b95d658-5f20-45ab-b663-60abf1f860dbup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:04:17image-primary6a790a319-cd27-4a01-b342-b61db0aa2b94up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:04:17image-primary759cd13a4-5c88-4a91-a100-f622774b37f9up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:04:17image-primary8e5807591-ea6d-4541-b1c3-0785c5e797baup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:04:17image-primary97d07d66e-fe3f-46b2-83dc-67fa35b43f0eup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:04:17' 2024-09-12T13:04:24.560 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:04:24.560 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary7'\'']/last_update' 2024-09-12T13:04:24.566 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:04:17' 2024-09-12T13:04:24.566 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary7'\'']/state' 2024-09-12T13:04:24.569 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+unknown 2024-09-12T13:04:24.570 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:04:24.570 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+unknown 2024-09-12T13:04:24.572 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:04:24.573 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:04:24.573 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:04:24.573 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ promote_image cluster2 mirror image-primary7 2024-09-12T13:04:24.573 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:04:24.573 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:04:24.574 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:04:24.574 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local force= 2024-09-12T13:04:24.574 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 mirror image promote mirror/image-primary7 2024-09-12T13:04:24.744 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary2 up+unknown '' '' 2024-09-12T13:04:24.744 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:04:24.744 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:04:24.744 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:04:24.744 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:04:24.744 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:04:24.744 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:04:24.744 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:04:24.745 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:04:24.745 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary2 2024-09-12T13:04:24.774 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Image promoted to primary 2024-09-12T13:04:24.776 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Image promoted to primary 2024-09-12T13:04:24.787 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Image promoted to primary 2024-09-12T13:04:24.787 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Image promoted to primary 2024-09-12T13:04:24.793 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Image promoted to primary 2024-09-12T13:04:24.806 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \n\b\d ]] 2024-09-12T13:04:24.807 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \k\r\b\d ]] 2024-09-12T13:04:24.808 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo rbd --cluster cluster2 device map -t krbd mirror/image-primary7 2024-09-12T13:04:24.810 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \n\b\d ]] 2024-09-12T13:04:24.810 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \k\r\b\d ]] 2024-09-12T13:04:24.810 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \n\b\d ]] 2024-09-12T13:04:24.810 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \k\r\b\d ]] 2024-09-12T13:04:24.810 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo rbd --cluster cluster2 device map -t krbd mirror/image-primary5 2024-09-12T13:04:24.812 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo rbd --cluster cluster2 device map -t krbd mirror/image-primary9 2024-09-12T13:04:24.821 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \n\b\d ]] 2024-09-12T13:04:24.821 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \k\r\b\d ]] 2024-09-12T13:04:24.821 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \n\b\d ]] 2024-09-12T13:04:24.821 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \k\r\b\d ]] 2024-09-12T13:04:24.821 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo rbd --cluster cluster2 device map -t krbd mirror/image-primary4 2024-09-12T13:04:24.823 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo rbd --cluster cluster2 device map -t krbd mirror/image-primary1 2024-09-12T13:04:24.874 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary2: 2024-09-12T13:04:24.874 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: b7ebe2f2-cab7-4e57-a03a-3091186a2999 2024-09-12T13:04:24.874 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+unknown 2024-09-12T13:04:24.874 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: remote image is not primary 2024-09-12T13:04:24.875 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:04:24.875 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:04:17' 2024-09-12T13:04:24.875 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:04:24.879 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+unknown 2024-09-12T13:04:24.880 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:04:24.885 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: remote image is not primary 2024-09-12T13:04:24.885 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:04:24.887 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:04:24.887 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:04:24.890 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:04:24.895 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:04:24.896 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:04:24.897 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:04:24.899 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:04:24.906 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:04:17' 2024-09-12T13:04:24.906 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary2 up+unknown '2024-09-12 13:04:17' 2024-09-12T13:04:24.906 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:04:24.906 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:04:24.906 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:04:24.906 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:04:24.906 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:04:17' 2024-09-12T13:04:24.906 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:04:24.907 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:04:24.908 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-09-12T13:04:25.048 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ dev=/dev/rbd3 2024-09-12T13:04:25.051 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo md5sum /dev/rbd3 2024-09-12T13:04:25.054 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ awk '{print $1}' 2024-09-12T13:04:25.071 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ dev=/dev/rbd5 2024-09-12T13:04:25.073 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ dev=/dev/rbd2 2024-09-12T13:04:25.074 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo md5sum /dev/rbd5 2024-09-12T13:04:25.076 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ dev=/dev/rbd4 2024-09-12T13:04:25.076 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ awk '{print $1}' 2024-09-12T13:04:25.077 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo md5sum /dev/rbd2 2024-09-12T13:04:25.077 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ awk '{print $1}' 2024-09-12T13:04:25.079 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo md5sum /dev/rbd4 2024-09-12T13:04:25.080 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ awk '{print $1}' 2024-09-12T13:04:25.104 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ dev=/dev/rbd6 2024-09-12T13:04:25.106 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ awk '{print $1}' 2024-09-12T13:04:25.106 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo md5sum /dev/rbd6 2024-09-12T13:04:25.159 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041934232mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary15a3f10b5-2891-464a-a2f2-921068cdf6e7up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:04:17image-primary10e8086552-9fee-4a93-b94f-d2d6dc6bff63up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:04:17image-primary2b7ebe2f2-cab7-4e57-a03a-3091186a2999up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:04:17image-primary3c469479f-c19b-4534-a75c-ca68f8c28929up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:04:17image-primary4f5942d86-af5f-4ca1-8ebc-da4cbe6dc314up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:04:17image-primary58b95d658-5f20-45ab-b663-60abf1f860dbup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:04:17image-primary6a790a319-cd27-4a01-b342-b61db0aa2b94up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:04:17image-primary759cd13a4-5c88-4a91-a100-f622774b37f9up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:04:17image-primary8e5807591-ea6d-4541-b1c3-0785c5e797baup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:04:17image-primary97d07d66e-fe3f-46b2-83dc-67fa35b43f0eup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:04:17' 2024-09-12T13:04:25.160 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:04:25.162 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary2'\'']/last_update' 2024-09-12T13:04:25.169 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:04:17' 2024-09-12T13:04:25.171 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary2'\'']/state' 2024-09-12T13:04:25.180 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+unknown 2024-09-12T13:04:25.183 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:04:25.183 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+unknown 2024-09-12T13:04:25.186 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:04:25.194 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:04:25.194 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:04:25.194 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster2 mirror image-primary2 up+unknown 2024-09-12T13:04:25.194 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:04:25.194 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:04:25.194 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:04:25.194 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:04:25.195 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:04:25.195 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:04:25.195 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:04:25.195 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:04:25.448 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster2 mirror image-primary3 up+unknown '' '' 2024-09-12T13:04:25.466 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:04:25.467 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:04:25.467 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:04:25.467 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:04:25.467 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:04:25.467 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:04:25.467 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:04:25.468 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:04:25.468 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster2 mirror image status mirror/image-primary3 2024-09-12T13:04:25.468 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster2 mirror image-primary10 up+unknown '' '' 2024-09-12T13:04:25.468 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:04:25.468 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:04:25.468 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:04:25.469 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:04:25.469 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:04:25.469 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:04:25.469 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:04:25.470 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:04:25.475 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster2 mirror image status mirror/image-primary10 2024-09-12T13:04:25.634 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary3: 2024-09-12T13:04:25.634 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: c469479f-c19b-4534-a75c-ca68f8c28929 2024-09-12T13:04:25.634 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+unknown 2024-09-12T13:04:25.634 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: remote image is not primary 2024-09-12T13:04:25.634 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:04:25.634 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:04:17' 2024-09-12T13:04:25.635 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:04:25.639 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+unknown 2024-09-12T13:04:25.640 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:04:25.648 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: remote image is not primary 2024-09-12T13:04:25.649 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:04:25.650 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:04:25.651 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:04:25.655 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:04:25.662 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary10: 2024-09-12T13:04:25.662 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: e8086552-9fee-4a93-b94f-d2d6dc6bff63 2024-09-12T13:04:25.662 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+unknown 2024-09-12T13:04:25.662 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: remote image is not primary 2024-09-12T13:04:25.662 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:04:25.662 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:04:17' 2024-09-12T13:04:25.662 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:04:25.666 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:04:25.668 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+unknown 2024-09-12T13:04:25.669 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:04:25.673 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: remote image is not primary 2024-09-12T13:04:25.673 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:04:25.675 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:04:25.676 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:04:25.677 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:04:25.678 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:04:25.680 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:04:25.682 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:04:25.682 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:04:25.687 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:04:25.688 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:04:25.690 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:04:25.697 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:04:17' 2024-09-12T13:04:25.697 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster2 mirror image-primary3 up+unknown '2024-09-12 13:04:17' 2024-09-12T13:04:25.697 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:04:25.697 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:04:25.697 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:04:25.697 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:04:17' 2024-09-12T13:04:25.697 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:04:25.698 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster2 mirror image-primary10 up+unknown '2024-09-12 13:04:17' 2024-09-12T13:04:25.698 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:04:25.698 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:04:17' 2024-09-12T13:04:25.698 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:04:25.698 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:04:25.698 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:04:25.698 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:04:25.698 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:04:17' 2024-09-12T13:04:25.698 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:04:25.698 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:04:25.699 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2024-09-12T13:04:25.701 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:04:25.701 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2024-09-12T13:04:26.067 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041924231mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary15a3f10b5-2891-464a-a2f2-921068cdf6e7up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:04:17image-primary10e8086552-9fee-4a93-b94f-d2d6dc6bff63up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:04:17image-primary2b7ebe2f2-cab7-4e57-a03a-3091186a2999up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:04:17image-primary3c469479f-c19b-4534-a75c-ca68f8c28929up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:04:17image-primary4f5942d86-af5f-4ca1-8ebc-da4cbe6dc314up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:04:17image-primary58b95d658-5f20-45ab-b663-60abf1f860dbup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:04:17image-primary6a790a319-cd27-4a01-b342-b61db0aa2b94up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:04:17image-primary759cd13a4-5c88-4a91-a100-f622774b37f9up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:04:17image-primary8e5807591-ea6d-4541-b1c3-0785c5e797baup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:04:17image-primary97d07d66e-fe3f-46b2-83dc-67fa35b43f0eup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:04:17' 2024-09-12T13:04:26.067 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:04:26.069 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary3'\'']/last_update' 2024-09-12T13:04:26.077 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:04:17' 2024-09-12T13:04:26.079 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary3'\'']/state' 2024-09-12T13:04:26.087 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+unknown 2024-09-12T13:04:26.090 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:04:26.091 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+unknown 2024-09-12T13:04:26.094 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:04:26.096 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:04:26.096 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:04:26.096 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ promote_image cluster2 mirror image-primary3 2024-09-12T13:04:26.096 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:04:26.096 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:04:26.096 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:04:26.096 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local force= 2024-09-12T13:04:26.096 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 mirror image promote mirror/image-primary3 2024-09-12T13:04:26.099 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041924231mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary15a3f10b5-2891-464a-a2f2-921068cdf6e7up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:04:17image-primary10e8086552-9fee-4a93-b94f-d2d6dc6bff63up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:04:17image-primary2b7ebe2f2-cab7-4e57-a03a-3091186a2999up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:04:17image-primary3c469479f-c19b-4534-a75c-ca68f8c28929up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:04:17image-primary4f5942d86-af5f-4ca1-8ebc-da4cbe6dc314up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:04:17image-primary58b95d658-5f20-45ab-b663-60abf1f860dbup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:04:17image-primary6a790a319-cd27-4a01-b342-b61db0aa2b94up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:04:17image-primary759cd13a4-5c88-4a91-a100-f622774b37f9up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:04:17image-primary8e5807591-ea6d-4541-b1c3-0785c5e797baup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:04:17image-primary97d07d66e-fe3f-46b2-83dc-67fa35b43f0eup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:04:17' 2024-09-12T13:04:26.099 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:04:26.101 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary10'\'']/last_update' 2024-09-12T13:04:26.118 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:04:17' 2024-09-12T13:04:26.121 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary10'\'']/state' 2024-09-12T13:04:26.135 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+unknown 2024-09-12T13:04:26.136 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:04:26.139 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+unknown 2024-09-12T13:04:26.142 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:04:26.151 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:04:26.151 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:04:26.151 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ promote_image cluster2 mirror image-primary10 2024-09-12T13:04:26.151 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:04:26.151 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:04:26.151 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:04:26.151 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local force= 2024-09-12T13:04:26.152 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 mirror image promote mirror/image-primary10 2024-09-12T13:04:26.204 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster2 mirror image-primary2 up+unknown '' '' 2024-09-12T13:04:26.204 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:04:26.205 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:04:26.205 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:04:26.205 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:04:26.205 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:04:26.205 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:04:26.205 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:04:26.208 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:04:26.208 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster2 mirror image status mirror/image-primary2 2024-09-12T13:04:26.395 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary2: 2024-09-12T13:04:26.395 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: b7ebe2f2-cab7-4e57-a03a-3091186a2999 2024-09-12T13:04:26.395 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+unknown 2024-09-12T13:04:26.396 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: remote image is not primary 2024-09-12T13:04:26.396 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:04:26.396 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:04:17' 2024-09-12T13:04:26.396 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:04:26.399 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+unknown 2024-09-12T13:04:26.400 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:04:26.404 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: remote image is not primary 2024-09-12T13:04:26.405 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:04:26.408 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:04:26.408 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:04:26.412 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:04:26.412 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:04:26.418 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:04:26.418 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:04:26.420 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:04:26.428 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:04:17' 2024-09-12T13:04:26.428 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster2 mirror image-primary2 up+unknown '2024-09-12 13:04:17' 2024-09-12T13:04:26.428 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:04:26.428 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:04:26.428 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:04:26.428 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:04:26.428 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:04:17' 2024-09-12T13:04:26.428 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:04:26.430 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:04:26.430 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2024-09-12T13:04:26.667 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041924231mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary15a3f10b5-2891-464a-a2f2-921068cdf6e7up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:04:17image-primary10e8086552-9fee-4a93-b94f-d2d6dc6bff63up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:04:17image-primary2b7ebe2f2-cab7-4e57-a03a-3091186a2999up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:04:17image-primary3c469479f-c19b-4534-a75c-ca68f8c28929up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:04:17image-primary4f5942d86-af5f-4ca1-8ebc-da4cbe6dc314up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:04:17image-primary58b95d658-5f20-45ab-b663-60abf1f860dbup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:04:17image-primary6a790a319-cd27-4a01-b342-b61db0aa2b94up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:04:17image-primary759cd13a4-5c88-4a91-a100-f622774b37f9up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:04:17image-primary8e5807591-ea6d-4541-b1c3-0785c5e797baup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:04:17image-primary97d07d66e-fe3f-46b2-83dc-67fa35b43f0eup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:04:17' 2024-09-12T13:04:26.668 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:04:26.669 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary2'\'']/last_update' 2024-09-12T13:04:26.677 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:04:17' 2024-09-12T13:04:26.678 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary2'\'']/state' 2024-09-12T13:04:26.686 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+unknown 2024-09-12T13:04:26.687 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:04:26.688 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+unknown 2024-09-12T13:04:26.692 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:04:26.693 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:04:26.693 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:04:26.694 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ promote_image cluster2 mirror image-primary2 2024-09-12T13:04:26.694 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:04:26.694 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:04:26.694 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:04:26.694 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local force= 2024-09-12T13:04:26.694 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 mirror image promote mirror/image-primary2 2024-09-12T13:04:26.809 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Image promoted to primary 2024-09-12T13:04:26.819 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Image promoted to primary 2024-09-12T13:04:26.866 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \n\b\d ]] 2024-09-12T13:04:26.866 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \k\r\b\d ]] 2024-09-12T13:04:26.867 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \n\b\d ]] 2024-09-12T13:04:26.867 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \k\r\b\d ]] 2024-09-12T13:04:26.869 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo rbd --cluster cluster2 device map -t krbd mirror/image-primary3 2024-09-12T13:04:26.869 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo rbd --cluster cluster2 device map -t krbd mirror/image-primary10 2024-09-12T13:04:27.185 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ dev=/dev/rbd7 2024-09-12T13:04:27.191 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo md5sum /dev/rbd7 2024-09-12T13:04:27.197 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ awk '{print $1}' 2024-09-12T13:04:27.232 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ dev=/dev/rbd8 2024-09-12T13:04:27.238 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo md5sum /dev/rbd8 2024-09-12T13:04:27.239 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ awk '{print $1}' 2024-09-12T13:04:27.830 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Image promoted to primary 2024-09-12T13:04:27.869 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \n\b\d ]] 2024-09-12T13:04:27.869 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \k\r\b\d ]] 2024-09-12T13:04:27.873 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo rbd --cluster cluster2 device map -t krbd mirror/image-primary2 2024-09-12T13:04:28.286 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ dev=/dev/rbd9 2024-09-12T13:04:28.293 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo md5sum /dev/rbd9 2024-09-12T13:04:28.294 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ awk '{print $1}' 2024-09-12T13:05:13.868 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ promote_md5=b91f8d6f5e9db3c178acac32a0fbaddd 2024-09-12T13:05:13.869 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo rbd --cluster cluster2 device unmap -t krbd /dev/rbd0 2024-09-12T13:05:14.590 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ b91f8d6f5e9db3c178acac32a0fbaddd != \b\9\1\f\8\d\6\f\5\e\9\d\b\3\c\1\7\8\a\c\a\c\3\2\a\0\f\b\a\d\d\d ]] 2024-09-12T13:05:17.119 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ promote_md5=1c46232c0659be87c589995110ffb9af 2024-09-12T13:05:17.119 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo rbd --cluster cluster2 device unmap -t krbd /dev/rbd1 2024-09-12T13:05:17.296 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ promote_md5=ed95043a70577e18d5c8cd422e6d8d86 2024-09-12T13:05:17.297 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo rbd --cluster cluster2 device unmap -t krbd /dev/rbd4 2024-09-12T13:05:17.371 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ 1c46232c0659be87c589995110ffb9af != \1\c\4\6\2\3\2\c\0\6\5\9\b\e\8\7\c\5\8\9\9\9\5\1\1\0\f\f\b\9\a\f ]] 2024-09-12T13:05:17.523 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ ed95043a70577e18d5c8cd422e6d8d86 != \e\d\9\5\0\4\3\a\7\0\5\7\7\e\1\8\d\5\c\8\c\d\4\2\2\e\6\d\8\d\8\6 ]] 2024-09-12T13:05:17.524 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${COMPARE_PIDS[@]} 2024-09-12T13:05:17.524 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 42838 2024-09-12T13:05:17.927 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ promote_md5=556b8bb44ef8d2d6fce0910eee93534d 2024-09-12T13:05:17.927 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo rbd --cluster cluster2 device unmap -t krbd /dev/rbd3 2024-09-12T13:05:18.095 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ 556b8bb44ef8d2d6fce0910eee93534d != \5\5\6\b\8\b\b\4\4\e\f\8\d\2\d\6\f\c\e\0\9\1\0\e\e\e\9\3\5\3\4\d ]] 2024-09-12T13:05:19.093 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ promote_md5=60fdbc55a27e7b769003b837ab23f416 2024-09-12T13:05:19.093 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo rbd --cluster cluster2 device unmap -t krbd /dev/rbd6 2024-09-12T13:05:19.184 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ promote_md5=6f74976b8ee4ca5d9633fddc227d3915 2024-09-12T13:05:19.184 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo rbd --cluster cluster2 device unmap -t krbd /dev/rbd2 2024-09-12T13:05:19.225 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ 60fdbc55a27e7b769003b837ab23f416 != \6\0\f\d\b\c\5\5\a\2\7\e\7\b\7\6\9\0\0\3\b\8\3\7\a\b\2\3\f\4\1\6 ]] 2024-09-12T13:05:19.327 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ promote_md5=7069169804ded02d8cc7feac58e1c5fd 2024-09-12T13:05:19.327 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo rbd --cluster cluster2 device unmap -t krbd /dev/rbd5 2024-09-12T13:05:19.345 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ 6f74976b8ee4ca5d9633fddc227d3915 != \6\f\7\4\9\7\6\b\8\e\e\4\c\a\5\d\9\6\3\3\f\d\d\c\2\2\7\d\3\9\1\5 ]] 2024-09-12T13:05:19.526 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ 7069169804ded02d8cc7feac58e1c5fd != \7\0\6\9\1\6\9\8\0\4\d\e\d\0\2\d\8\c\c\7\f\e\a\c\5\8\e\1\c\5\f\d ]] 2024-09-12T13:05:20.633 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ promote_md5=8973418b9c4ac572630aa950c6432bf0 2024-09-12T13:05:20.633 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo rbd --cluster cluster2 device unmap -t krbd /dev/rbd9 2024-09-12T13:05:20.817 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ 8973418b9c4ac572630aa950c6432bf0 != \8\9\7\3\4\1\8\b\9\c\4\a\c\5\7\2\6\3\0\a\a\9\5\0\c\6\4\3\2\b\f\0 ]] 2024-09-12T13:05:20.818 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${COMPARE_PIDS[@]} 2024-09-12T13:05:20.818 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 42840 2024-09-12T13:05:20.854 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ promote_md5=02c7b7be3bb22b94ac7599660ca16db7 2024-09-12T13:05:20.854 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo rbd --cluster cluster2 device unmap -t krbd /dev/rbd7 2024-09-12T13:05:20.945 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ 02c7b7be3bb22b94ac7599660ca16db7 != \0\2\c\7\b\7\b\e\3\b\b\2\2\b\9\4\a\c\7\5\9\9\6\6\0\c\a\1\6\d\b\7 ]] 2024-09-12T13:05:20.945 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${COMPARE_PIDS[@]} 2024-09-12T13:05:20.945 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 42842 2024-09-12T13:05:20.946 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${COMPARE_PIDS[@]} 2024-09-12T13:05:20.946 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 42844 2024-09-12T13:05:20.946 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${COMPARE_PIDS[@]} 2024-09-12T13:05:20.946 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 42846 2024-09-12T13:05:20.946 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${COMPARE_PIDS[@]} 2024-09-12T13:05:20.946 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 42848 2024-09-12T13:05:20.946 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${COMPARE_PIDS[@]} 2024-09-12T13:05:20.946 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 42849 2024-09-12T13:05:20.946 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${COMPARE_PIDS[@]} 2024-09-12T13:05:20.947 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 42852 2024-09-12T13:05:20.947 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${COMPARE_PIDS[@]} 2024-09-12T13:05:20.947 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 42853 2024-09-12T13:05:22.239 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ promote_md5=5fa22ff55b5cbbd7ac3ddea465d7059c 2024-09-12T13:05:22.239 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo rbd --cluster cluster2 device unmap -t krbd /dev/rbd8 2024-09-12T13:05:22.344 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ 5fa22ff55b5cbbd7ac3ddea465d7059c != \5\f\a\2\2\f\f\5\5\b\5\c\b\b\d\7\a\c\3\d\d\e\a\4\6\5\d\7\0\5\9\c ]] 2024-09-12T13:05:22.346 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ (( RET != 0 )) 2024-09-12T13:05:22.346 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:05:22.346 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ IMG=image-primary1 2024-09-12T13:05:22.346 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_replaying_status_in_pool_dir cluster1 mirror image-primary1 2024-09-12T13:05:22.346 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:05:22.346 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:05:22.346 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:05:22.346 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' snapshot = journal ']' 2024-09-12T13:05:22.346 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary1 up+replaying 2024-09-12T13:05:22.346 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:05:22.346 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:05:22.346 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:05:22.347 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:05:22.347 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:05:22.347 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:05:22.347 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:05:22.347 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:05:23.362 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary1 up+replaying '' '' 2024-09-12T13:05:23.363 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:05:23.363 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:05:23.363 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:05:23.363 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:05:23.363 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:05:23.363 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:05:23.363 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:05:23.363 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:05:23.363 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary1 2024-09-12T13:05:23.444 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary1: 2024-09-12T13:05:23.444 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 5a3f10b5-2891-464a-a2f2-921068cdf6e7 2024-09-12T13:05:23.444 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+replaying 2024-09-12T13:05:23.444 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: replaying, {"bytes_per_second":3260416.0,"bytes_per_snapshot":65208320.0,"last_snapshot_bytes":65208320,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726146264,"remote_snapshot_timestamp":1726146264,"replay_state":"idle"} 2024-09-12T13:05:23.444 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:05:23.445 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:05:17' 2024-09-12T13:05:23.445 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+replaying' 2024-09-12T13:05:23.447 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+replaying 2024-09-12T13:05:23.447 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:05:23.449 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: replaying, {"bytes_per_second":3260416.0,"bytes_per_snapshot":65208320.0,"last_snapshot_bytes":65208320,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726146264,"remote_snapshot_timestamp":1726146264,"replay_state":"idle"} 2024-09-12T13:05:23.450 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:05:23.450 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+replaying 2024-09-12T13:05:23.451 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:05:23.453 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+replaying 2024-09-12T13:05:23.453 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:05:23.456 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:05:23.456 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:05:23.457 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:05:23.461 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:05:17' 2024-09-12T13:05:23.461 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary1 up+replaying '2024-09-12 13:05:17' 2024-09-12T13:05:23.461 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:05:23.461 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:05:23.461 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:05:23.461 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:05:23.461 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:05:17' 2024-09-12T13:05:23.461 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:05:23.465 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:05:23.465 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-09-12T13:05:23.566 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041934232mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary15a3f10b5-2891-464a-a2f2-921068cdf6e7up+replayingreplaying, {"bytes_per_second":3260416.0,"bytes_per_snapshot":65208320.0,"last_snapshot_bytes":65208320,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726146264,"remote_snapshot_timestamp":1726146264,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:05:17image-primary10e8086552-9fee-4a93-b94f-d2d6dc6bff63up+replayingreplaying, {"bytes_per_second":3165184.0,"bytes_per_snapshot":63303680.0,"last_snapshot_bytes":63303680,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726146266,"remote_snapshot_timestamp":1726146266,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:05:17image-primary2b7ebe2f2-cab7-4e57-a03a-3091186a2999up+replayingreplaying, {"bytes_per_second":3351347.2,"bytes_per_snapshot":67026944.0,"last_snapshot_bytes":67026944,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726146267,"remote_snapshot_timestamp":1726146267,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:05:17image-primary3c469479f-c19b-4534-a75c-ca68f8c28929up+replayingreplaying, {"bytes_per_second":3364454.4,"bytes_per_snapshot":67289088.0,"last_snapshot_bytes":67289088,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726146266,"remote_snapshot_timestamp":1726146266,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:05:17image-primary4f5942d86-af5f-4ca1-8ebc-da4cbe6dc314up+replayingreplaying, {"bytes_per_second":3322265.6,"bytes_per_snapshot":66445312.0,"last_snapshot_bytes":66445312,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726146264,"remote_snapshot_timestamp":1726146264,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:05:17image-primary58b95d658-5f20-45ab-b663-60abf1f860dbup+replayingreplaying, {"bytes_per_second":3260825.6,"bytes_per_snapshot":65216512.0,"last_snapshot_bytes":65216512,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726146264,"remote_snapshot_timestamp":1726146264,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:05:17image-primary6a790a319-cd27-4a01-b342-b61db0aa2b94up+replayingreplaying, {"bytes_per_second":3424870.4,"bytes_per_snapshot":68497408.0,"last_snapshot_bytes":68497408,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726146259,"remote_snapshot_timestamp":1726146259,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:05:17image-primary759cd13a4-5c88-4a91-a100-f622774b37f9up+replayingreplaying, {"bytes_per_second":3338444.8,"bytes_per_snapshot":66768896.0,"last_snapshot_bytes":66768896,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726146264,"remote_snapshot_timestamp":1726146264,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:05:17image-primary8e5807591-ea6d-4541-b1c3-0785c5e797baup+replayingreplaying, {"bytes_per_second":3327590.4,"bytes_per_snapshot":66551808.0,"last_snapshot_bytes":66551808,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726146263,"remote_snapshot_timestamp":1726146263,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:05:17image-primary97d07d66e-fe3f-46b2-83dc-67fa35b43f0eup+replayingreplaying, {"bytes_per_second":3346227.2,"bytes_per_snapshot":66924544.0,"last_snapshot_bytes":66924544,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726146264,"remote_snapshot_timestamp":1726146264,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:05:17' 2024-09-12T13:05:23.566 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:05:23.566 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary1'\'']/last_update' 2024-09-12T13:05:23.604 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:05:17' 2024-09-12T13:05:23.605 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary1'\'']/state' 2024-09-12T13:05:23.609 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+replaying 2024-09-12T13:05:23.609 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+replaying 2024-09-12T13:05:23.609 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+replaying 2024-09-12T13:05:23.611 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+replaying 2024-09-12T13:05:23.612 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:05:23.612 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:05:23.612 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ remove_image cluster2 mirror image-primary1 2024-09-12T13:05:23.612 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:05:23.612 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:05:23.613 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:05:23.613 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 snap purge mirror/image-primary1 2024-09-12T13:05:23.672 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 rm mirror/image-primary1 2024-09-12T13:05:23.798 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:05:23.798+0000 7f67e4800640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x560555bf5040 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-09-12T13:05:23.799 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:05:23.798+0000 7f67ed400640 -1 rbd::mirror::ImageReplayer: 0x5605561ac000 [3/5a3f10b5-2891-464a-a2f2-921068cdf6e7] operator(): start failed: (85) Interrupted system call should be restarted 2024-09-12T13:05:24.982 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-09-12T13:05:24.988 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_image_removal cluster1 mirror image-primary1 2024-09-12T13:05:24.988 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:05:24.988 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:05:24.988 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:05:24.988 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:05:24.989 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 ls mirror 2024-09-12T13:05:24.989 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep -wq image-primary1 2024-09-12T13:05:25.039 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:05:25.039 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rm -rf test-primary1 2024-09-12T13:05:25.052 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:05:25.052 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ IMG=image-primary2 2024-09-12T13:05:25.052 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_replaying_status_in_pool_dir cluster1 mirror image-primary2 2024-09-12T13:05:25.052 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:05:25.052 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:05:25.052 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:05:25.052 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' snapshot = journal ']' 2024-09-12T13:05:25.053 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary2 up+replaying 2024-09-12T13:05:25.053 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:05:25.053 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:05:25.053 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:05:25.053 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:05:25.053 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:05:25.053 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:05:25.053 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:05:25.053 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:05:26.054 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary2 up+replaying '' '' 2024-09-12T13:05:26.054 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:05:26.054 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:05:26.054 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:05:26.054 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:05:26.054 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:05:26.054 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:05:26.055 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:05:26.055 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:05:26.055 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary2 2024-09-12T13:05:26.131 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary2: 2024-09-12T13:05:26.131 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: b7ebe2f2-cab7-4e57-a03a-3091186a2999 2024-09-12T13:05:26.131 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+replaying 2024-09-12T13:05:26.131 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: replaying, {"bytes_per_second":3351347.2,"bytes_per_snapshot":67026944.0,"last_snapshot_bytes":67026944,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726146267,"remote_snapshot_timestamp":1726146267,"replay_state":"idle"} 2024-09-12T13:05:26.131 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:05:26.131 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:05:17' 2024-09-12T13:05:26.132 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+replaying' 2024-09-12T13:05:26.133 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+replaying 2024-09-12T13:05:26.134 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:05:26.136 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: replaying, {"bytes_per_second":3351347.2,"bytes_per_snapshot":67026944.0,"last_snapshot_bytes":67026944,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726146267,"remote_snapshot_timestamp":1726146267,"replay_state":"idle"} 2024-09-12T13:05:26.137 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:05:26.137 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+replaying 2024-09-12T13:05:26.137 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:05:26.139 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+replaying 2024-09-12T13:05:26.140 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:05:26.142 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:05:26.142 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:05:26.143 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:05:26.147 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:05:17' 2024-09-12T13:05:26.147 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary2 up+replaying '2024-09-12 13:05:17' 2024-09-12T13:05:26.147 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:05:26.147 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:05:26.147 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:05:26.147 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:05:26.147 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:05:17' 2024-09-12T13:05:26.148 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:05:26.148 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:05:26.148 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-09-12T13:05:26.249 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING941934232mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary10e8086552-9fee-4a93-b94f-d2d6dc6bff63up+replayingreplaying, {"bytes_per_second":3165184.0,"bytes_per_snapshot":63303680.0,"last_snapshot_bytes":63303680,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726146266,"remote_snapshot_timestamp":1726146266,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:05:17image-primary2b7ebe2f2-cab7-4e57-a03a-3091186a2999up+replayingreplaying, {"bytes_per_second":3351347.2,"bytes_per_snapshot":67026944.0,"last_snapshot_bytes":67026944,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726146267,"remote_snapshot_timestamp":1726146267,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:05:17image-primary3c469479f-c19b-4534-a75c-ca68f8c28929up+replayingreplaying, {"bytes_per_second":3364454.4,"bytes_per_snapshot":67289088.0,"last_snapshot_bytes":67289088,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726146266,"remote_snapshot_timestamp":1726146266,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:05:17image-primary4f5942d86-af5f-4ca1-8ebc-da4cbe6dc314up+replayingreplaying, {"bytes_per_second":3322265.6,"bytes_per_snapshot":66445312.0,"last_snapshot_bytes":66445312,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726146264,"remote_snapshot_timestamp":1726146264,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:05:17image-primary58b95d658-5f20-45ab-b663-60abf1f860dbup+replayingreplaying, {"bytes_per_second":3260825.6,"bytes_per_snapshot":65216512.0,"last_snapshot_bytes":65216512,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726146264,"remote_snapshot_timestamp":1726146264,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:05:17image-primary6a790a319-cd27-4a01-b342-b61db0aa2b94up+replayingreplaying, {"bytes_per_second":3424870.4,"bytes_per_snapshot":68497408.0,"last_snapshot_bytes":68497408,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726146259,"remote_snapshot_timestamp":1726146259,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:05:17image-primary759cd13a4-5c88-4a91-a100-f622774b37f9up+replayingreplaying, {"bytes_per_second":3338444.8,"bytes_per_snapshot":66768896.0,"last_snapshot_bytes":66768896,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726146264,"remote_snapshot_timestamp":1726146264,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:05:17image-primary8e5807591-ea6d-4541-b1c3-0785c5e797baup+replayingreplaying, {"bytes_per_second":3327590.4,"bytes_per_snapshot":66551808.0,"last_snapshot_bytes":66551808,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726146263,"remote_snapshot_timestamp":1726146263,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:05:17image-primary97d07d66e-fe3f-46b2-83dc-67fa35b43f0eup+replayingreplaying, {"bytes_per_second":3346227.2,"bytes_per_snapshot":66924544.0,"last_snapshot_bytes":66924544,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726146264,"remote_snapshot_timestamp":1726146264,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:05:17' 2024-09-12T13:05:26.249 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:05:26.249 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary2'\'']/last_update' 2024-09-12T13:05:26.253 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:05:17' 2024-09-12T13:05:26.254 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary2'\'']/state' 2024-09-12T13:05:26.257 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+replaying 2024-09-12T13:05:26.258 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+replaying 2024-09-12T13:05:26.258 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+replaying 2024-09-12T13:05:26.260 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+replaying 2024-09-12T13:05:26.261 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:05:26.261 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:05:26.261 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ remove_image cluster2 mirror image-primary2 2024-09-12T13:05:26.261 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:05:26.261 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:05:26.261 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:05:26.262 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 snap purge mirror/image-primary2 2024-09-12T13:05:26.320 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 rm mirror/image-primary2 2024-09-12T13:05:26.452 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:05:26.452+0000 7f67e5200640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x560555a03380 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-09-12T13:05:26.453 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:05:26.452+0000 7f67ed400640 -1 rbd::mirror::ImageReplayer: 0x5605561ac500 [3/b7ebe2f2-cab7-4e57-a03a-3091186a2999] operator(): start failed: (85) Interrupted system call should be restarted 2024-09-12T13:05:28.002 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-09-12T13:05:28.008 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_image_removal cluster1 mirror image-primary2 2024-09-12T13:05:28.008 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:05:28.008 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:05:28.008 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:05:28.008 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:05:28.009 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 ls mirror 2024-09-12T13:05:28.009 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep -wq image-primary2 2024-09-12T13:05:28.059 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:05:28.060 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rm -rf test-primary2 2024-09-12T13:05:28.062 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:05:28.062 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ IMG=image-primary3 2024-09-12T13:05:28.062 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_replaying_status_in_pool_dir cluster1 mirror image-primary3 2024-09-12T13:05:28.062 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:05:28.062 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:05:28.062 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:05:28.062 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' snapshot = journal ']' 2024-09-12T13:05:28.062 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary3 up+replaying 2024-09-12T13:05:28.062 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:05:28.063 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:05:28.063 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:05:28.063 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:05:28.063 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:05:28.063 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:05:28.063 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:05:28.063 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:05:29.064 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary3 up+replaying '' '' 2024-09-12T13:05:29.064 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:05:29.064 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:05:29.065 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:05:29.065 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:05:29.065 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:05:29.065 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:05:29.065 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:05:29.065 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:05:29.065 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary3 2024-09-12T13:05:29.140 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary3: 2024-09-12T13:05:29.141 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: c469479f-c19b-4534-a75c-ca68f8c28929 2024-09-12T13:05:29.141 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+replaying 2024-09-12T13:05:29.141 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: replaying, {"bytes_per_second":3364454.4,"bytes_per_snapshot":67289088.0,"last_snapshot_bytes":67289088,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726146266,"remote_snapshot_timestamp":1726146266,"replay_state":"idle"} 2024-09-12T13:05:29.141 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:05:29.141 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:05:17' 2024-09-12T13:05:29.141 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+replaying' 2024-09-12T13:05:29.143 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+replaying 2024-09-12T13:05:29.143 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:05:29.145 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: replaying, {"bytes_per_second":3364454.4,"bytes_per_snapshot":67289088.0,"last_snapshot_bytes":67289088,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726146266,"remote_snapshot_timestamp":1726146266,"replay_state":"idle"} 2024-09-12T13:05:29.146 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:05:29.147 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+replaying 2024-09-12T13:05:29.147 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:05:29.149 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+replaying 2024-09-12T13:05:29.149 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:05:29.151 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:05:29.152 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:05:29.153 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:05:29.156 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:05:17' 2024-09-12T13:05:29.156 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary3 up+replaying '2024-09-12 13:05:17' 2024-09-12T13:05:29.156 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:05:29.157 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:05:29.157 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:05:29.157 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:05:29.157 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:05:17' 2024-09-12T13:05:29.157 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:05:29.157 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:05:29.157 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-09-12T13:05:29.256 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING841934232mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary10e8086552-9fee-4a93-b94f-d2d6dc6bff63up+replayingreplaying, {"bytes_per_second":3165184.0,"bytes_per_snapshot":63303680.0,"last_snapshot_bytes":63303680,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726146266,"remote_snapshot_timestamp":1726146266,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:05:17image-primary3c469479f-c19b-4534-a75c-ca68f8c28929up+replayingreplaying, {"bytes_per_second":3364454.4,"bytes_per_snapshot":67289088.0,"last_snapshot_bytes":67289088,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726146266,"remote_snapshot_timestamp":1726146266,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:05:17image-primary4f5942d86-af5f-4ca1-8ebc-da4cbe6dc314up+replayingreplaying, {"bytes_per_second":3322265.6,"bytes_per_snapshot":66445312.0,"last_snapshot_bytes":66445312,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726146264,"remote_snapshot_timestamp":1726146264,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:05:17image-primary58b95d658-5f20-45ab-b663-60abf1f860dbup+replayingreplaying, {"bytes_per_second":3260825.6,"bytes_per_snapshot":65216512.0,"last_snapshot_bytes":65216512,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726146264,"remote_snapshot_timestamp":1726146264,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:05:17image-primary6a790a319-cd27-4a01-b342-b61db0aa2b94up+replayingreplaying, {"bytes_per_second":3424870.4,"bytes_per_snapshot":68497408.0,"last_snapshot_bytes":68497408,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726146259,"remote_snapshot_timestamp":1726146259,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:05:17image-primary759cd13a4-5c88-4a91-a100-f622774b37f9up+replayingreplaying, {"bytes_per_second":3338444.8,"bytes_per_snapshot":66768896.0,"last_snapshot_bytes":66768896,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726146264,"remote_snapshot_timestamp":1726146264,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:05:17image-primary8e5807591-ea6d-4541-b1c3-0785c5e797baup+replayingreplaying, {"bytes_per_second":3327590.4,"bytes_per_snapshot":66551808.0,"last_snapshot_bytes":66551808,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726146263,"remote_snapshot_timestamp":1726146263,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:05:17image-primary97d07d66e-fe3f-46b2-83dc-67fa35b43f0eup+replayingreplaying, {"bytes_per_second":3346227.2,"bytes_per_snapshot":66924544.0,"last_snapshot_bytes":66924544,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726146264,"remote_snapshot_timestamp":1726146264,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:05:17' 2024-09-12T13:05:29.257 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:05:29.257 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary3'\'']/last_update' 2024-09-12T13:05:29.261 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:05:17' 2024-09-12T13:05:29.261 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary3'\'']/state' 2024-09-12T13:05:29.265 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+replaying 2024-09-12T13:05:29.266 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+replaying 2024-09-12T13:05:29.266 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+replaying 2024-09-12T13:05:29.268 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+replaying 2024-09-12T13:05:29.269 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:05:29.269 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:05:29.269 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ remove_image cluster2 mirror image-primary3 2024-09-12T13:05:29.269 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:05:29.269 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:05:29.269 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:05:29.269 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 snap purge mirror/image-primary3 2024-09-12T13:05:29.328 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 rm mirror/image-primary3 2024-09-12T13:05:29.457 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:05:29.457+0000 7f67e5200640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x560555a03a00 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-09-12T13:05:29.457 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:05:29.457+0000 7f67ed400640 -1 rbd::mirror::ImageReplayer: 0x5605561ac780 [3/c469479f-c19b-4534-a75c-ca68f8c28929] operator(): start failed: (85) Interrupted system call should be restarted 2024-09-12T13:05:30.809 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-09-12T13:05:30.816 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_image_removal cluster1 mirror image-primary3 2024-09-12T13:05:30.816 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:05:30.816 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:05:30.817 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:05:30.817 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:05:30.817 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 ls mirror 2024-09-12T13:05:30.817 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep -wq image-primary3 2024-09-12T13:05:30.866 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:05:30.866 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rm -rf test-primary3 2024-09-12T13:05:30.869 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:05:30.869 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ IMG=image-primary4 2024-09-12T13:05:30.869 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_replaying_status_in_pool_dir cluster1 mirror image-primary4 2024-09-12T13:05:30.869 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:05:30.869 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:05:30.869 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:05:30.869 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' snapshot = journal ']' 2024-09-12T13:05:30.869 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary4 up+replaying 2024-09-12T13:05:30.869 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:05:30.869 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:05:30.869 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:05:30.869 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:05:30.870 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:05:30.870 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:05:30.870 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:05:30.870 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:05:31.872 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary4 up+replaying '' '' 2024-09-12T13:05:31.872 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:05:31.872 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:05:31.872 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:05:31.873 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:05:31.873 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:05:31.873 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:05:31.873 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:05:31.873 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:05:31.873 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary4 2024-09-12T13:05:31.953 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary4: 2024-09-12T13:05:31.953 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: f5942d86-af5f-4ca1-8ebc-da4cbe6dc314 2024-09-12T13:05:31.953 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+replaying 2024-09-12T13:05:31.953 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: replaying, {"bytes_per_second":3322265.6,"bytes_per_snapshot":66445312.0,"last_snapshot_bytes":66445312,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726146264,"remote_snapshot_timestamp":1726146264,"replay_state":"idle"} 2024-09-12T13:05:31.953 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:05:31.953 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:05:17' 2024-09-12T13:05:31.953 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+replaying' 2024-09-12T13:05:31.955 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+replaying 2024-09-12T13:05:31.955 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:05:31.958 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: replaying, {"bytes_per_second":3322265.6,"bytes_per_snapshot":66445312.0,"last_snapshot_bytes":66445312,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726146264,"remote_snapshot_timestamp":1726146264,"replay_state":"idle"} 2024-09-12T13:05:31.958 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:05:31.959 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+replaying 2024-09-12T13:05:31.959 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:05:31.961 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+replaying 2024-09-12T13:05:31.962 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:05:31.964 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:05:31.965 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:05:31.965 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:05:31.969 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:05:17' 2024-09-12T13:05:31.970 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary4 up+replaying '2024-09-12 13:05:17' 2024-09-12T13:05:31.970 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:05:31.970 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:05:31.971 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:05:31.971 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:05:31.971 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:05:17' 2024-09-12T13:05:31.972 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:05:31.972 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:05:31.972 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-09-12T13:05:32.069 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING741934232mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary10e8086552-9fee-4a93-b94f-d2d6dc6bff63up+replayingreplaying, {"bytes_per_second":3165184.0,"bytes_per_snapshot":63303680.0,"last_snapshot_bytes":63303680,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726146266,"remote_snapshot_timestamp":1726146266,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:05:17image-primary4f5942d86-af5f-4ca1-8ebc-da4cbe6dc314up+replayingreplaying, {"bytes_per_second":3322265.6,"bytes_per_snapshot":66445312.0,"last_snapshot_bytes":66445312,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726146264,"remote_snapshot_timestamp":1726146264,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:05:17image-primary58b95d658-5f20-45ab-b663-60abf1f860dbup+replayingreplaying, {"bytes_per_second":3260825.6,"bytes_per_snapshot":65216512.0,"last_snapshot_bytes":65216512,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726146264,"remote_snapshot_timestamp":1726146264,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:05:17image-primary6a790a319-cd27-4a01-b342-b61db0aa2b94up+replayingreplaying, {"bytes_per_second":3424870.4,"bytes_per_snapshot":68497408.0,"last_snapshot_bytes":68497408,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726146259,"remote_snapshot_timestamp":1726146259,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:05:17image-primary759cd13a4-5c88-4a91-a100-f622774b37f9up+replayingreplaying, {"bytes_per_second":3338444.8,"bytes_per_snapshot":66768896.0,"last_snapshot_bytes":66768896,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726146264,"remote_snapshot_timestamp":1726146264,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:05:17image-primary8e5807591-ea6d-4541-b1c3-0785c5e797baup+replayingreplaying, {"bytes_per_second":3327590.4,"bytes_per_snapshot":66551808.0,"last_snapshot_bytes":66551808,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726146263,"remote_snapshot_timestamp":1726146263,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:05:17image-primary97d07d66e-fe3f-46b2-83dc-67fa35b43f0eup+replayingreplaying, {"bytes_per_second":3346227.2,"bytes_per_snapshot":66924544.0,"last_snapshot_bytes":66924544,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726146264,"remote_snapshot_timestamp":1726146264,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:05:17' 2024-09-12T13:05:32.070 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:05:32.070 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary4'\'']/last_update' 2024-09-12T13:05:32.074 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:05:17' 2024-09-12T13:05:32.075 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary4'\'']/state' 2024-09-12T13:05:32.079 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+replaying 2024-09-12T13:05:32.079 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+replaying 2024-09-12T13:05:32.079 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+replaying 2024-09-12T13:05:32.081 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+replaying 2024-09-12T13:05:32.082 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:05:32.082 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:05:32.082 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ remove_image cluster2 mirror image-primary4 2024-09-12T13:05:32.082 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:05:32.082 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:05:32.082 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:05:32.083 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 snap purge mirror/image-primary4 2024-09-12T13:05:32.141 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 rm mirror/image-primary4 2024-09-12T13:05:32.270 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:05:32.271+0000 7f67e4800640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x560555bf51e0 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-09-12T13:05:32.270 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:05:32.271+0000 7f67eca00640 -1 rbd::mirror::ImageReplayer: 0x5605561acc80 [3/f5942d86-af5f-4ca1-8ebc-da4cbe6dc314] operator(): start failed: (85) Interrupted system call should be restarted 2024-09-12T13:05:33.831 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-09-12T13:05:33.837 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_image_removal cluster1 mirror image-primary4 2024-09-12T13:05:33.837 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:05:33.837 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:05:33.837 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:05:33.837 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:05:33.837 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 ls mirror 2024-09-12T13:05:33.837 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep -wq image-primary4 2024-09-12T13:05:33.887 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:05:33.887 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rm -rf test-primary4 2024-09-12T13:05:33.889 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:05:33.889 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ IMG=image-primary5 2024-09-12T13:05:33.890 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_replaying_status_in_pool_dir cluster1 mirror image-primary5 2024-09-12T13:05:33.890 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:05:33.890 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:05:33.890 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:05:33.890 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' snapshot = journal ']' 2024-09-12T13:05:33.890 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary5 up+replaying 2024-09-12T13:05:33.890 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:05:33.890 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:05:33.890 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:05:33.890 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:05:33.890 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:05:33.890 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:05:33.890 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:05:33.890 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:05:34.891 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary5 up+replaying '' '' 2024-09-12T13:05:34.892 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:05:34.892 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:05:34.892 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:05:34.892 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:05:34.892 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:05:34.892 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:05:34.892 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:05:34.892 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:05:34.893 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary5 2024-09-12T13:05:34.967 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary5: 2024-09-12T13:05:34.967 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 8b95d658-5f20-45ab-b663-60abf1f860db 2024-09-12T13:05:34.967 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+replaying 2024-09-12T13:05:34.967 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: replaying, {"bytes_per_second":3260825.6,"bytes_per_snapshot":65216512.0,"last_snapshot_bytes":65216512,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726146264,"remote_snapshot_timestamp":1726146264,"replay_state":"idle"} 2024-09-12T13:05:34.968 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:05:34.968 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:05:17' 2024-09-12T13:05:34.968 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+replaying' 2024-09-12T13:05:34.970 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+replaying 2024-09-12T13:05:34.970 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:05:34.972 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: replaying, {"bytes_per_second":3260825.6,"bytes_per_snapshot":65216512.0,"last_snapshot_bytes":65216512,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726146264,"remote_snapshot_timestamp":1726146264,"replay_state":"idle"} 2024-09-12T13:05:34.973 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:05:34.973 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+replaying 2024-09-12T13:05:34.974 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:05:34.975 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+replaying 2024-09-12T13:05:34.976 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:05:34.978 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:05:34.979 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:05:34.979 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:05:34.983 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:05:17' 2024-09-12T13:05:34.983 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary5 up+replaying '2024-09-12 13:05:17' 2024-09-12T13:05:34.983 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:05:34.984 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:05:34.984 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:05:34.984 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:05:34.984 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:05:17' 2024-09-12T13:05:34.984 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:05:34.984 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:05:34.984 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-09-12T13:05:35.074 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING641934232mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary10e8086552-9fee-4a93-b94f-d2d6dc6bff63up+replayingreplaying, {"bytes_per_second":3165184.0,"bytes_per_snapshot":63303680.0,"last_snapshot_bytes":63303680,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726146266,"remote_snapshot_timestamp":1726146266,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:05:17image-primary58b95d658-5f20-45ab-b663-60abf1f860dbup+replayingreplaying, {"bytes_per_second":3260825.6,"bytes_per_snapshot":65216512.0,"last_snapshot_bytes":65216512,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726146264,"remote_snapshot_timestamp":1726146264,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:05:17image-primary6a790a319-cd27-4a01-b342-b61db0aa2b94up+replayingreplaying, {"bytes_per_second":3424870.4,"bytes_per_snapshot":68497408.0,"last_snapshot_bytes":68497408,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726146259,"remote_snapshot_timestamp":1726146259,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:05:17image-primary759cd13a4-5c88-4a91-a100-f622774b37f9up+replayingreplaying, {"bytes_per_second":3338444.8,"bytes_per_snapshot":66768896.0,"last_snapshot_bytes":66768896,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726146264,"remote_snapshot_timestamp":1726146264,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:05:17image-primary8e5807591-ea6d-4541-b1c3-0785c5e797baup+replayingreplaying, {"bytes_per_second":3327590.4,"bytes_per_snapshot":66551808.0,"last_snapshot_bytes":66551808,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726146263,"remote_snapshot_timestamp":1726146263,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:05:17image-primary97d07d66e-fe3f-46b2-83dc-67fa35b43f0eup+replayingreplaying, {"bytes_per_second":3346227.2,"bytes_per_snapshot":66924544.0,"last_snapshot_bytes":66924544,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726146264,"remote_snapshot_timestamp":1726146264,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:05:17' 2024-09-12T13:05:35.075 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:05:35.075 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary5'\'']/last_update' 2024-09-12T13:05:35.079 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:05:17' 2024-09-12T13:05:35.080 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary5'\'']/state' 2024-09-12T13:05:35.083 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+replaying 2024-09-12T13:05:35.084 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+replaying 2024-09-12T13:05:35.084 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+replaying 2024-09-12T13:05:35.086 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+replaying 2024-09-12T13:05:35.087 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:05:35.087 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:05:35.087 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ remove_image cluster2 mirror image-primary5 2024-09-12T13:05:35.087 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:05:35.087 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:05:35.087 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:05:35.087 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 snap purge mirror/image-primary5 2024-09-12T13:05:35.164 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 rm mirror/image-primary5 2024-09-12T13:05:35.290 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:05:35.290+0000 7f67e5200640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x560555bf5380 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-09-12T13:05:35.290 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:05:35.290+0000 7f67ed400640 -1 rbd::mirror::ImageReplayer: 0x5605561aca00 [3/8b95d658-5f20-45ab-b663-60abf1f860db] operator(): start failed: (85) Interrupted system call should be restarted 2024-09-12T13:05:36.836 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-09-12T13:05:36.842 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_image_removal cluster1 mirror image-primary5 2024-09-12T13:05:36.842 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:05:36.842 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:05:36.842 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:05:36.842 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:05:36.843 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 ls mirror 2024-09-12T13:05:36.843 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep -wq image-primary5 2024-09-12T13:05:36.892 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:05:36.892 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rm -rf test-primary5 2024-09-12T13:05:36.894 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:05:36.895 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ IMG=image-primary6 2024-09-12T13:05:36.895 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_replaying_status_in_pool_dir cluster1 mirror image-primary6 2024-09-12T13:05:36.895 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:05:36.895 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:05:36.895 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:05:36.895 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' snapshot = journal ']' 2024-09-12T13:05:36.895 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary6 up+replaying 2024-09-12T13:05:36.895 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:05:36.895 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:05:36.895 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:05:36.895 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:05:36.895 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:05:36.895 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:05:36.896 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:05:36.896 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:05:37.897 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary6 up+replaying '' '' 2024-09-12T13:05:37.897 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:05:37.897 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:05:37.897 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:05:37.897 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:05:37.897 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:05:37.897 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:05:37.897 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:05:37.898 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:05:37.898 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary6 2024-09-12T13:05:38.003 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary6: 2024-09-12T13:05:38.003 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: a790a319-cd27-4a01-b342-b61db0aa2b94 2024-09-12T13:05:38.004 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+replaying 2024-09-12T13:05:38.004 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: replaying, {"bytes_per_second":3424870.4,"bytes_per_snapshot":68497408.0,"last_snapshot_bytes":68497408,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726146259,"remote_snapshot_timestamp":1726146259,"replay_state":"idle"} 2024-09-12T13:05:38.004 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:05:38.004 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:05:17' 2024-09-12T13:05:38.004 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+replaying' 2024-09-12T13:05:38.006 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+replaying 2024-09-12T13:05:38.006 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:05:38.008 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: replaying, {"bytes_per_second":3424870.4,"bytes_per_snapshot":68497408.0,"last_snapshot_bytes":68497408,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726146259,"remote_snapshot_timestamp":1726146259,"replay_state":"idle"} 2024-09-12T13:05:38.009 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:05:38.009 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+replaying 2024-09-12T13:05:38.010 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:05:38.013 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+replaying 2024-09-12T13:05:38.013 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:05:38.015 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:05:38.015 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:05:38.016 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:05:38.020 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:05:17' 2024-09-12T13:05:38.021 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary6 up+replaying '2024-09-12 13:05:17' 2024-09-12T13:05:38.021 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:05:38.021 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:05:38.021 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:05:38.021 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:05:38.021 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:05:17' 2024-09-12T13:05:38.021 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:05:38.021 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:05:38.022 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-09-12T13:05:38.142 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING541934232mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary10e8086552-9fee-4a93-b94f-d2d6dc6bff63up+replayingreplaying, {"bytes_per_second":3165184.0,"bytes_per_snapshot":63303680.0,"last_snapshot_bytes":63303680,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726146266,"remote_snapshot_timestamp":1726146266,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:05:17image-primary6a790a319-cd27-4a01-b342-b61db0aa2b94up+replayingreplaying, {"bytes_per_second":3424870.4,"bytes_per_snapshot":68497408.0,"last_snapshot_bytes":68497408,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726146259,"remote_snapshot_timestamp":1726146259,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:05:17image-primary759cd13a4-5c88-4a91-a100-f622774b37f9up+replayingreplaying, {"bytes_per_second":3338444.8,"bytes_per_snapshot":66768896.0,"last_snapshot_bytes":66768896,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726146264,"remote_snapshot_timestamp":1726146264,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:05:17image-primary8e5807591-ea6d-4541-b1c3-0785c5e797baup+replayingreplaying, {"bytes_per_second":3327590.4,"bytes_per_snapshot":66551808.0,"last_snapshot_bytes":66551808,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726146263,"remote_snapshot_timestamp":1726146263,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:05:17image-primary97d07d66e-fe3f-46b2-83dc-67fa35b43f0eup+replayingreplaying, {"bytes_per_second":3346227.2,"bytes_per_snapshot":66924544.0,"last_snapshot_bytes":66924544,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726146264,"remote_snapshot_timestamp":1726146264,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:05:17' 2024-09-12T13:05:38.142 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:05:38.142 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary6'\'']/last_update' 2024-09-12T13:05:38.146 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:05:17' 2024-09-12T13:05:38.147 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary6'\'']/state' 2024-09-12T13:05:38.151 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+replaying 2024-09-12T13:05:38.151 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+replaying 2024-09-12T13:05:38.152 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+replaying 2024-09-12T13:05:38.154 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+replaying 2024-09-12T13:05:38.155 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:05:38.155 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:05:38.155 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ remove_image cluster2 mirror image-primary6 2024-09-12T13:05:38.155 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:05:38.155 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:05:38.155 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:05:38.155 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 snap purge mirror/image-primary6 2024-09-12T13:05:38.217 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 rm mirror/image-primary6 2024-09-12T13:05:38.386 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:05:38.385+0000 7f67e4800640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x560555a05a00 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-09-12T13:05:38.386 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:05:38.385+0000 7f67ed400640 -1 rbd::mirror::ImageReplayer: 0x5605561ad180 [3/a790a319-cd27-4a01-b342-b61db0aa2b94] operator(): start failed: (85) Interrupted system call should be restarted 2024-09-12T13:05:39.810 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-09-12T13:05:39.815 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_image_removal cluster1 mirror image-primary6 2024-09-12T13:05:39.816 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:05:39.816 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:05:39.816 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:05:39.816 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:05:39.816 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 ls mirror 2024-09-12T13:05:39.817 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep -wq image-primary6 2024-09-12T13:05:39.865 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:05:39.865 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rm -rf test-primary6 2024-09-12T13:05:39.867 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:05:39.868 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ IMG=image-primary7 2024-09-12T13:05:39.868 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_replaying_status_in_pool_dir cluster1 mirror image-primary7 2024-09-12T13:05:39.868 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:05:39.868 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:05:39.868 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:05:39.868 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' snapshot = journal ']' 2024-09-12T13:05:39.868 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary7 up+replaying 2024-09-12T13:05:39.868 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:05:39.868 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:05:39.868 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:05:39.868 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:05:39.868 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:05:39.868 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:05:39.869 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:05:39.869 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:05:40.870 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary7 up+replaying '' '' 2024-09-12T13:05:40.870 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:05:40.870 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:05:40.870 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:05:40.870 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:05:40.870 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:05:40.870 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:05:40.870 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:05:40.871 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:05:40.871 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary7 2024-09-12T13:05:40.950 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary7: 2024-09-12T13:05:40.950 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 59cd13a4-5c88-4a91-a100-f622774b37f9 2024-09-12T13:05:40.950 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+replaying 2024-09-12T13:05:40.950 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: replaying, {"bytes_per_second":3338444.8,"bytes_per_snapshot":66768896.0,"last_snapshot_bytes":66768896,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726146264,"remote_snapshot_timestamp":1726146264,"replay_state":"idle"} 2024-09-12T13:05:40.950 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:05:40.950 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:05:17' 2024-09-12T13:05:40.950 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+replaying' 2024-09-12T13:05:40.952 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+replaying 2024-09-12T13:05:40.953 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:05:40.955 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: replaying, {"bytes_per_second":3338444.8,"bytes_per_snapshot":66768896.0,"last_snapshot_bytes":66768896,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726146264,"remote_snapshot_timestamp":1726146264,"replay_state":"idle"} 2024-09-12T13:05:40.956 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:05:40.956 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+replaying 2024-09-12T13:05:40.957 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:05:40.959 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+replaying 2024-09-12T13:05:40.959 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:05:40.962 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:05:40.962 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:05:40.963 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:05:40.967 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:05:17' 2024-09-12T13:05:40.967 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary7 up+replaying '2024-09-12 13:05:17' 2024-09-12T13:05:40.968 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:05:40.968 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:05:40.968 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:05:40.968 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:05:40.968 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:05:17' 2024-09-12T13:05:40.968 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:05:40.968 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:05:40.969 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-09-12T13:05:41.055 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING441934232mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary10e8086552-9fee-4a93-b94f-d2d6dc6bff63up+replayingreplaying, {"bytes_per_second":3165184.0,"bytes_per_snapshot":63303680.0,"last_snapshot_bytes":63303680,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726146266,"remote_snapshot_timestamp":1726146266,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:05:17image-primary759cd13a4-5c88-4a91-a100-f622774b37f9up+replayingreplaying, {"bytes_per_second":3338444.8,"bytes_per_snapshot":66768896.0,"last_snapshot_bytes":66768896,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726146264,"remote_snapshot_timestamp":1726146264,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:05:17image-primary8e5807591-ea6d-4541-b1c3-0785c5e797baup+replayingreplaying, {"bytes_per_second":3327590.4,"bytes_per_snapshot":66551808.0,"last_snapshot_bytes":66551808,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726146263,"remote_snapshot_timestamp":1726146263,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:05:17image-primary97d07d66e-fe3f-46b2-83dc-67fa35b43f0eup+replayingreplaying, {"bytes_per_second":3346227.2,"bytes_per_snapshot":66924544.0,"last_snapshot_bytes":66924544,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726146264,"remote_snapshot_timestamp":1726146264,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:05:17' 2024-09-12T13:05:41.055 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:05:41.055 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary7'\'']/last_update' 2024-09-12T13:05:41.060 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:05:17' 2024-09-12T13:05:41.061 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary7'\'']/state' 2024-09-12T13:05:41.065 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+replaying 2024-09-12T13:05:41.065 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+replaying 2024-09-12T13:05:41.065 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+replaying 2024-09-12T13:05:41.067 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+replaying 2024-09-12T13:05:41.067 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:05:41.067 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:05:41.068 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ remove_image cluster2 mirror image-primary7 2024-09-12T13:05:41.068 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:05:41.068 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:05:41.068 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:05:41.068 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 snap purge mirror/image-primary7 2024-09-12T13:05:41.128 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 rm mirror/image-primary7 2024-09-12T13:05:41.252 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:05:41.252+0000 7f67e5200640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x560555bf5520 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-09-12T13:05:41.252 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:05:41.252+0000 7f67ed400640 -1 rbd::mirror::ImageReplayer: 0x5605561ad400 [3/59cd13a4-5c88-4a91-a100-f622774b37f9] operator(): start failed: (85) Interrupted system call should be restarted 2024-09-12T13:05:42.859 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-09-12T13:05:42.865 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_image_removal cluster1 mirror image-primary7 2024-09-12T13:05:42.865 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:05:42.865 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:05:42.865 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:05:42.865 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:05:42.866 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 ls mirror 2024-09-12T13:05:42.866 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep -wq image-primary7 2024-09-12T13:05:42.916 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:05:42.916 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rm -rf test-primary7 2024-09-12T13:05:42.919 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:05:42.919 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ IMG=image-primary8 2024-09-12T13:05:42.919 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_replaying_status_in_pool_dir cluster1 mirror image-primary8 2024-09-12T13:05:42.919 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:05:42.919 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:05:42.919 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:05:42.919 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' snapshot = journal ']' 2024-09-12T13:05:42.919 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary8 up+replaying 2024-09-12T13:05:42.919 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:05:42.919 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:05:42.919 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:05:42.919 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:05:42.920 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:05:42.920 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:05:42.920 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:05:42.920 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:05:43.921 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary8 up+replaying '' '' 2024-09-12T13:05:43.921 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:05:43.921 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:05:43.921 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:05:43.921 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:05:43.922 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:05:43.922 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:05:43.922 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:05:43.922 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:05:43.922 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary8 2024-09-12T13:05:44.001 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary8: 2024-09-12T13:05:44.001 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: e5807591-ea6d-4541-b1c3-0785c5e797ba 2024-09-12T13:05:44.001 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+replaying 2024-09-12T13:05:44.001 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: replaying, {"bytes_per_second":3327590.4,"bytes_per_snapshot":66551808.0,"last_snapshot_bytes":66551808,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726146263,"remote_snapshot_timestamp":1726146263,"replay_state":"idle"} 2024-09-12T13:05:44.002 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:05:44.002 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:05:17' 2024-09-12T13:05:44.002 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+replaying' 2024-09-12T13:05:44.004 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+replaying 2024-09-12T13:05:44.004 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:05:44.007 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: replaying, {"bytes_per_second":3327590.4,"bytes_per_snapshot":66551808.0,"last_snapshot_bytes":66551808,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726146263,"remote_snapshot_timestamp":1726146263,"replay_state":"idle"} 2024-09-12T13:05:44.007 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:05:44.008 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+replaying 2024-09-12T13:05:44.008 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:05:44.010 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+replaying 2024-09-12T13:05:44.010 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:05:44.013 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:05:44.013 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:05:44.014 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:05:44.018 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:05:17' 2024-09-12T13:05:44.018 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary8 up+replaying '2024-09-12 13:05:17' 2024-09-12T13:05:44.018 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:05:44.019 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:05:44.019 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:05:44.019 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:05:44.019 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:05:17' 2024-09-12T13:05:44.019 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:05:44.019 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:05:44.019 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-09-12T13:05:44.102 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING341934232mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary10e8086552-9fee-4a93-b94f-d2d6dc6bff63up+replayingreplaying, {"bytes_per_second":3165184.0,"bytes_per_snapshot":63303680.0,"last_snapshot_bytes":63303680,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726146266,"remote_snapshot_timestamp":1726146266,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:05:17image-primary8e5807591-ea6d-4541-b1c3-0785c5e797baup+replayingreplaying, {"bytes_per_second":3327590.4,"bytes_per_snapshot":66551808.0,"last_snapshot_bytes":66551808,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726146263,"remote_snapshot_timestamp":1726146263,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:05:17image-primary97d07d66e-fe3f-46b2-83dc-67fa35b43f0eup+replayingreplaying, {"bytes_per_second":3346227.2,"bytes_per_snapshot":66924544.0,"last_snapshot_bytes":66924544,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726146264,"remote_snapshot_timestamp":1726146264,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:05:17' 2024-09-12T13:05:44.102 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:05:44.102 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary8'\'']/last_update' 2024-09-12T13:05:44.106 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:05:17' 2024-09-12T13:05:44.107 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary8'\'']/state' 2024-09-12T13:05:44.111 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+replaying 2024-09-12T13:05:44.111 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+replaying 2024-09-12T13:05:44.112 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+replaying 2024-09-12T13:05:44.113 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+replaying 2024-09-12T13:05:44.114 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:05:44.114 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:05:44.114 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ remove_image cluster2 mirror image-primary8 2024-09-12T13:05:44.114 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:05:44.114 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:05:44.115 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:05:44.115 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 snap purge mirror/image-primary8 2024-09-12T13:05:44.175 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 rm mirror/image-primary8 2024-09-12T13:05:44.309 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:05:44.309+0000 7f67e4800640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x560555bf56c0 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-09-12T13:05:44.309 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:05:44.309+0000 7f67eca00640 -1 rbd::mirror::ImageReplayer: 0x5605561acf00 [3/e5807591-ea6d-4541-b1c3-0785c5e797ba] operator(): start failed: (85) Interrupted system call should be restarted 2024-09-12T13:05:45.794 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-09-12T13:05:45.800 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_image_removal cluster1 mirror image-primary8 2024-09-12T13:05:45.800 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:05:45.800 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:05:45.800 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:05:45.800 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:05:45.801 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 ls mirror 2024-09-12T13:05:45.801 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep -wq image-primary8 2024-09-12T13:05:45.857 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:05:45.857 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rm -rf test-primary8 2024-09-12T13:05:45.860 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:05:45.860 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ IMG=image-primary9 2024-09-12T13:05:45.860 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_replaying_status_in_pool_dir cluster1 mirror image-primary9 2024-09-12T13:05:45.860 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:05:45.860 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:05:45.860 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:05:45.860 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' snapshot = journal ']' 2024-09-12T13:05:45.860 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary9 up+replaying 2024-09-12T13:05:45.860 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:05:45.861 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:05:45.861 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:05:45.861 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:05:45.861 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:05:45.861 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:05:45.861 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:05:45.861 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:05:46.862 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary9 up+replaying '' '' 2024-09-12T13:05:46.862 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:05:46.862 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:05:46.863 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:05:46.863 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:05:46.863 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:05:46.863 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:05:46.863 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:05:46.863 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:05:46.864 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary9 2024-09-12T13:05:46.937 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary9: 2024-09-12T13:05:46.937 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 7d07d66e-fe3f-46b2-83dc-67fa35b43f0e 2024-09-12T13:05:46.937 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+replaying 2024-09-12T13:05:46.937 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: replaying, {"bytes_per_second":3346227.2,"bytes_per_snapshot":66924544.0,"last_snapshot_bytes":66924544,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726146264,"remote_snapshot_timestamp":1726146264,"replay_state":"idle"} 2024-09-12T13:05:46.937 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:05:46.937 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:05:17' 2024-09-12T13:05:46.937 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+replaying' 2024-09-12T13:05:46.938 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+replaying 2024-09-12T13:05:46.939 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:05:46.941 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: replaying, {"bytes_per_second":3346227.2,"bytes_per_snapshot":66924544.0,"last_snapshot_bytes":66924544,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726146264,"remote_snapshot_timestamp":1726146264,"replay_state":"idle"} 2024-09-12T13:05:46.942 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:05:46.942 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+replaying 2024-09-12T13:05:46.942 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:05:46.945 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+replaying 2024-09-12T13:05:46.945 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:05:46.947 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:05:46.947 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:05:46.948 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:05:46.953 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:05:17' 2024-09-12T13:05:46.953 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary9 up+replaying '2024-09-12 13:05:17' 2024-09-12T13:05:46.953 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:05:46.953 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:05:46.953 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:05:46.953 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:05:46.953 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:05:17' 2024-09-12T13:05:46.953 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:05:46.953 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:05:46.953 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-09-12T13:05:47.030 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING241934232mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary10e8086552-9fee-4a93-b94f-d2d6dc6bff63up+replayingreplaying, {"bytes_per_second":3165184.0,"bytes_per_snapshot":63303680.0,"last_snapshot_bytes":63303680,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726146266,"remote_snapshot_timestamp":1726146266,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:05:17image-primary97d07d66e-fe3f-46b2-83dc-67fa35b43f0eup+replayingreplaying, {"bytes_per_second":3346227.2,"bytes_per_snapshot":66924544.0,"last_snapshot_bytes":66924544,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726146264,"remote_snapshot_timestamp":1726146264,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:05:17' 2024-09-12T13:05:47.030 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:05:47.030 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary9'\'']/last_update' 2024-09-12T13:05:47.034 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:05:17' 2024-09-12T13:05:47.035 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary9'\'']/state' 2024-09-12T13:05:47.038 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+replaying 2024-09-12T13:05:47.039 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+replaying 2024-09-12T13:05:47.039 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+replaying 2024-09-12T13:05:47.041 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+replaying 2024-09-12T13:05:47.042 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:05:47.042 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:05:47.042 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ remove_image cluster2 mirror image-primary9 2024-09-12T13:05:47.042 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:05:47.042 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:05:47.042 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:05:47.043 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 snap purge mirror/image-primary9 2024-09-12T13:05:47.101 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 rm mirror/image-primary9 2024-09-12T13:05:47.223 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:05:47.222+0000 7f67e5200640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x560555a05ba0 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-09-12T13:05:47.223 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:05:47.222+0000 7f67ed400640 -1 rbd::mirror::ImageReplayer: 0x5605561ad680 [3/7d07d66e-fe3f-46b2-83dc-67fa35b43f0e] operator(): start failed: (85) Interrupted system call should be restarted 2024-09-12T13:05:47.891 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:05:47.891+0000 7f67e5200640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x560555bf5860 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-09-12T13:05:47.891 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:05:47.891+0000 7f67ed400640 -1 rbd::mirror::ImageReplayer: 0x5605561ad680 [3/7d07d66e-fe3f-46b2-83dc-67fa35b43f0e] operator(): start failed: (85) Interrupted system call should be restarted 2024-09-12T13:05:48.802 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-09-12T13:05:48.807 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_image_removal cluster1 mirror image-primary9 2024-09-12T13:05:48.808 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:05:48.808 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:05:48.808 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:05:48.808 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:05:48.808 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 ls mirror 2024-09-12T13:05:48.808 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep -wq image-primary9 2024-09-12T13:05:48.858 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:05:48.858 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rm -rf test-primary9 2024-09-12T13:05:48.860 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:05:48.861 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ IMG=image-primary10 2024-09-12T13:05:48.861 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_replaying_status_in_pool_dir cluster1 mirror image-primary10 2024-09-12T13:05:48.861 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:05:48.861 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:05:48.861 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:05:48.861 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' snapshot = journal ']' 2024-09-12T13:05:48.861 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary10 up+replaying 2024-09-12T13:05:48.861 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:05:48.861 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:05:48.861 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:05:48.861 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:05:48.861 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:05:48.861 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:05:48.862 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:05:48.862 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:05:49.863 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary10 up+replaying '' '' 2024-09-12T13:05:49.863 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:05:49.863 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:05:49.863 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:05:49.863 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:05:49.863 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:05:49.863 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:05:49.863 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:05:49.864 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:05:49.864 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary10 2024-09-12T13:05:49.938 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary10: 2024-09-12T13:05:49.938 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: e8086552-9fee-4a93-b94f-d2d6dc6bff63 2024-09-12T13:05:49.938 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+replaying 2024-09-12T13:05:49.938 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: replaying, {"bytes_per_second":0.0,"bytes_per_snapshot":63303680.0,"last_snapshot_bytes":63303680,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726146266,"remote_snapshot_timestamp":1726146266,"replay_state":"idle"} 2024-09-12T13:05:49.939 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:05:49.939 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:05:47' 2024-09-12T13:05:49.939 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+replaying' 2024-09-12T13:05:49.941 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+replaying 2024-09-12T13:05:49.941 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:05:49.944 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: replaying, {"bytes_per_second":0.0,"bytes_per_snapshot":63303680.0,"last_snapshot_bytes":63303680,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726146266,"remote_snapshot_timestamp":1726146266,"replay_state":"idle"} 2024-09-12T13:05:49.944 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:05:49.944 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+replaying 2024-09-12T13:05:49.945 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:05:49.946 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+replaying 2024-09-12T13:05:49.947 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:05:49.949 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:05:49.950 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:05:49.950 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:05:49.954 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:05:47' 2024-09-12T13:05:49.954 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary10 up+replaying '2024-09-12 13:05:47' 2024-09-12T13:05:49.954 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:05:49.955 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:05:49.955 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:05:49.955 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:05:49.955 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:05:47' 2024-09-12T13:05:49.955 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:05:49.955 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:05:49.955 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-09-12T13:05:50.036 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING141934232mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary10e8086552-9fee-4a93-b94f-d2d6dc6bff63up+replayingreplaying, {"bytes_per_second":0.0,"bytes_per_snapshot":63303680.0,"last_snapshot_bytes":63303680,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726146266,"remote_snapshot_timestamp":1726146266,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:05:47' 2024-09-12T13:05:50.036 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:05:50.037 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary10'\'']/last_update' 2024-09-12T13:05:50.040 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:05:47' 2024-09-12T13:05:50.041 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary10'\'']/state' 2024-09-12T13:05:50.045 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+replaying 2024-09-12T13:05:50.045 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+replaying 2024-09-12T13:05:50.046 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+replaying 2024-09-12T13:05:50.047 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+replaying 2024-09-12T13:05:50.048 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:05:50.048 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:05:50.048 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ remove_image cluster2 mirror image-primary10 2024-09-12T13:05:50.048 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:05:50.049 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:05:50.049 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:05:50.049 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 snap purge mirror/image-primary10 2024-09-12T13:05:50.121 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 rm mirror/image-primary10 2024-09-12T13:05:50.245 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:05:50.244+0000 7f67e4800640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x560555bf5ba0 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-09-12T13:05:50.245 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:05:50.244+0000 7f67eca00640 -1 rbd::mirror::ImageReplayer: 0x5605561ad900 [3/e8086552-9fee-4a93-b94f-d2d6dc6bff63] operator(): start failed: (85) Interrupted system call should be restarted 2024-09-12T13:05:51.814 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-09-12T13:05:51.821 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_image_removal cluster1 mirror image-primary10 2024-09-12T13:05:51.821 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:05:51.821 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:05:51.821 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:05:51.821 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:05:51.822 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 ls mirror 2024-09-12T13:05:51.822 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep -wq image-primary10 2024-09-12T13:05:51.871 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:05:51.871 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rm -rf test-primary10 2024-09-12T13:05:51.874 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..10} 2024-09-12T13:05:51.874 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEVS=() 2024-09-12T13:05:51.874 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ SNAP_PIDS=() 2024-09-12T13:05:51.874 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ COMPARE_PIDS=() 2024-09-12T13:05:51.874 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ WORKLOAD_PIDS=() 2024-09-12T13:05:51.874 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ RET=0 2024-09-12T13:05:51.874 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:05:51.874 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ IMG=image-primary1 2024-09-12T13:05:51.874 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ MNTPT=test-primary1 2024-09-12T13:05:51.874 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ create_image_and_enable_mirror cluster1 mirror image-primary1 snapshot 10G 2024-09-12T13:05:51.874 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:05:51.874 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:05:51.875 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:05:51.875 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:05:51.875 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:05:51.875 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:05:51.875 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mode=snapshot 2024-09-12T13:05:51.875 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n snapshot ']' 2024-09-12T13:05:51.875 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:05:51.875 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ create_image cluster1 mirror image-primary1 10G 2024-09-12T13:05:51.875 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:05:51.875 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:05:51.875 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:05:51.875 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:05:51.875 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:05:51.875 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:05:51.876 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local size=128 2024-09-12T13:05:51.876 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n 10G ']' 2024-09-12T13:05:51.876 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ size=10G 2024-09-12T13:05:51.876 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:05:51.876 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 create --size 10G --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image-primary1 2024-09-12T13:05:51.940 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ is_pool_mirror_mode_image mirror 2024-09-12T13:05:51.940 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:05:51.940 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' image = image ']' 2024-09-12T13:05:51.941 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:05:51.941 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ enable_mirror cluster1 mirror image-primary1 snapshot 2024-09-12T13:05:51.941 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:05:51.941 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:05:51.941 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:05:51.941 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mode=snapshot 2024-09-12T13:05:51.941 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 mirror image enable mirror/image-primary1 snapshot 2024-09-12T13:05:52.775 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Mirroring enabled 2024-09-12T13:05:52.786 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 info mirror/image-primary1 2024-09-12T13:05:52.843 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:rbd image 'image-primary1': 2024-09-12T13:05:52.843 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: size 10 GiB in 2560 objects 2024-09-12T13:05:52.843 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: order 22 (4 MiB objects) 2024-09-12T13:05:52.843 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: snapshot_count: 1 2024-09-12T13:05:52.844 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: id: 157513c846d6 2024-09-12T13:05:52.844 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: block_name_prefix: rbd_data.157513c846d6 2024-09-12T13:05:52.844 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: format: 2 2024-09-12T13:05:52.844 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: features: layering, exclusive-lock, object-map, fast-diff 2024-09-12T13:05:52.844 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: op_features: 2024-09-12T13:05:52.844 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: flags: 2024-09-12T13:05:52.844 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: create_timestamp: Thu Sep 12 13:05:51 2024 2024-09-12T13:05:52.844 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: access_timestamp: Thu Sep 12 13:05:51 2024 2024-09-12T13:05:52.844 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: modify_timestamp: Thu Sep 12 13:05:51 2024 2024-09-12T13:05:52.844 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring state: enabled 2024-09-12T13:05:52.845 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring mode: snapshot 2024-09-12T13:05:52.845 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring global id: 1df40b54-54f7-41fe-a2f4-c900ccf2e016 2024-09-12T13:05:52.845 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring primary: true 2024-09-12T13:05:52.849 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \n\b\d ]] 2024-09-12T13:05:52.849 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \k\r\b\d ]] 2024-09-12T13:05:52.849 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo rbd --cluster cluster1 device map -t krbd mirror/image-primary1 2024-09-12T13:05:52.981 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEV=/dev/rbd0 2024-09-12T13:05:52.981 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEVS+=($DEV) 2024-09-12T13:05:52.981 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo mkfs.ext4 /dev/rbd0 2024-09-12T13:05:53.023 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:mke2fs 1.46.5 (30-Dec-2021) 2024-09-12T13:05:53.036 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Discarding device blocks: 0/2621440 done 2024-09-12T13:05:53.036 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Creating filesystem with 2621440 4k blocks and 655360 inodes 2024-09-12T13:05:53.036 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Filesystem UUID: 2d3c1458-45fa-44d9-8a52-8751bff4f668 2024-09-12T13:05:53.036 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Superblock backups stored on blocks: 2024-09-12T13:05:53.036 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 2024-09-12T13:05:53.036 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 2024-09-12T13:05:53.036 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Allocating group tables: 0/80 done 2024-09-12T13:05:53.036 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Writing inode tables: 0/80 done 2024-09-12T13:05:53.042 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Creating journal (16384 blocks): done 2024-09-12T13:05:53.084 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Writing superblocks and filesystem accounting information: 0/80 done 2024-09-12T13:05:53.084 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 2024-09-12T13:05:53.086 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mkdir test-primary1 2024-09-12T13:05:53.089 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo mount /dev/rbd0 test-primary1 2024-09-12T13:05:53.148 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ whoami 2024-09-12T13:05:53.159 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo chown ubuntu test-primary1 2024-09-12T13:05:53.190 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ SNAP_PIDS+=($!) 2024-09-12T13:05:53.190 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ take_mirror_snapshots cluster1 mirror image-primary1 2024-09-12T13:05:53.190 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:05:53.190 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:05:53.190 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:05:53.190 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:05:53.190 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:05:53.191 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:05:53.191 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:05:53.191 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:05:53.191 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ WORKLOAD_PIDS+=($!) 2024-09-12T13:05:53.191 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:05:53.191 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:05:53.191 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ IMG=image-primary2 2024-09-12T13:05:53.191 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ MNTPT=test-primary2 2024-09-12T13:05:53.191 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ slow_untar_workload test-primary1 2024-09-12T13:05:53.192 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ create_image_and_enable_mirror cluster1 mirror image-primary2 snapshot 10G 2024-09-12T13:05:53.193 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:05:53.193 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:05:53.193 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:05:53.193 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:05:53.193 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:05:53.193 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:05:53.193 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mode=snapshot 2024-09-12T13:05:53.193 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n snapshot ']' 2024-09-12T13:05:53.194 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:05:53.194 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mountpt=test-primary1 2024-09-12T13:05:53.194 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ create_image cluster1 mirror image-primary2 10G 2024-09-12T13:05:53.194 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ cp linux-5.4.tar.gz test-primary1 2024-09-12T13:05:53.194 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:05:53.194 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:05:53.194 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:05:53.194 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:05:53.194 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:05:53.194 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:05:53.194 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local size=128 2024-09-12T13:05:53.194 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n 10G ']' 2024-09-12T13:05:53.194 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ size=10G 2024-09-12T13:05:53.195 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:05:53.195 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 create --size 10G --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image-primary2 2024-09-12T13:05:53.261 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ is_pool_mirror_mode_image mirror 2024-09-12T13:05:53.261 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:05:53.261 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' image = image ']' 2024-09-12T13:05:53.261 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:05:53.261 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ enable_mirror cluster1 mirror image-primary2 snapshot 2024-09-12T13:05:53.261 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:05:53.261 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:05:53.261 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:05:53.261 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mode=snapshot 2024-09-12T13:05:53.261 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 mirror image enable mirror/image-primary2 snapshot 2024-09-12T13:05:53.475 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local ret=0 2024-09-12T13:05:53.475 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ timeout 5m bash -c 'zcat test-primary1/linux-5.4.tar.gz | pv -L 256K | tar xf - -C test-primary1' 2024-09-12T13:05:53.819 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Mirroring enabled 2024-09-12T13:05:53.821 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 334 2024-09-12T13:05:53.834 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:05:53.837 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 info mirror/image-primary2 2024-09-12T13:05:53.907 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:rbd image 'image-primary2': 2024-09-12T13:05:53.908 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: size 10 GiB in 2560 objects 2024-09-12T13:05:53.908 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: order 22 (4 MiB objects) 2024-09-12T13:05:53.908 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: snapshot_count: 1 2024-09-12T13:05:53.908 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: id: 158418638f04 2024-09-12T13:05:53.908 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: block_name_prefix: rbd_data.158418638f04 2024-09-12T13:05:53.908 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: format: 2 2024-09-12T13:05:53.908 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: features: layering, exclusive-lock, object-map, fast-diff 2024-09-12T13:05:53.908 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: op_features: 2024-09-12T13:05:53.908 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: flags: 2024-09-12T13:05:53.908 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: create_timestamp: Thu Sep 12 13:05:53 2024 2024-09-12T13:05:53.908 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: access_timestamp: Thu Sep 12 13:05:53 2024 2024-09-12T13:05:53.909 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: modify_timestamp: Thu Sep 12 13:05:53 2024 2024-09-12T13:05:53.909 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring state: enabled 2024-09-12T13:05:53.909 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring mode: snapshot 2024-09-12T13:05:53.909 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring global id: 23ee0857-2880-4d74-a09e-17095d384c6b 2024-09-12T13:05:53.909 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring primary: true 2024-09-12T13:05:53.913 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \n\b\d ]] 2024-09-12T13:05:53.913 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \k\r\b\d ]] 2024-09-12T13:05:53.913 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo rbd --cluster cluster1 device map -t krbd mirror/image-primary2 2024-09-12T13:05:54.061 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEV=/dev/rbd1 2024-09-12T13:05:54.061 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEVS+=($DEV) 2024-09-12T13:05:54.061 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo mkfs.ext4 /dev/rbd1 2024-09-12T13:05:54.082 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:mke2fs 1.46.5 (30-Dec-2021) 2024-09-12T13:05:54.093 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Discarding device blocks: 0/2621440 done 2024-09-12T13:05:54.093 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Creating filesystem with 2621440 4k blocks and 655360 inodes 2024-09-12T13:05:54.093 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Filesystem UUID: 8ce49241-8a48-4804-bdb9-96c7e2dd869b 2024-09-12T13:05:54.093 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Superblock backups stored on blocks: 2024-09-12T13:05:54.094 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 2024-09-12T13:05:54.094 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 2024-09-12T13:05:54.094 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Allocating group tables: 0/80 done 2024-09-12T13:05:54.094 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Writing inode tables: 0/80 done 2024-09-12T13:05:54.098 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Creating journal (16384 blocks): done 2024-09-12T13:05:54.161 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Writing superblocks and filesystem accounting information: 0/80 done 2024-09-12T13:05:54.161 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 2024-09-12T13:05:54.164 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mkdir test-primary2 2024-09-12T13:05:54.166 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo mount /dev/rbd1 test-primary2 2024-09-12T13:05:54.243 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ whoami 2024-09-12T13:05:54.245 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo chown ubuntu test-primary2 2024-09-12T13:05:54.270 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ SNAP_PIDS+=($!) 2024-09-12T13:05:54.270 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ take_mirror_snapshots cluster1 mirror image-primary2 2024-09-12T13:05:54.270 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:05:54.270 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:05:54.270 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:05:54.270 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:05:54.271 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:05:54.271 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:05:54.271 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:05:54.271 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:05:54.271 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ WORKLOAD_PIDS+=($!) 2024-09-12T13:05:54.271 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:05:54.271 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:05:54.271 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ IMG=image-primary3 2024-09-12T13:05:54.273 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ MNTPT=test-primary3 2024-09-12T13:05:54.273 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ create_image_and_enable_mirror cluster1 mirror image-primary3 snapshot 10G 2024-09-12T13:05:54.273 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ slow_untar_workload test-primary2 2024-09-12T13:05:54.273 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:05:54.273 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:05:54.273 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:05:54.274 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:05:54.274 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:05:54.274 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:05:54.274 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mode=snapshot 2024-09-12T13:05:54.274 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n snapshot ']' 2024-09-12T13:05:54.274 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:05:54.274 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ create_image cluster1 mirror image-primary3 10G 2024-09-12T13:05:54.274 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mountpt=test-primary2 2024-09-12T13:05:54.274 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:05:54.274 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:05:54.274 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:05:54.274 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:05:54.274 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:05:54.274 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ cp linux-5.4.tar.gz test-primary2 2024-09-12T13:05:54.275 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:05:54.275 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local size=128 2024-09-12T13:05:54.275 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n 10G ']' 2024-09-12T13:05:54.275 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ size=10G 2024-09-12T13:05:54.275 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:05:54.275 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 create --size 10G --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image-primary3 2024-09-12T13:05:54.360 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ is_pool_mirror_mode_image mirror 2024-09-12T13:05:54.361 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:05:54.361 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' image = image ']' 2024-09-12T13:05:54.361 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:05:54.361 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ enable_mirror cluster1 mirror image-primary3 snapshot 2024-09-12T13:05:54.361 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:05:54.361 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:05:54.361 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:05:54.361 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mode=snapshot 2024-09-12T13:05:54.361 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 mirror image enable mirror/image-primary3 snapshot 2024-09-12T13:05:54.577 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local ret=0 2024-09-12T13:05:54.577 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ timeout 5m bash -c 'zcat test-primary2/linux-5.4.tar.gz | pv -L 256K | tar xf - -C test-primary2' 2024-09-12T13:05:54.811 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Mirroring enabled 2024-09-12T13:05:54.828 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 336 2024-09-12T13:05:54.831 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 info mirror/image-primary3 2024-09-12T13:05:54.838 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:05:55.143 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:rbd image 'image-primary3': 2024-09-12T13:05:55.143 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: size 10 GiB in 2560 objects 2024-09-12T13:05:55.144 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: order 22 (4 MiB objects) 2024-09-12T13:05:55.144 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: snapshot_count: 1 2024-09-12T13:05:55.144 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: id: 1592d91da04b 2024-09-12T13:05:55.144 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: block_name_prefix: rbd_data.1592d91da04b 2024-09-12T13:05:55.144 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: format: 2 2024-09-12T13:05:55.144 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: features: layering, exclusive-lock, object-map, fast-diff 2024-09-12T13:05:55.144 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: op_features: 2024-09-12T13:05:55.144 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: flags: 2024-09-12T13:05:55.144 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: create_timestamp: Thu Sep 12 13:05:54 2024 2024-09-12T13:05:55.144 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: access_timestamp: Thu Sep 12 13:05:54 2024 2024-09-12T13:05:55.144 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: modify_timestamp: Thu Sep 12 13:05:54 2024 2024-09-12T13:05:55.144 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring state: enabled 2024-09-12T13:05:55.144 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring mode: snapshot 2024-09-12T13:05:55.144 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring global id: 92498ee6-eac1-4596-be69-c222174caeb6 2024-09-12T13:05:55.145 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring primary: true 2024-09-12T13:05:55.149 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \n\b\d ]] 2024-09-12T13:05:55.149 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \k\r\b\d ]] 2024-09-12T13:05:55.150 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo rbd --cluster cluster1 device map -t krbd mirror/image-primary3 2024-09-12T13:05:55.265 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEV=/dev/rbd2 2024-09-12T13:05:55.265 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEVS+=($DEV) 2024-09-12T13:05:55.265 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo mkfs.ext4 /dev/rbd2 2024-09-12T13:05:55.285 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:mke2fs 1.46.5 (30-Dec-2021) 2024-09-12T13:05:55.297 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Discarding device blocks: 0/2621440 done 2024-09-12T13:05:55.297 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Creating filesystem with 2621440 4k blocks and 655360 inodes 2024-09-12T13:05:55.297 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Filesystem UUID: b0e7a742-c2a8-4373-a417-e0b670e9b13a 2024-09-12T13:05:55.297 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Superblock backups stored on blocks: 2024-09-12T13:05:55.297 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 2024-09-12T13:05:55.297 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 2024-09-12T13:05:55.297 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Allocating group tables: 0/80 done 2024-09-12T13:05:55.297 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Writing inode tables: 0/80 done 2024-09-12T13:05:55.302 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Creating journal (16384 blocks): done 2024-09-12T13:05:55.341 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Writing superblocks and filesystem accounting information: 0/80 done 2024-09-12T13:05:55.342 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 2024-09-12T13:05:55.344 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mkdir test-primary3 2024-09-12T13:05:55.347 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo mount /dev/rbd2 test-primary3 2024-09-12T13:05:55.402 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ whoami 2024-09-12T13:05:55.404 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo chown ubuntu test-primary3 2024-09-12T13:05:55.429 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ SNAP_PIDS+=($!) 2024-09-12T13:05:55.429 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ take_mirror_snapshots cluster1 mirror image-primary3 2024-09-12T13:05:55.429 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:05:55.429 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:05:55.429 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:05:55.429 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:05:55.430 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:05:55.430 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:05:55.430 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:05:55.430 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:05:55.430 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:05:55.430 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ WORKLOAD_PIDS+=($!) 2024-09-12T13:05:55.430 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:05:55.430 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ slow_untar_workload test-primary3 2024-09-12T13:05:55.430 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ IMG=image-primary4 2024-09-12T13:05:55.430 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ MNTPT=test-primary4 2024-09-12T13:05:55.430 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ create_image_and_enable_mirror cluster1 mirror image-primary4 snapshot 10G 2024-09-12T13:05:55.430 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mountpt=test-primary3 2024-09-12T13:05:55.431 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:05:55.431 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:05:55.431 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ cp linux-5.4.tar.gz test-primary3 2024-09-12T13:05:55.431 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:05:55.431 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:05:55.431 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:05:55.431 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:05:55.431 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mode=snapshot 2024-09-12T13:05:55.431 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n snapshot ']' 2024-09-12T13:05:55.431 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:05:55.431 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ create_image cluster1 mirror image-primary4 10G 2024-09-12T13:05:55.431 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:05:55.431 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:05:55.432 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:05:55.432 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:05:55.432 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:05:55.432 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:05:55.432 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local size=128 2024-09-12T13:05:55.432 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n 10G ']' 2024-09-12T13:05:55.432 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ size=10G 2024-09-12T13:05:55.432 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:05:55.432 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 create --size 10G --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image-primary4 2024-09-12T13:05:55.497 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ is_pool_mirror_mode_image mirror 2024-09-12T13:05:55.497 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:05:55.497 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' image = image ']' 2024-09-12T13:05:55.497 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:05:55.498 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ enable_mirror cluster1 mirror image-primary4 snapshot 2024-09-12T13:05:55.498 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:05:55.498 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:05:55.498 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:05:55.498 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mode=snapshot 2024-09-12T13:05:55.498 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 mirror image enable mirror/image-primary4 snapshot 2024-09-12T13:05:55.728 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local ret=0 2024-09-12T13:05:55.728 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ timeout 5m bash -c 'zcat test-primary3/linux-5.4.tar.gz | pv -L 256K | tar xf - -C test-primary3' 2024-09-12T13:05:55.796 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Mirroring enabled 2024-09-12T13:05:55.801 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 338 2024-09-12T13:05:55.808 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 info mirror/image-primary4 2024-09-12T13:05:55.809 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:05:55.867 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:rbd image 'image-primary4': 2024-09-12T13:05:55.867 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: size 10 GiB in 2560 objects 2024-09-12T13:05:55.867 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: order 22 (4 MiB objects) 2024-09-12T13:05:55.868 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: snapshot_count: 1 2024-09-12T13:05:55.868 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: id: 15a1c319bd93 2024-09-12T13:05:55.868 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: block_name_prefix: rbd_data.15a1c319bd93 2024-09-12T13:05:55.868 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: format: 2 2024-09-12T13:05:55.868 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: features: layering, exclusive-lock, object-map, fast-diff 2024-09-12T13:05:55.868 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: op_features: 2024-09-12T13:05:55.868 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: flags: 2024-09-12T13:05:55.868 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: create_timestamp: Thu Sep 12 13:05:55 2024 2024-09-12T13:05:55.868 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: access_timestamp: Thu Sep 12 13:05:55 2024 2024-09-12T13:05:55.868 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: modify_timestamp: Thu Sep 12 13:05:55 2024 2024-09-12T13:05:55.868 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring state: enabled 2024-09-12T13:05:55.868 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring mode: snapshot 2024-09-12T13:05:55.869 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring global id: 11f002da-1dfb-4012-a01a-c5237417a099 2024-09-12T13:05:55.869 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring primary: true 2024-09-12T13:05:55.872 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \n\b\d ]] 2024-09-12T13:05:55.872 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \k\r\b\d ]] 2024-09-12T13:05:55.873 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo rbd --cluster cluster1 device map -t krbd mirror/image-primary4 2024-09-12T13:05:55.989 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEV=/dev/rbd3 2024-09-12T13:05:55.989 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEVS+=($DEV) 2024-09-12T13:05:55.989 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo mkfs.ext4 /dev/rbd3 2024-09-12T13:05:56.010 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:mke2fs 1.46.5 (30-Dec-2021) 2024-09-12T13:05:56.022 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Discarding device blocks: 0/2621440 done 2024-09-12T13:05:56.022 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Creating filesystem with 2621440 4k blocks and 655360 inodes 2024-09-12T13:05:56.022 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Filesystem UUID: 5c008fbd-db0b-4c02-9409-b27c000686b1 2024-09-12T13:05:56.022 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Superblock backups stored on blocks: 2024-09-12T13:05:56.023 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 2024-09-12T13:05:56.023 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 2024-09-12T13:05:56.023 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Allocating group tables: 0/80 done 2024-09-12T13:05:56.023 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Writing inode tables: 0/80 done 2024-09-12T13:05:56.027 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Creating journal (16384 blocks): done 2024-09-12T13:05:56.064 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Writing superblocks and filesystem accounting information: 0/80 done 2024-09-12T13:05:56.064 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 2024-09-12T13:05:56.068 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mkdir test-primary4 2024-09-12T13:05:56.069 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo mount /dev/rbd3 test-primary4 2024-09-12T13:05:56.123 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ whoami 2024-09-12T13:05:56.125 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo chown ubuntu test-primary4 2024-09-12T13:05:56.150 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ SNAP_PIDS+=($!) 2024-09-12T13:05:56.150 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ take_mirror_snapshots cluster1 mirror image-primary4 2024-09-12T13:05:56.150 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:05:56.150 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:05:56.150 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:05:56.150 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:05:56.150 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:05:56.150 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ WORKLOAD_PIDS+=($!) 2024-09-12T13:05:56.151 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:05:56.151 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:05:56.151 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:05:56.151 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:05:56.151 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:05:56.151 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ IMG=image-primary5 2024-09-12T13:05:56.151 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ MNTPT=test-primary5 2024-09-12T13:05:56.151 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ create_image_and_enable_mirror cluster1 mirror image-primary5 snapshot 10G 2024-09-12T13:05:56.151 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ slow_untar_workload test-primary4 2024-09-12T13:05:56.151 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:05:56.151 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:05:56.152 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:05:56.152 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:05:56.152 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:05:56.152 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:05:56.152 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mode=snapshot 2024-09-12T13:05:56.152 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n snapshot ']' 2024-09-12T13:05:56.152 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:05:56.152 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ create_image cluster1 mirror image-primary5 10G 2024-09-12T13:05:56.152 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:05:56.152 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:05:56.152 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:05:56.152 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:05:56.153 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mountpt=test-primary4 2024-09-12T13:05:56.153 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:05:56.153 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:05:56.153 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local size=128 2024-09-12T13:05:56.153 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n 10G ']' 2024-09-12T13:05:56.153 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ size=10G 2024-09-12T13:05:56.153 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ cp linux-5.4.tar.gz test-primary4 2024-09-12T13:05:56.153 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:05:56.153 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 create --size 10G --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image-primary5 2024-09-12T13:05:56.216 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ is_pool_mirror_mode_image mirror 2024-09-12T13:05:56.216 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:05:56.216 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' image = image ']' 2024-09-12T13:05:56.216 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:05:56.216 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ enable_mirror cluster1 mirror image-primary5 snapshot 2024-09-12T13:05:56.216 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:05:56.216 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:05:56.217 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:05:56.217 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mode=snapshot 2024-09-12T13:05:56.217 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 mirror image enable mirror/image-primary5 snapshot 2024-09-12T13:05:56.437 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local ret=0 2024-09-12T13:05:56.437 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ timeout 5m bash -c 'zcat test-primary4/linux-5.4.tar.gz | pv -L 256K | tar xf - -C test-primary4' 2024-09-12T13:05:56.809 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Mirroring enabled 2024-09-12T13:05:56.810 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 340 2024-09-12T13:05:56.819 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:05:56.824 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 info mirror/image-primary5 2024-09-12T13:05:56.837 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:05:56.837 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:05:56.837 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:05:56.837 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:05:56.837 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:05:56.837 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:05:56.883 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:rbd image 'image-primary5': 2024-09-12T13:05:56.883 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: size 10 GiB in 2560 objects 2024-09-12T13:05:56.883 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: order 22 (4 MiB objects) 2024-09-12T13:05:56.883 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: snapshot_count: 1 2024-09-12T13:05:56.883 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: id: 15b072e984f2 2024-09-12T13:05:56.883 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: block_name_prefix: rbd_data.15b072e984f2 2024-09-12T13:05:56.884 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: format: 2 2024-09-12T13:05:56.884 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: features: layering, exclusive-lock, object-map, fast-diff 2024-09-12T13:05:56.884 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: op_features: 2024-09-12T13:05:56.884 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: flags: 2024-09-12T13:05:56.884 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: create_timestamp: Thu Sep 12 13:05:56 2024 2024-09-12T13:05:56.884 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: access_timestamp: Thu Sep 12 13:05:56 2024 2024-09-12T13:05:56.884 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: modify_timestamp: Thu Sep 12 13:05:56 2024 2024-09-12T13:05:56.884 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring state: enabled 2024-09-12T13:05:56.884 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring mode: snapshot 2024-09-12T13:05:56.884 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring global id: 261d375d-e3b2-4dc4-91bd-36dfede8b276 2024-09-12T13:05:56.884 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring primary: true 2024-09-12T13:05:56.889 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \n\b\d ]] 2024-09-12T13:05:56.889 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \k\r\b\d ]] 2024-09-12T13:05:56.890 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo rbd --cluster cluster1 device map -t krbd mirror/image-primary5 2024-09-12T13:05:57.007 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEV=/dev/rbd4 2024-09-12T13:05:57.007 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEVS+=($DEV) 2024-09-12T13:05:57.007 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo mkfs.ext4 /dev/rbd4 2024-09-12T13:05:57.027 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:mke2fs 1.46.5 (30-Dec-2021) 2024-09-12T13:05:57.041 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Discarding device blocks: 0/2621440 done 2024-09-12T13:05:57.041 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Creating filesystem with 2621440 4k blocks and 655360 inodes 2024-09-12T13:05:57.041 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Filesystem UUID: 923778da-8461-4979-9c7b-3606ee253278 2024-09-12T13:05:57.041 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Superblock backups stored on blocks: 2024-09-12T13:05:57.041 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 2024-09-12T13:05:57.041 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 2024-09-12T13:05:57.041 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Allocating group tables: 0/80 done 2024-09-12T13:05:57.041 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Writing inode tables: 0/80 done 2024-09-12T13:05:57.046 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Creating journal (16384 blocks): done 2024-09-12T13:05:57.090 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Writing superblocks and filesystem accounting information: 0/80 done 2024-09-12T13:05:57.090 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 2024-09-12T13:05:57.093 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mkdir test-primary5 2024-09-12T13:05:57.095 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo mount /dev/rbd4 test-primary5 2024-09-12T13:05:57.150 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ whoami 2024-09-12T13:05:57.151 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo chown ubuntu test-primary5 2024-09-12T13:05:57.177 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ SNAP_PIDS+=($!) 2024-09-12T13:05:57.177 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ take_mirror_snapshots cluster1 mirror image-primary5 2024-09-12T13:05:57.177 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:05:57.177 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:05:57.177 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:05:57.177 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:05:57.177 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:05:57.177 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:05:57.177 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:05:57.177 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:05:57.177 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:05:57.177 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ WORKLOAD_PIDS+=($!) 2024-09-12T13:05:57.178 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:05:57.178 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ slow_untar_workload test-primary5 2024-09-12T13:05:57.178 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ IMG=image-primary6 2024-09-12T13:05:57.178 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ MNTPT=test-primary6 2024-09-12T13:05:57.178 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ create_image_and_enable_mirror cluster1 mirror image-primary6 snapshot 10G 2024-09-12T13:05:57.178 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:05:57.178 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:05:57.178 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:05:57.178 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:05:57.178 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:05:57.178 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:05:57.178 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mode=snapshot 2024-09-12T13:05:57.178 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mountpt=test-primary5 2024-09-12T13:05:57.178 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n snapshot ']' 2024-09-12T13:05:57.179 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:05:57.179 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ cp linux-5.4.tar.gz test-primary5 2024-09-12T13:05:57.179 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ create_image cluster1 mirror image-primary6 10G 2024-09-12T13:05:57.179 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:05:57.179 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:05:57.179 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:05:57.179 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:05:57.179 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:05:57.179 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:05:57.179 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local size=128 2024-09-12T13:05:57.179 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n 10G ']' 2024-09-12T13:05:57.179 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ size=10G 2024-09-12T13:05:57.179 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:05:57.179 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 create --size 10G --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image-primary6 2024-09-12T13:05:57.245 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ is_pool_mirror_mode_image mirror 2024-09-12T13:05:57.245 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:05:57.245 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' image = image ']' 2024-09-12T13:05:57.246 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:05:57.246 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ enable_mirror cluster1 mirror image-primary6 snapshot 2024-09-12T13:05:57.246 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:05:57.246 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:05:57.246 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:05:57.246 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mode=snapshot 2024-09-12T13:05:57.246 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 mirror image enable mirror/image-primary6 snapshot 2024-09-12T13:05:57.464 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local ret=0 2024-09-12T13:05:57.464 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ timeout 5m bash -c 'zcat test-primary5/linux-5.4.tar.gz | pv -L 256K | tar xf - -C test-primary5' 2024-09-12T13:05:57.769 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Mirroring enabled 2024-09-12T13:05:57.769 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 342 2024-09-12T13:05:57.772 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 343 2024-09-12T13:05:57.781 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:05:57.782 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:05:57.785 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 info mirror/image-primary6 2024-09-12T13:05:57.841 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:05:57.841 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:05:57.841 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:05:57.842 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:05:57.842 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:05:57.842 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:05:57.848 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:rbd image 'image-primary6': 2024-09-12T13:05:57.848 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: size 10 GiB in 2560 objects 2024-09-12T13:05:57.848 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: order 22 (4 MiB objects) 2024-09-12T13:05:57.848 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: snapshot_count: 1 2024-09-12T13:05:57.849 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: id: 15c2157ec30f 2024-09-12T13:05:57.849 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: block_name_prefix: rbd_data.15c2157ec30f 2024-09-12T13:05:57.849 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: format: 2 2024-09-12T13:05:57.849 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: features: layering, exclusive-lock, object-map, fast-diff 2024-09-12T13:05:57.849 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: op_features: 2024-09-12T13:05:57.849 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: flags: 2024-09-12T13:05:57.849 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: create_timestamp: Thu Sep 12 13:05:57 2024 2024-09-12T13:05:57.849 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: access_timestamp: Thu Sep 12 13:05:57 2024 2024-09-12T13:05:57.849 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: modify_timestamp: Thu Sep 12 13:05:57 2024 2024-09-12T13:05:57.849 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring state: enabled 2024-09-12T13:05:57.849 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring mode: snapshot 2024-09-12T13:05:57.849 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring global id: b711ef4c-c601-46a4-a872-1c8abdd1ca45 2024-09-12T13:05:57.849 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring primary: true 2024-09-12T13:05:57.853 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \n\b\d ]] 2024-09-12T13:05:57.853 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \k\r\b\d ]] 2024-09-12T13:05:57.855 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo rbd --cluster cluster1 device map -t krbd mirror/image-primary6 2024-09-12T13:05:57.971 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEV=/dev/rbd5 2024-09-12T13:05:57.971 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEVS+=($DEV) 2024-09-12T13:05:57.971 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo mkfs.ext4 /dev/rbd5 2024-09-12T13:05:57.993 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:mke2fs 1.46.5 (30-Dec-2021) 2024-09-12T13:05:58.006 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Discarding device blocks: 0/26214401572864/2621440 done 2024-09-12T13:05:58.006 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Creating filesystem with 2621440 4k blocks and 655360 inodes 2024-09-12T13:05:58.006 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Filesystem UUID: a01e2ca7-8d6e-450b-bcf6-d5971e8da7be 2024-09-12T13:05:58.006 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Superblock backups stored on blocks: 2024-09-12T13:05:58.006 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 2024-09-12T13:05:58.006 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 2024-09-12T13:05:58.006 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Allocating group tables: 0/80 done 2024-09-12T13:05:58.006 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Writing inode tables: 0/80 done 2024-09-12T13:05:58.011 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Creating journal (16384 blocks): done 2024-09-12T13:05:58.051 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Writing superblocks and filesystem accounting information: 0/80 done 2024-09-12T13:05:58.051 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 2024-09-12T13:05:58.054 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mkdir test-primary6 2024-09-12T13:05:58.057 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo mount /dev/rbd5 test-primary6 2024-09-12T13:05:58.109 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ whoami 2024-09-12T13:05:58.111 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo chown ubuntu test-primary6 2024-09-12T13:05:58.136 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ SNAP_PIDS+=($!) 2024-09-12T13:05:58.137 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ take_mirror_snapshots cluster1 mirror image-primary6 2024-09-12T13:05:58.137 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:05:58.137 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:05:58.137 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:05:58.137 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:05:58.137 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:05:58.137 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:05:58.137 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:05:58.137 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:05:58.137 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:05:58.137 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ WORKLOAD_PIDS+=($!) 2024-09-12T13:05:58.137 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:05:58.138 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ slow_untar_workload test-primary6 2024-09-12T13:05:58.138 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ IMG=image-primary7 2024-09-12T13:05:58.138 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ MNTPT=test-primary7 2024-09-12T13:05:58.138 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ create_image_and_enable_mirror cluster1 mirror image-primary7 snapshot 10G 2024-09-12T13:05:58.138 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mountpt=test-primary6 2024-09-12T13:05:58.138 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:05:58.138 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ cp linux-5.4.tar.gz test-primary6 2024-09-12T13:05:58.138 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:05:58.138 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:05:58.138 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:05:58.138 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:05:58.138 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:05:58.138 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mode=snapshot 2024-09-12T13:05:58.138 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n snapshot ']' 2024-09-12T13:05:58.139 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:05:58.139 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ create_image cluster1 mirror image-primary7 10G 2024-09-12T13:05:58.139 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:05:58.139 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:05:58.139 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:05:58.139 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:05:58.139 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:05:58.139 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:05:58.139 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local size=128 2024-09-12T13:05:58.139 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n 10G ']' 2024-09-12T13:05:58.139 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ size=10G 2024-09-12T13:05:58.139 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:05:58.139 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 create --size 10G --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image-primary7 2024-09-12T13:05:58.209 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ is_pool_mirror_mode_image mirror 2024-09-12T13:05:58.210 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:05:58.210 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' image = image ']' 2024-09-12T13:05:58.210 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:05:58.210 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ enable_mirror cluster1 mirror image-primary7 snapshot 2024-09-12T13:05:58.210 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:05:58.210 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:05:58.210 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:05:58.210 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mode=snapshot 2024-09-12T13:05:58.210 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 mirror image enable mirror/image-primary7 snapshot 2024-09-12T13:05:58.423 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local ret=0 2024-09-12T13:05:58.423 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ timeout 5m bash -c 'zcat test-primary6/linux-5.4.tar.gz | pv -L 256K | tar xf - -C test-primary6' 2024-09-12T13:05:58.765 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Mirroring enabled 2024-09-12T13:05:58.770 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 345 2024-09-12T13:05:58.776 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 346 2024-09-12T13:05:58.779 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 info mirror/image-primary7 2024-09-12T13:05:58.781 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:05:58.786 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:05:58.812 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:05:58.812 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:05:58.812 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:05:58.812 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:05:58.812 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:05:58.813 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:05:58.845 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:rbd image 'image-primary7': 2024-09-12T13:05:58.846 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: size 10 GiB in 2560 objects 2024-09-12T13:05:58.846 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: order 22 (4 MiB objects) 2024-09-12T13:05:58.846 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: snapshot_count: 1 2024-09-12T13:05:58.846 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: id: 15d3f8bd03a7 2024-09-12T13:05:58.846 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: block_name_prefix: rbd_data.15d3f8bd03a7 2024-09-12T13:05:58.846 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: format: 2 2024-09-12T13:05:58.846 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: features: layering, exclusive-lock, object-map, fast-diff 2024-09-12T13:05:58.846 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: op_features: 2024-09-12T13:05:58.846 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: flags: 2024-09-12T13:05:58.846 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: create_timestamp: Thu Sep 12 13:05:58 2024 2024-09-12T13:05:58.846 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: access_timestamp: Thu Sep 12 13:05:58 2024 2024-09-12T13:05:58.846 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: modify_timestamp: Thu Sep 12 13:05:58 2024 2024-09-12T13:05:58.847 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring state: enabled 2024-09-12T13:05:58.847 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring mode: snapshot 2024-09-12T13:05:58.847 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring global id: 73a79146-a78a-4543-843c-6cef99c271d3 2024-09-12T13:05:58.847 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring primary: true 2024-09-12T13:05:58.854 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \n\b\d ]] 2024-09-12T13:05:58.854 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \k\r\b\d ]] 2024-09-12T13:05:58.854 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo rbd --cluster cluster1 device map -t krbd mirror/image-primary7 2024-09-12T13:05:58.976 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEV=/dev/rbd6 2024-09-12T13:05:58.976 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEVS+=($DEV) 2024-09-12T13:05:58.976 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo mkfs.ext4 /dev/rbd6 2024-09-12T13:05:58.996 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:mke2fs 1.46.5 (30-Dec-2021) 2024-09-12T13:05:59.009 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Discarding device blocks: 0/26214401048576/2621440 done 2024-09-12T13:05:59.009 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Creating filesystem with 2621440 4k blocks and 655360 inodes 2024-09-12T13:05:59.009 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Filesystem UUID: 209df5ad-04fb-46dc-b200-ca22fc39766e 2024-09-12T13:05:59.009 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Superblock backups stored on blocks: 2024-09-12T13:05:59.009 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 2024-09-12T13:05:59.009 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 2024-09-12T13:05:59.010 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Allocating group tables: 0/80 done 2024-09-12T13:05:59.010 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Writing inode tables: 0/80 done 2024-09-12T13:05:59.014 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Creating journal (16384 blocks): done 2024-09-12T13:05:59.054 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Writing superblocks and filesystem accounting information: 0/80 done 2024-09-12T13:05:59.054 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 2024-09-12T13:05:59.057 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mkdir test-primary7 2024-09-12T13:05:59.059 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo mount /dev/rbd6 test-primary7 2024-09-12T13:05:59.113 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ whoami 2024-09-12T13:05:59.115 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo chown ubuntu test-primary7 2024-09-12T13:05:59.140 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ SNAP_PIDS+=($!) 2024-09-12T13:05:59.140 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ take_mirror_snapshots cluster1 mirror image-primary7 2024-09-12T13:05:59.140 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:05:59.140 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:05:59.140 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:05:59.141 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:05:59.141 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:05:59.141 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:05:59.141 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:05:59.141 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:05:59.141 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:05:59.141 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ WORKLOAD_PIDS+=($!) 2024-09-12T13:05:59.141 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:05:59.141 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ slow_untar_workload test-primary7 2024-09-12T13:05:59.141 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ IMG=image-primary8 2024-09-12T13:05:59.141 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ MNTPT=test-primary8 2024-09-12T13:05:59.141 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ create_image_and_enable_mirror cluster1 mirror image-primary8 snapshot 10G 2024-09-12T13:05:59.141 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mountpt=test-primary7 2024-09-12T13:05:59.141 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ cp linux-5.4.tar.gz test-primary7 2024-09-12T13:05:59.142 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:05:59.142 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:05:59.142 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:05:59.142 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:05:59.142 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:05:59.142 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:05:59.142 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mode=snapshot 2024-09-12T13:05:59.142 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n snapshot ']' 2024-09-12T13:05:59.142 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:05:59.142 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ create_image cluster1 mirror image-primary8 10G 2024-09-12T13:05:59.142 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:05:59.142 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:05:59.142 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:05:59.142 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:05:59.142 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:05:59.143 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:05:59.143 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local size=128 2024-09-12T13:05:59.143 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n 10G ']' 2024-09-12T13:05:59.143 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ size=10G 2024-09-12T13:05:59.143 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:05:59.143 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 create --size 10G --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image-primary8 2024-09-12T13:05:59.207 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ is_pool_mirror_mode_image mirror 2024-09-12T13:05:59.207 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:05:59.207 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' image = image ']' 2024-09-12T13:05:59.207 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:05:59.207 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ enable_mirror cluster1 mirror image-primary8 snapshot 2024-09-12T13:05:59.207 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:05:59.208 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:05:59.208 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:05:59.208 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mode=snapshot 2024-09-12T13:05:59.208 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 mirror image enable mirror/image-primary8 snapshot 2024-09-12T13:05:59.427 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local ret=0 2024-09-12T13:05:59.427 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ timeout 5m bash -c 'zcat test-primary7/linux-5.4.tar.gz | pv -L 256K | tar xf - -C test-primary7' 2024-09-12T13:05:59.767 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Mirroring enabled 2024-09-12T13:05:59.771 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 349 2024-09-12T13:05:59.773 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 348 2024-09-12T13:05:59.782 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:05:59.782 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 info mirror/image-primary8 2024-09-12T13:05:59.782 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:05:59.822 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:05:59.823 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:05:59.823 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:05:59.823 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:05:59.823 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:05:59.823 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:05:59.843 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:rbd image 'image-primary8': 2024-09-12T13:05:59.843 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: size 10 GiB in 2560 objects 2024-09-12T13:05:59.843 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: order 22 (4 MiB objects) 2024-09-12T13:05:59.843 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: snapshot_count: 1 2024-09-12T13:05:59.843 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: id: 15e4b19eecfd 2024-09-12T13:05:59.843 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: block_name_prefix: rbd_data.15e4b19eecfd 2024-09-12T13:05:59.843 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: format: 2 2024-09-12T13:05:59.843 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: features: layering, exclusive-lock, object-map, fast-diff 2024-09-12T13:05:59.843 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: op_features: 2024-09-12T13:05:59.844 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: flags: 2024-09-12T13:05:59.844 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: create_timestamp: Thu Sep 12 13:05:59 2024 2024-09-12T13:05:59.844 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: access_timestamp: Thu Sep 12 13:05:59 2024 2024-09-12T13:05:59.844 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: modify_timestamp: Thu Sep 12 13:05:59 2024 2024-09-12T13:05:59.844 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring state: enabled 2024-09-12T13:05:59.844 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring mode: snapshot 2024-09-12T13:05:59.844 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring global id: fa7b2037-727d-4ca0-abbc-a51e250f5ecf 2024-09-12T13:05:59.844 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring primary: true 2024-09-12T13:05:59.848 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \n\b\d ]] 2024-09-12T13:05:59.848 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \k\r\b\d ]] 2024-09-12T13:05:59.849 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo rbd --cluster cluster1 device map -t krbd mirror/image-primary8 2024-09-12T13:05:59.978 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEV=/dev/rbd7 2024-09-12T13:05:59.978 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEVS+=($DEV) 2024-09-12T13:05:59.978 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo mkfs.ext4 /dev/rbd7 2024-09-12T13:05:59.999 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:mke2fs 1.46.5 (30-Dec-2021) 2024-09-12T13:06:00.013 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Discarding device blocks: 0/2621440 done 2024-09-12T13:06:00.013 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Creating filesystem with 2621440 4k blocks and 655360 inodes 2024-09-12T13:06:00.013 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Filesystem UUID: 13bbef67-1e72-49e4-891f-003bf4834b12 2024-09-12T13:06:00.013 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Superblock backups stored on blocks: 2024-09-12T13:06:00.013 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 2024-09-12T13:06:00.013 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 2024-09-12T13:06:00.013 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Allocating group tables: 0/80 done 2024-09-12T13:06:00.013 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Writing inode tables: 0/80 done 2024-09-12T13:06:00.018 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Creating journal (16384 blocks): done 2024-09-12T13:06:00.060 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Writing superblocks and filesystem accounting information: 0/80 done 2024-09-12T13:06:00.060 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 2024-09-12T13:06:00.062 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mkdir test-primary8 2024-09-12T13:06:00.065 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo mount /dev/rbd7 test-primary8 2024-09-12T13:06:00.123 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ whoami 2024-09-12T13:06:00.124 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo chown ubuntu test-primary8 2024-09-12T13:06:00.149 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ SNAP_PIDS+=($!) 2024-09-12T13:06:00.149 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ take_mirror_snapshots cluster1 mirror image-primary8 2024-09-12T13:06:00.149 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:06:00.149 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:06:00.149 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:06:00.149 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:06:00.150 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:06:00.150 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:06:00.150 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:06:00.150 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:06:00.150 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ WORKLOAD_PIDS+=($!) 2024-09-12T13:06:00.150 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:06:00.150 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:06:00.150 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ IMG=image-primary9 2024-09-12T13:06:00.150 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ MNTPT=test-primary9 2024-09-12T13:06:00.150 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ create_image_and_enable_mirror cluster1 mirror image-primary9 snapshot 10G 2024-09-12T13:06:00.150 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:06:00.150 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:06:00.150 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:06:00.150 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:06:00.151 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:06:00.151 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:06:00.151 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mode=snapshot 2024-09-12T13:06:00.151 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n snapshot ']' 2024-09-12T13:06:00.151 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:06:00.151 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ create_image cluster1 mirror image-primary9 10G 2024-09-12T13:06:00.151 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ slow_untar_workload test-primary8 2024-09-12T13:06:00.151 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:06:00.151 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:06:00.151 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:06:00.151 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:06:00.151 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:06:00.151 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:06:00.151 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local size=128 2024-09-12T13:06:00.152 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n 10G ']' 2024-09-12T13:06:00.152 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ size=10G 2024-09-12T13:06:00.152 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:06:00.152 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 create --size 10G --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image-primary9 2024-09-12T13:06:00.152 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mountpt=test-primary8 2024-09-12T13:06:00.152 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ cp linux-5.4.tar.gz test-primary8 2024-09-12T13:06:00.216 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ is_pool_mirror_mode_image mirror 2024-09-12T13:06:00.217 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:06:00.217 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' image = image ']' 2024-09-12T13:06:00.217 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:06:00.217 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ enable_mirror cluster1 mirror image-primary9 snapshot 2024-09-12T13:06:00.217 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:06:00.217 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:06:00.217 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:06:00.217 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mode=snapshot 2024-09-12T13:06:00.217 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 mirror image enable mirror/image-primary9 snapshot 2024-09-12T13:06:00.437 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local ret=0 2024-09-12T13:06:00.437 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ timeout 5m bash -c 'zcat test-primary8/linux-5.4.tar.gz | pv -L 256K | tar xf - -C test-primary8' 2024-09-12T13:06:00.777 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Mirroring enabled 2024-09-12T13:06:00.779 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 351 2024-09-12T13:06:00.784 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:06:00.784 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:06:00.784 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:06:00.784 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:06:00.785 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:06:00.785 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:06:00.785 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 352 2024-09-12T13:06:00.785 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:06:00.786 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:06:00.786 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:06:00.786 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:06:00.786 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:06:00.786 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:06:00.801 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:06:00.805 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 info mirror/image-primary9 2024-09-12T13:06:00.813 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:06:00.889 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:rbd image 'image-primary9': 2024-09-12T13:06:00.889 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: size 10 GiB in 2560 objects 2024-09-12T13:06:00.890 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: order 22 (4 MiB objects) 2024-09-12T13:06:00.890 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: snapshot_count: 1 2024-09-12T13:06:00.890 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: id: 15f68bccda68 2024-09-12T13:06:00.890 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: block_name_prefix: rbd_data.15f68bccda68 2024-09-12T13:06:00.890 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: format: 2 2024-09-12T13:06:00.890 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: features: layering, exclusive-lock, object-map, fast-diff 2024-09-12T13:06:00.890 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: op_features: 2024-09-12T13:06:00.890 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: flags: 2024-09-12T13:06:00.890 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: create_timestamp: Thu Sep 12 13:06:00 2024 2024-09-12T13:06:00.890 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: access_timestamp: Thu Sep 12 13:06:00 2024 2024-09-12T13:06:00.891 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: modify_timestamp: Thu Sep 12 13:06:00 2024 2024-09-12T13:06:00.891 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring state: enabled 2024-09-12T13:06:00.891 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring mode: snapshot 2024-09-12T13:06:00.891 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring global id: 77edc192-6758-44c1-b93f-e8f7df97992a 2024-09-12T13:06:00.891 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring primary: true 2024-09-12T13:06:00.896 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \n\b\d ]] 2024-09-12T13:06:00.896 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \k\r\b\d ]] 2024-09-12T13:06:00.897 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo rbd --cluster cluster1 device map -t krbd mirror/image-primary9 2024-09-12T13:06:01.014 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEV=/dev/rbd8 2024-09-12T13:06:01.015 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEVS+=($DEV) 2024-09-12T13:06:01.015 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo mkfs.ext4 /dev/rbd8 2024-09-12T13:06:01.035 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:mke2fs 1.46.5 (30-Dec-2021) 2024-09-12T13:06:01.048 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Discarding device blocks: 0/2621440 done 2024-09-12T13:06:01.048 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Creating filesystem with 2621440 4k blocks and 655360 inodes 2024-09-12T13:06:01.048 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Filesystem UUID: ec612728-ef05-4300-b089-7b71320d9e7c 2024-09-12T13:06:01.048 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Superblock backups stored on blocks: 2024-09-12T13:06:01.048 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 2024-09-12T13:06:01.048 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 2024-09-12T13:06:01.048 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Allocating group tables: 0/80 done 2024-09-12T13:06:01.048 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Writing inode tables: 0/80 done 2024-09-12T13:06:01.053 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Creating journal (16384 blocks): done 2024-09-12T13:06:01.093 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Writing superblocks and filesystem accounting information: 0/80 done 2024-09-12T13:06:01.093 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 2024-09-12T13:06:01.096 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mkdir test-primary9 2024-09-12T13:06:01.099 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo mount /dev/rbd8 test-primary9 2024-09-12T13:06:01.204 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ whoami 2024-09-12T13:06:01.206 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo chown ubuntu test-primary9 2024-09-12T13:06:01.232 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ SNAP_PIDS+=($!) 2024-09-12T13:06:01.232 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ take_mirror_snapshots cluster1 mirror image-primary9 2024-09-12T13:06:01.232 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:06:01.232 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:06:01.232 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:06:01.232 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:06:01.232 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:06:01.233 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:06:01.233 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:06:01.233 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:06:01.233 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:06:01.233 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ WORKLOAD_PIDS+=($!) 2024-09-12T13:06:01.233 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:06:01.233 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ IMG=image-primary10 2024-09-12T13:06:01.233 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ MNTPT=test-primary10 2024-09-12T13:06:01.233 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ create_image_and_enable_mirror cluster1 mirror image-primary10 snapshot 10G 2024-09-12T13:06:01.233 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ slow_untar_workload test-primary9 2024-09-12T13:06:01.233 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:06:01.233 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:06:01.233 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:06:01.233 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:06:01.234 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:06:01.234 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:06:01.234 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mode=snapshot 2024-09-12T13:06:01.234 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n snapshot ']' 2024-09-12T13:06:01.234 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:06:01.234 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ create_image cluster1 mirror image-primary10 10G 2024-09-12T13:06:01.234 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:06:01.234 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:06:01.234 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:06:01.234 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mountpt=test-primary9 2024-09-12T13:06:01.234 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:06:01.234 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:06:01.234 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:06:01.234 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local size=128 2024-09-12T13:06:01.235 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n 10G ']' 2024-09-12T13:06:01.235 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ cp linux-5.4.tar.gz test-primary9 2024-09-12T13:06:01.235 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ size=10G 2024-09-12T13:06:01.235 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:06:01.235 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 create --size 10G --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image-primary10 2024-09-12T13:06:01.301 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ is_pool_mirror_mode_image mirror 2024-09-12T13:06:01.301 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:06:01.301 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' image = image ']' 2024-09-12T13:06:01.301 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:06:01.301 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ enable_mirror cluster1 mirror image-primary10 snapshot 2024-09-12T13:06:01.301 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:06:01.302 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:06:01.302 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:06:01.302 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mode=snapshot 2024-09-12T13:06:01.302 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 mirror image enable mirror/image-primary10 snapshot 2024-09-12T13:06:01.525 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local ret=0 2024-09-12T13:06:01.525 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ timeout 5m bash -c 'zcat test-primary9/linux-5.4.tar.gz | pv -L 256K | tar xf - -C test-primary9' 2024-09-12T13:06:01.785 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:06:01.785 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:06:01.785 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:06:01.785 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:06:01.785 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:06:01.785 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:06:01.790 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:06:01.790 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:06:01.790 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:06:01.790 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:06:01.790 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:06:01.790 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:06:01.795 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Mirroring enabled 2024-09-12T13:06:01.811 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 354 2024-09-12T13:06:01.811 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 356 2024-09-12T13:06:01.815 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 355 2024-09-12T13:06:01.828 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:06:01.828 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 info mirror/image-primary10 2024-09-12T13:06:01.840 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:06:01.859 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:06:01.918 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:rbd image 'image-primary10': 2024-09-12T13:06:01.919 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: size 10 GiB in 2560 objects 2024-09-12T13:06:01.919 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: order 22 (4 MiB objects) 2024-09-12T13:06:01.919 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: snapshot_count: 1 2024-09-12T13:06:01.919 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: id: 160b4ff77494 2024-09-12T13:06:01.919 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: block_name_prefix: rbd_data.160b4ff77494 2024-09-12T13:06:01.919 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: format: 2 2024-09-12T13:06:01.919 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: features: layering, exclusive-lock, object-map, fast-diff 2024-09-12T13:06:01.919 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: op_features: 2024-09-12T13:06:01.919 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: flags: 2024-09-12T13:06:01.919 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: create_timestamp: Thu Sep 12 13:06:01 2024 2024-09-12T13:06:01.919 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: access_timestamp: Thu Sep 12 13:06:01 2024 2024-09-12T13:06:01.919 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: modify_timestamp: Thu Sep 12 13:06:01 2024 2024-09-12T13:06:01.919 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring state: enabled 2024-09-12T13:06:01.920 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring mode: snapshot 2024-09-12T13:06:01.920 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring global id: 2286adb1-ff3d-442a-9aac-6b38a8513aaa 2024-09-12T13:06:01.920 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring primary: true 2024-09-12T13:06:01.924 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \n\b\d ]] 2024-09-12T13:06:01.924 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \k\r\b\d ]] 2024-09-12T13:06:01.925 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo rbd --cluster cluster1 device map -t krbd mirror/image-primary10 2024-09-12T13:06:02.078 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEV=/dev/rbd9 2024-09-12T13:06:02.078 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEVS+=($DEV) 2024-09-12T13:06:02.078 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo mkfs.ext4 /dev/rbd9 2024-09-12T13:06:02.099 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:mke2fs 1.46.5 (30-Dec-2021) 2024-09-12T13:06:02.111 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Discarding device blocks: 0/2621440 done 2024-09-12T13:06:02.111 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Creating filesystem with 2621440 4k blocks and 655360 inodes 2024-09-12T13:06:02.111 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Filesystem UUID: 66dc8125-f2e6-4b4a-8a1f-adf2e7a1f433 2024-09-12T13:06:02.111 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Superblock backups stored on blocks: 2024-09-12T13:06:02.112 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 2024-09-12T13:06:02.112 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 2024-09-12T13:06:02.112 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Allocating group tables: 0/80 done 2024-09-12T13:06:02.112 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Writing inode tables: 0/80 done 2024-09-12T13:06:02.118 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Creating journal (16384 blocks): done 2024-09-12T13:06:02.260 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Writing superblocks and filesystem accounting information: 0/80 done 2024-09-12T13:06:02.260 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 2024-09-12T13:06:02.263 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mkdir test-primary10 2024-09-12T13:06:02.265 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo mount /dev/rbd9 test-primary10 2024-09-12T13:06:02.342 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ whoami 2024-09-12T13:06:02.344 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo chown ubuntu test-primary10 2024-09-12T13:06:02.368 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ SNAP_PIDS+=($!) 2024-09-12T13:06:02.369 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ take_mirror_snapshots cluster1 mirror image-primary10 2024-09-12T13:06:02.369 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:06:02.369 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:06:02.369 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:06:02.369 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:06:02.369 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:06:02.369 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:06:02.369 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:06:02.369 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:06:02.369 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ WORKLOAD_PIDS+=($!) 2024-09-12T13:06:02.369 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:06:02.369 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${SNAP_PIDS[@]} 2024-09-12T13:06:02.369 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 47165 2024-09-12T13:06:02.370 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ slow_untar_workload test-primary10 2024-09-12T13:06:02.370 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mountpt=test-primary10 2024-09-12T13:06:02.370 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ cp linux-5.4.tar.gz test-primary10 2024-09-12T13:06:02.673 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local ret=0 2024-09-12T13:06:02.674 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ timeout 5m bash -c 'zcat test-primary10/linux-5.4.tar.gz | pv -L 256K | tar xf - -C test-primary10' 2024-09-12T13:06:02.785 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:06:02.785 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:06:02.785 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:06:02.785 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:06:02.785 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:06:02.785 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:06:02.786 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:06:02.786 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:06:02.787 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:06:02.787 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:06:02.787 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:06:02.787 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:06:02.808 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 359 2024-09-12T13:06:02.825 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 360 2024-09-12T13:06:02.841 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 358 2024-09-12T13:06:02.862 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:06:02.868 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:06:02.874 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:06:03.805 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:06:03.805 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:06:03.805 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:06:03.806 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:06:03.806 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:06:03.806 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:06:03.816 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:06:03.818 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:06:03.818 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:06:03.818 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:06:03.818 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:06:03.818 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:06:03.827 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 361 2024-09-12T13:06:03.849 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 362 2024-09-12T13:06:03.857 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:06:03.874 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:06:04.832 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:06:04.832 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:06:04.832 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:06:04.832 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:06:04.832 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:06:04.832 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:06:04.842 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:06:04.845 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:06:04.846 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:06:04.846 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:06:04.846 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:06:04.846 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:06:04.851 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 364 2024-09-12T13:06:04.864 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:06:04.864 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:06:04.864 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:06:04.864 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:06:04.864 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:06:04.865 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:06:04.868 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 363 2024-09-12T13:06:04.880 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:06:04.895 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:06:05.869 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:06:05.869 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:06:05.869 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:06:05.869 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:06:05.869 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:06:05.870 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:06:05.873 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 366 2024-09-12T13:06:05.873 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:06:05.874 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:06:05.874 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:06:05.874 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:06:05.874 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:06:05.874 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:06:05.879 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:06:05.879 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:06:05.879 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:06:05.879 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:06:05.879 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:06:05.879 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:06:05.879 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 367 2024-09-12T13:06:05.880 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 365 2024-09-12T13:06:05.915 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:06:05.918 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:06:05.927 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:06:06.867 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:06:06.867 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:06:06.867 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:06:06.867 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:06:06.867 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:06:06.867 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:06:06.874 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 369 2024-09-12T13:06:06.882 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:06:06.882 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:06:06.882 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:06:06.882 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:06:06.882 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:06:06.882 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:06:06.889 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 370 2024-09-12T13:06:06.892 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 368 2024-09-12T13:06:06.918 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:06:06.929 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:06:06.943 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:06:07.773 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 372 2024-09-12T13:06:07.775 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 371 2024-09-12T13:06:07.796 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:06:07.810 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:06:07.884 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:06:07.884 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:06:07.884 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:06:07.885 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:06:07.885 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:06:07.885 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:06:07.902 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:06:07.902 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:06:07.902 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:06:07.902 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:06:07.903 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:06:07.903 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:06:08.766 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 374 2024-09-12T13:06:08.777 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 373 2024-09-12T13:06:08.789 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:06:08.797 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:06:08.925 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:06:08.925 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:06:08.925 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:06:08.925 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:06:08.926 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:06:08.926 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:06:08.926 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:06:08.926 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:06:08.926 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:06:08.926 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:06:08.926 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:06:08.926 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:06:08.934 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:06:08.934 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:06:08.934 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:06:08.934 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:06:08.935 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:06:08.935 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:06:09.834 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 377 2024-09-12T13:06:09.836 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 376 2024-09-12T13:06:09.852 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 375 2024-09-12T13:06:09.879 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:06:09.882 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:06:09.886 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:06:09.925 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:06:09.925 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:06:09.925 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:06:09.925 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:06:09.925 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:06:09.925 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:06:09.938 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:06:09.939 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:06:09.939 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:06:09.939 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:06:09.939 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:06:09.939 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:06:09.950 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:06:09.950 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:06:09.950 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:06:09.950 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:06:09.950 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:06:09.950 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:06:10.800 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:06:10.800 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:06:10.801 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:06:10.801 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:06:10.801 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:06:10.801 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:06:10.812 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:06:10.812 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:06:10.812 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:06:10.812 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:06:10.812 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:06:10.813 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:06:10.825 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 379 2024-09-12T13:06:10.884 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:06:10.891 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 378 2024-09-12T13:06:10.931 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:06:10.948 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 380 2024-09-12T13:06:10.991 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:06:11.785 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 382 2024-09-12T13:06:11.792 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:06:11.793 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:06:11.793 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:06:11.793 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:06:11.793 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:06:11.793 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:06:11.805 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:06:11.805 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:06:11.805 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:06:11.805 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:06:11.805 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:06:11.805 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:06:11.811 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 381 2024-09-12T13:06:11.828 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:06:11.851 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:06:12.799 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 383 2024-09-12T13:06:12.804 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 384 2024-09-12T13:06:12.816 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:06:12.831 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:06:12.884 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:06:12.884 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:06:12.885 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:06:12.885 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:06:12.885 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:06:12.885 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:06:12.887 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:06:12.888 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:06:12.888 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:06:12.889 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:06:12.889 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:06:12.889 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:06:12.889 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:06:12.889 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:06:12.889 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:06:12.889 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:06:12.889 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:06:12.889 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:06:13.806 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 385 2024-09-12T13:06:13.832 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 387 2024-09-12T13:06:13.836 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:06:13.848 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:06:13.895 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:06:13.895 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:06:13.895 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:06:13.895 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:06:13.895 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:06:13.896 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:06:13.939 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:06:13.940 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:06:13.941 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:06:13.941 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:06:13.941 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:06:13.941 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:06:13.995 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:06:13.995 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:06:13.995 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:06:13.995 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:06:13.995 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:06:13.995 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:06:14.827 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 388 2024-09-12T13:06:14.834 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:06:14.834 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:06:14.834 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:06:14.834 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:06:14.834 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:06:14.834 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:06:14.860 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:06:14.860 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:06:14.860 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:06:14.860 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:06:14.860 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:06:14.860 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:06:14.961 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:06:14.965 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 389 2024-09-12T13:06:14.983 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:06:15.782 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 386 2024-09-12T13:06:15.820 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:06:15.820 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:06:15.820 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:06:15.821 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:06:15.821 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:06:15.821 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:06:15.821 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:06:15.836 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:06:15.836 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:06:15.836 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:06:15.836 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:06:15.836 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:06:15.836 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:06:15.865 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 390 2024-09-12T13:06:15.890 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:06:15.918 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 391 2024-09-12T13:06:16.017 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:06:16.820 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 392 2024-09-12T13:06:16.840 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:06:16.840 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:06:16.840 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:06:16.841 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:06:16.841 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:06:16.841 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:06:16.843 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:06:16.854 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:06:16.854 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:06:16.854 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:06:16.854 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:06:16.855 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:06:16.855 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:06:16.863 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 393 2024-09-12T13:06:16.877 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:06:17.790 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 394 2024-09-12T13:06:17.813 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:06:17.853 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 395 2024-09-12T13:06:17.863 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:06:17.964 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:06:17.964 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:06:17.964 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:06:17.964 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:06:17.964 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:06:17.964 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:06:17.986 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:06:17.986 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:06:17.986 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:06:17.986 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:06:17.986 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:06:17.986 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:06:18.824 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:06:18.824 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:06:18.824 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:06:18.824 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:06:18.825 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:06:18.825 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:06:18.859 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 397 2024-09-12T13:06:18.888 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:06:18.894 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:06:18.894 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:06:18.894 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:06:18.895 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:06:18.895 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:06:18.895 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:06:19.021 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:06:19.021 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:06:19.021 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:06:19.021 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:06:19.021 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:06:19.021 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:06:19.798 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 396 2024-09-12T13:06:19.836 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:06:19.846 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:06:19.846 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:06:19.846 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:06:19.846 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:06:19.847 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:06:19.847 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:06:19.865 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 399 2024-09-12T13:06:19.879 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:06:19.880 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:06:19.880 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:06:19.880 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:06:19.880 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:06:19.880 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:06:19.880 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:06:20.817 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 398 2024-09-12T13:06:20.821 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:06:20.821 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:06:20.821 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:06:20.821 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:06:20.821 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:06:20.822 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:06:20.868 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:06:20.868 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:06:20.868 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:06:20.868 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:06:20.869 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:06:20.869 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:06:20.888 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:06:20.929 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 400 2024-09-12T13:06:20.932 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:06:20.932+0000 7f67dd000640 -1 rbd::mirror::image_replayer::snapshot::Replayer: 0x560556cd5400 handle_unlink_peer: failed to unlink local peer from remote image: (22) Invalid argument 2024-09-12T13:06:20.947 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:06:21.809 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 401 2024-09-12T13:06:21.856 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 403 2024-09-12T13:06:21.859 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:06:21.862 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 402 2024-09-12T13:06:21.875 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:06:21.891 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:06:21.891 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:06:21.891 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:06:21.892 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:06:21.892 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:06:21.892 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:06:21.969 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:06:22.839 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:06:22.839 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:06:22.839 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:06:22.840 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:06:22.840 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:06:22.840 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:06:22.882 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:06:22.882 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:06:22.882 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:06:22.882 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:06:22.882 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:06:22.882 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:06:23.818 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 404 2024-09-12T13:06:23.824 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 405 2024-09-12T13:06:23.855 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:06:23.861 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:06:23.896 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:06:23.897 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:06:23.897 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:06:23.897 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:06:23.897 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:06:23.897 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:06:23.943 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:06:23.944+0000 7f67dc600640 -1 rbd::mirror::image_replayer::snapshot::Replayer: 0x56055c2b2000 handle_unlink_peer: failed to unlink local peer from remote image: (22) Invalid argument 2024-09-12T13:06:23.950 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:06:23.950 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:06:23.950 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:06:23.950 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:06:23.950 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:06:23.951 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:06:23.969 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 407 2024-09-12T13:06:23.986 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:06:24.812 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 406 2024-09-12T13:06:24.845 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:06:24.862 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:06:24.862 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:06:24.862 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:06:24.862 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:06:24.862 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:06:24.862 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:06:24.878 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:06:24.878 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:06:24.879 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:06:24.879 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:06:24.879 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:06:24.879 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:06:24.908 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 409 2024-09-12T13:06:24.915 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:06:24.971 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:06:24.971 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:06:24.971 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:06:24.971 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:06:24.971 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:06:24.972 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:06:25.833 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 408 2024-09-12T13:06:25.896 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:06:25.924 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 410 2024-09-12T13:06:25.947 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:06:25.947 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 411 2024-09-12T13:06:25.999 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:06:26.858 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:06:26.859 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:06:26.859 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:06:26.859 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:06:26.859 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:06:26.859 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:06:26.864 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:06:26.865 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:06:26.865 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:06:26.865 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:06:26.865 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:06:26.865 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:06:26.895 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 412 2024-09-12T13:06:26.924 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 413 2024-09-12T13:06:26.946 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:06:26.969 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:06:26.989 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:06:26.989 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:06:26.989 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:06:26.989 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:06:26.989 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:06:26.989 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:06:27.777 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 415 2024-09-12T13:06:27.790 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:06:27.793 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 414 2024-09-12T13:06:27.810 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:06:27.849 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:06:27.849 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:06:27.850 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:06:27.850 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:06:27.850 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:06:27.850 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:06:27.919 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:06:27.922 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:06:27.922 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:06:27.923 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:06:27.923 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:06:27.923 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:06:28.795 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 417 2024-09-12T13:06:28.824 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:06:28.899 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:06:28.899 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:06:28.899 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:06:28.899 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:06:28.899 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:06:28.900 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:06:28.951 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:06:28.951 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:06:28.952 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:06:28.952 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:06:28.952 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:06:28.952 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:06:29.004 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:06:29.005 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:06:29.005 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:06:29.005 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:06:29.005 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:06:29.005 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:06:29.750 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 416 2024-09-12T13:06:29.790 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:06:29.867 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 421 2024-09-12T13:06:29.871 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 419 2024-09-12T13:06:29.877 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:06:29.882 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:06:29.958 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:06:29.958 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:06:29.958 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:06:29.959 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:06:29.959 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:06:29.959 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:06:29.973 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:06:29.973 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:06:29.973 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:06:29.973 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:06:29.973 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:06:29.974 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:06:30.759 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 418 2024-09-12T13:06:30.792 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:06:30.793 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:06:30.793 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:06:30.793 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:06:30.793 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:06:30.793 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:06:30.793 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:06:30.807 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 420 2024-09-12T13:06:30.813 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:06:30.813 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:06:30.813 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:06:30.814 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:06:30.814 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:06:30.814 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:06:30.821 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:06:31.825 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 423 2024-09-12T13:06:31.827 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:06:31.827 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:06:31.828 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:06:31.828 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:06:31.828 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:06:31.828 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:06:31.854 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:06:32.747 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 422 2024-09-12T13:06:32.778 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:06:32.793 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:06:32.793 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:06:32.793 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:06:32.793 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:06:32.794 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:06:32.794 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:06:32.799 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 425 2024-09-12T13:06:32.806 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 424 2024-09-12T13:06:32.827 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:06:32.853 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:06:32.854 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 426 2024-09-12T13:06:32.871 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:06:32.881 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:06:32.881 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:06:32.881 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:06:32.882 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:06:32.882 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:06:32.882 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:06:32.886 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:06:32.886 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:06:32.886 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:06:32.886 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:06:32.886 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:06:32.887 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:06:33.794 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:06:33.795 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:06:33.795 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:06:33.795 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:06:33.795 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:06:33.795 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:06:33.825 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:06:33.826 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:06:33.826 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:06:33.826 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:06:33.826 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:06:33.826 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:06:33.848 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 427 2024-09-12T13:06:33.883 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 429 2024-09-12T13:06:33.892 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:06:33.908 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:06:34.861 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:06:34.861 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:06:34.861 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:06:34.861 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:06:34.861 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:06:34.861 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:06:34.876 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 430 2024-09-12T13:06:34.889 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:06:35.764 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 428 2024-09-12T13:06:35.781 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:06:35.782 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:06:35.782 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:06:35.782 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:06:35.782 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:06:35.782 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:06:35.801 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:06:35.821 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 431 2024-09-12T13:06:35.832 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:06:35.832 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:06:35.832 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:06:35.833 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:06:35.833 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:06:35.833 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:06:35.834 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:06:35.858 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:06:35.858 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:06:35.858 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:06:35.858 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:06:35.859 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:06:35.859 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:06:35.880 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:06:35.880 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:06:35.880 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:06:35.880 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:06:35.881 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:06:35.881 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:06:36.842 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 436 2024-09-12T13:06:36.858 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 433 2024-09-12T13:06:36.877 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:06:36.881 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:06:36.900 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:06:36.900 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:06:36.901 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:06:36.901 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:06:36.901 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:06:36.901 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:06:36.911 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:06:36.911 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:06:36.912 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:06:36.912 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:06:36.912 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:06:36.912 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:06:37.753 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 432 2024-09-12T13:06:37.808 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:06:37.818 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 434 2024-09-12T13:06:37.850 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:06:37.852 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 438 2024-09-12T13:06:37.869 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:06:37.892 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:06:37.893 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:06:37.893 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:06:37.893 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:06:37.893 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:06:37.893 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:06:38.776 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 435 2024-09-12T13:06:38.801 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:06:38.803 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:06:38.803 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:06:38.803 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:06:38.804 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:06:38.804 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:06:38.804 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:06:38.839 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:06:38.839 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:06:38.839 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:06:38.839 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:06:38.839 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:06:38.839 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:06:39.791 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 437 2024-09-12T13:06:39.822 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:06:39.871 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 440 2024-09-12T13:06:39.878 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:06:39.881 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:06:39.881 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:06:39.881 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:06:39.881 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:06:39.881 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:06:39.882 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:06:39.886 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:06:39.886 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:06:39.886 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:06:39.886 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:06:39.886 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:06:39.886 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:06:40.779 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 439 2024-09-12T13:06:40.809 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:06:40.809 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:06:40.809 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:06:40.809 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:06:40.809 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:06:40.809 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:06:40.818 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:06:40.842 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 441 2024-09-12T13:06:40.852 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:06:40.853 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 442 2024-09-12T13:06:40.853 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:06:40.854 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:06:40.854 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:06:40.854 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:06:40.854 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:06:40.854 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:06:40.873 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:06:40.873 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:06:40.873 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:06:40.873 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:06:40.873 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:06:40.873 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:06:40.878 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:06:41.804 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:06:41.805 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:06:41.805 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:06:41.805 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:06:41.805 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:06:41.805 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:06:41.872 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 444 2024-09-12T13:06:41.915 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:06:42.800 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 443 2024-09-12T13:06:42.807 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 446 2024-09-12T13:06:42.825 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:06:42.825 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:06:42.825 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:06:42.825 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:06:42.825 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:06:42.825 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:06:42.845 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:06:42.855 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:06:42.883 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:06:42.883 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:06:42.883 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:06:42.883 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:06:42.883 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:06:42.883 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:06:42.922 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 447 2024-09-12T13:06:42.949 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:06:43.797 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 445 2024-09-12T13:06:43.818 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:06:43.821 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:06:43.821 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:06:43.821 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:06:43.821 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:06:43.821 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:06:43.821 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:06:43.837 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 448 2024-09-12T13:06:43.850 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:06:43.855 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:06:43.856 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:06:43.856 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:06:43.856 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:06:43.856 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:06:43.856 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:06:43.882 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:06:43.882 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:06:43.882 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:06:43.882 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:06:43.882 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:06:43.882 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:06:44.874 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 450 2024-09-12T13:06:44.881 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:06:44.922 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:06:44.923 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:06:44.923 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:06:44.923 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:06:44.923 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:06:44.923 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:06:45.803 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 451 2024-09-12T13:06:45.810 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 449 2024-09-12T13:06:45.831 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:06:45.846 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:06:45.848 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:06:45.848 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:06:45.848 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:06:45.848 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:06:45.848 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:06:45.849 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:06:45.857 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:06:45.858 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:06:45.858 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:06:45.858 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:06:45.858 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:06:45.858 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:06:45.953 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:06:45.953 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:06:45.953 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:06:45.953 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:06:45.953 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:06:45.954 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:06:46.814 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 452 2024-09-12T13:06:46.821 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:06:46.821 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:06:46.821 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:06:46.821 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:06:46.821 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:06:46.821 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:06:46.846 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:06:46.853 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:06:46.853 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:06:46.853 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:06:46.853 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:06:46.853 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:06:46.853 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:06:46.870 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 454 2024-09-12T13:06:46.888 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:06:47.779 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 453 2024-09-12T13:06:47.790 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 455 2024-09-12T13:06:47.812 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:06:47.816 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:06:47.837 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 457 2024-09-12T13:06:47.844 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:06:47.884 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:06:47.884 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:06:47.884 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:06:47.884 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:06:47.884 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:06:47.885 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:06:48.807 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 456 2024-09-12T13:06:48.833 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:06:48.834 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:06:48.834 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:06:48.834 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:06:48.834 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:06:48.834 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:06:48.834 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:06:48.849 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:06:48.849 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:06:48.849 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:06:48.849 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:06:48.850 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:06:48.850 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:06:49.823 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 458 2024-09-12T13:06:49.850 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:06:49.851 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:06:49.851 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:06:49.851 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:06:49.851 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:06:49.851 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:06:49.852 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:06:49.889 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 461 2024-09-12T13:06:49.891 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:06:49.891 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:06:49.892 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:06:49.892 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:06:49.892 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:06:49.892 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:06:49.901 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:06:50.816 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:06:50.816 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:06:50.816 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:06:50.816 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:06:50.816 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:06:50.816 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:06:50.819 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:06:50.819 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:06:50.819 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:06:50.819 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:06:50.819 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:06:50.819 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:06:50.823 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 459 2024-09-12T13:06:50.847 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:06:50.847 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:06:50.847 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:06:50.847 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:06:50.847 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:06:50.847 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:06:50.865 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:06:50.874 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 460 2024-09-12T13:06:50.892 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:06:50.896 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 462 2024-09-12T13:06:50.910 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:06:51.837 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:06:51.837 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:06:51.837 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:06:51.837 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:06:51.837 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:06:51.837 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:06:51.861 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 464 2024-09-12T13:06:51.874 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:06:52.839 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 463 2024-09-12T13:06:52.859 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:06:52.859 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:06:52.859 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:06:52.859 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:06:52.860 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:06:52.860 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:06:52.872 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:06:52.898 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 467 2024-09-12T13:06:52.905 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:06:52.906 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:06:52.906 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:06:52.906 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:06:52.906 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:06:52.906 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:06:52.912 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:06:53.845 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 466 2024-09-12T13:06:53.868 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:06:53.868 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:06:53.868 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:06:53.868 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:06:53.868 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:06:53.868 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:06:53.877 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:06:53.883 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 468 2024-09-12T13:06:53.895 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:06:53.895 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:06:53.895 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:06:53.895 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:06:53.895 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:06:53.895 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:06:53.897 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:06:53.913 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:06:53.913 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:06:53.913 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:06:53.913 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:06:53.913 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:06:53.913 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:06:54.831 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 465 2024-09-12T13:06:54.873 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:06:54.877 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:06:54.877 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:06:54.877 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:06:54.877 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:06:54.877 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:06:54.877 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:06:54.950 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 470 2024-09-12T13:06:54.960 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:06:55.853 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 469 2024-09-12T13:06:55.874 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:06:55.874 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:06:55.875 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:06:55.875 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:06:55.875 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:06:55.875 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:06:55.885 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:06:55.884+0000 7f67dd000640 -1 rbd::mirror::image_replayer::snapshot::Replayer: 0x56055694d800 handle_unlink_peer: failed to unlink local peer from remote image: (22) Invalid argument 2024-09-12T13:06:55.886 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:06:55.915 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:06:55.915 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:06:55.915 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:06:55.916 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:06:55.916 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:06:55.916 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:06:56.863 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 472 2024-09-12T13:06:56.880 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:06:56.881 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:06:56.881 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:06:56.881 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:06:56.881 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:06:56.881 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:06:56.896 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:06:56.899 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 474 2024-09-12T13:06:56.900 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:06:56.900 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:06:56.900 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:06:56.900 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:06:56.900 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:06:56.900 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:06:56.916 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:06:57.764 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 473 2024-09-12T13:06:57.818 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 471 2024-09-12T13:06:57.820 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:06:57.842 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 475 2024-09-12T13:06:57.846 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:06:57.853 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:06:57.868 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 476 2024-09-12T13:06:57.876 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:06:57.876 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:06:57.876 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:06:57.877 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:06:57.877 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:06:57.877 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:06:57.880 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:06:57.963 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:06:57.963 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:06:57.963 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:06:57.963 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:06:57.963 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:06:57.964 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:06:58.890 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:06:58.890 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:06:58.890 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:06:58.891 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:06:58.891 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:06:58.891 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:06:59.845 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 477 2024-09-12T13:06:59.884 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:06:59.898 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:06:59.899 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:06:59.899 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:06:59.899 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:06:59.899 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:06:59.899 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:06:59.918 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:06:59.919 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:06:59.919 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:06:59.919 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:06:59.919 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:06:59.919 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:06:59.934 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 480 2024-09-12T13:06:59.943 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:07:00.824 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:07:00.824 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:07:00.824 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:07:00.824 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:07:00.824 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:07:00.824 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:07:00.849 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:07:00.849 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:07:00.849 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:07:00.849 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:07:00.850 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:07:00.850 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:07:00.855 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:07:00.855 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:07:00.855 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:07:00.855 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:07:00.855 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:07:00.855 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:07:00.863 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 479 2024-09-12T13:07:00.883 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:07:00.883 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:07:00.883 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:07:00.883 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:07:00.883 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:07:00.883 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:07:00.903 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:07:00.926 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 481 2024-09-12T13:07:00.941 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:07:01.868 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 478 2024-09-12T13:07:01.894 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:07:01.925 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 483 2024-09-12T13:07:01.932 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:07:02.765 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 482 2024-09-12T13:07:02.813 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:07:02.853 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 486 2024-09-12T13:07:02.857 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 485 2024-09-12T13:07:02.876 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:07:02.877 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:07:02.886 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:07:02.886 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:07:02.887 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:07:02.887 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:07:02.887 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:07:02.887 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:07:02.946 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:07:02.946 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:07:02.946 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:07:02.946 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:07:02.946 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:07:02.946 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:07:03.906 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:07:03.906 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:07:03.906 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:07:03.906 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:07:03.907 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:07:03.907 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:07:03.911 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 487 2024-09-12T13:07:03.922 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:07:03.945 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:07:03.945 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:07:03.945 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:07:03.945 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:07:03.946 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:07:03.946 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:07:04.867 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 484 2024-09-12T13:07:04.897 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:07:04.898 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:07:04.898 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:07:04.898 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:07:04.898 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:07:04.898 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:07:04.898 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:07:04.913 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 490 2024-09-12T13:07:04.928 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:07:04.935 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:07:04.935 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:07:04.935 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:07:04.935 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:07:04.935 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:07:04.935 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:07:04.936 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 489 2024-09-12T13:07:04.946 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:07:05.821 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:07:05.822 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:07:05.822 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:07:05.822 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:07:05.822 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:07:05.822 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:07:05.870 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 488 2024-09-12T13:07:05.881 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:07:05.881 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:07:05.881 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:07:05.881 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:07:05.881 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:07:05.881 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:07:05.881 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:07:05.882 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:07:05.882 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:07:05.882 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:07:05.882 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:07:05.882 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:07:05.888 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:07:05.917 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 491 2024-09-12T13:07:05.926 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:07:06.922 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 493 2024-09-12T13:07:06.925 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:07:06.925 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:07:06.925 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:07:06.925 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:07:06.925 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:07:06.925 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:07:06.935 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:07:07.757 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 492 2024-09-12T13:07:07.796 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:07:07.815 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 495 2024-09-12T13:07:07.829 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 494 2024-09-12T13:07:07.836 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:07:07.847 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:07:07.900 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:07:07.900 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:07:07.901 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:07:07.901 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:07:07.901 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:07:07.901 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:07:07.931 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:07:07.931 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:07:07.932 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:07:07.932 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:07:07.932 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:07:07.932 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:07:07.948 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:07:07.949 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:07:07.949 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:07:07.949 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:07:07.949 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:07:07.949 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:07:08.796 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 496 2024-09-12T13:07:08.809 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 497 2024-09-12T13:07:08.815 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 498 2024-09-12T13:07:08.818 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:07:08.833 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:07:08.835 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:07:08.891 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:07:08.892 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:07:08.892 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:07:08.892 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:07:08.892 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:07:08.892 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:07:08.929 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:07:08.929 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:07:08.930 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:07:08.930 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:07:08.930 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:07:08.930 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:07:09.807 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 500 2024-09-12T13:07:09.816 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:07:09.940 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:07:09.940 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:07:09.941 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:07:09.941 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:07:09.941 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:07:09.941 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:07:10.776 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 499 2024-09-12T13:07:10.798 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:07:10.800 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:07:10.800 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:07:10.800 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:07:10.800 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:07:10.801 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:07:10.801 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:07:10.839 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:07:10.839 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:07:10.839 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:07:10.839 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:07:10.839 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:07:10.839 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:07:10.850 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:07:10.850 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:07:10.850 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:07:10.850 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:07:10.851 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:07:10.851 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:07:11.784 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 501 2024-09-12T13:07:11.805 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 502 2024-09-12T13:07:11.816 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:07:11.821 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:07:11.822 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:07:11.822 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:07:11.822 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:07:11.822 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:07:11.822 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:07:11.837 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:07:11.839 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:07:11.839 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:07:11.839 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:07:11.839 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:07:11.839 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:07:11.839 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:07:11.839 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:07:11.839 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:07:11.839 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:07:11.839 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:07:11.839 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:07:11.854 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:07:11.859 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 503 2024-09-12T13:07:11.881 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:07:12.819 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 505 2024-09-12T13:07:12.819 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:07:12.819 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:07:12.820 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:07:12.820 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:07:12.820 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:07:12.820 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:07:12.831 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 504 2024-09-12T13:07:12.833 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 506 2024-09-12T13:07:12.839 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:07:12.847 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:07:12.854 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:07:13.801 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:07:13.801 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:07:13.802 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:07:13.802 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:07:13.802 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:07:13.802 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:07:13.821 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 508 2024-09-12T13:07:13.832 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 507 2024-09-12T13:07:13.846 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:07:13.859 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:07:14.819 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:07:14.819 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:07:14.819 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:07:14.819 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:07:14.820 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:07:14.820 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:07:14.824 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 510 2024-09-12T13:07:14.834 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:07:14.857 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:07:14.857 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:07:14.857 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:07:14.857 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:07:14.858 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:07:14.858 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:07:14.884 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:07:14.884 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:07:14.884 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:07:14.885 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:07:14.885 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:07:14.885 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:07:15.773 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 509 2024-09-12T13:07:15.796 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:07:15.825 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 511 2024-09-12T13:07:15.832 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:07:15.842 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:07:15.843 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:07:15.843 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:07:15.843 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:07:15.843 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:07:15.843 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:07:15.852 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:07:15.852 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:07:15.852 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:07:15.852 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:07:15.852 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:07:15.852 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:07:15.857 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:07:15.857 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:07:15.858 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:07:15.858 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:07:15.858 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:07:15.858 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:07:16.850 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:07:16.850 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:07:16.850 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:07:16.850 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:07:16.850 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:07:16.850 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:07:16.862 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:07:16.862 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:07:16.862 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:07:16.862 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:07:16.862 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:07:16.862 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:07:17.754 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 513 2024-09-12T13:07:17.773 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 515 2024-09-12T13:07:17.788 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 512 2024-09-12T13:07:17.794 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:07:17.810 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:07:17.819 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:07:17.837 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:07:17.837 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:07:17.837 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:07:17.838 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:07:17.838 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:07:17.838 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:07:18.795 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 516 2024-09-12T13:07:18.799 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:07:18.799 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:07:18.799 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:07:18.800 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:07:18.800 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:07:18.800 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:07:18.808 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 514 2024-09-12T13:07:18.835 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:07:18.835 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:07:18.835 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:07:18.835 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:07:18.835 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:07:18.835 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:07:18.836 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:07:18.847 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:07:19.808 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 517 2024-09-12T13:07:19.810 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 518 2024-09-12T13:07:19.835 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:07:19.841 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:07:19.874 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 520 2024-09-12T13:07:19.882 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:07:20.797 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:07:20.797 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:07:20.797 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:07:20.797 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:07:20.798 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:07:20.798 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:07:20.800 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 519 2024-09-12T13:07:20.812 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:07:20.813 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:07:20.813 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:07:20.813 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:07:20.813 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:07:20.813 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:07:20.821 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:07:20.822 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:07:20.822 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:07:20.822 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:07:20.822 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:07:20.822 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:07:20.831 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:07:21.797 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 521 2024-09-12T13:07:21.821 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:07:21.834 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 524 2024-09-12T13:07:21.839 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:07:21.839 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:07:21.839 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:07:21.839 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:07:21.839 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:07:21.839 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:07:21.847 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:07:21.850 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:07:21.850 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:07:21.850 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:07:21.850 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:07:21.851 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:07:21.851 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:07:21.851 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 522 2024-09-12T13:07:21.863 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:07:22.839 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:07:22.839 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:07:22.839 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:07:22.839 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:07:22.839 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:07:22.839 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:07:22.844 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:07:22.844 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:07:22.844 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:07:22.844 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:07:22.844 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:07:22.844 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:07:22.854 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 526 2024-09-12T13:07:22.857 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 525 2024-09-12T13:07:22.864 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:07:22.866 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:07:22.886 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:07:22.886 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:07:22.886 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:07:22.886 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:07:22.886 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:07:22.887 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:07:23.834 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:07:23.834 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:07:23.835 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:07:23.835 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:07:23.835 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:07:23.835 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:07:23.865 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 528 2024-09-12T13:07:23.871 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 527 2024-09-12T13:07:23.873 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:07:23.878 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:07:24.818 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 523 2024-09-12T13:07:24.825 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:07:24.825 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:07:24.825 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:07:24.825 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:07:24.825 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:07:24.825 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:07:24.846 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:07:24.850 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:07:24.850 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:07:24.850 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:07:24.850 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:07:24.850 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:07:24.851 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:07:24.867 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:07:24.867 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:07:24.867 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:07:24.867 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:07:24.867 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:07:24.867 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:07:24.887 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 530 2024-09-12T13:07:24.899 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:07:25.822 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 529 2024-09-12T13:07:25.843 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:07:25.866 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:07:25.866 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:07:25.866 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:07:25.866 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:07:25.866 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:07:25.867 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 531 2024-09-12T13:07:25.867 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:07:25.869 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:07:25.869 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:07:25.869 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:07:25.869 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:07:25.869 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:07:25.869 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:07:25.877 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:07:26.876 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:07:26.877 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:07:26.877 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:07:26.877 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:07:26.877 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:07:26.877 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:07:26.881 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:07:26.881 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:07:26.881 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:07:26.881 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:07:26.881 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:07:26.882 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:07:27.756 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 532 2024-09-12T13:07:27.797 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 533 2024-09-12T13:07:27.823 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:07:27.840 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:07:27.841 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 534 2024-09-12T13:07:27.849 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:07:27.849 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:07:27.849 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:07:27.849 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:07:27.849 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:07:27.849 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:07:27.859 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:07:27.859 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 535 2024-09-12T13:07:27.870 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:07:27.902 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:07:27.902 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:07:27.902 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:07:27.902 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:07:27.903 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:07:27.903 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:07:28.845 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:07:28.846 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:07:28.846 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:07:28.846 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:07:28.846 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:07:28.846 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:07:28.880 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:07:28.880 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:07:28.880 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:07:28.880 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:07:28.880 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:07:28.880 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:07:29.822 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 537 2024-09-12T13:07:29.833 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 536 2024-09-12T13:07:29.870 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:07:29.870 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:07:29.928 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 540 2024-09-12T13:07:29.937 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:07:30.824 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 539 2024-09-12T13:07:30.826 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:07:30.826 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:07:30.826 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:07:30.826 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:07:30.826 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:07:30.826 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:07:30.844 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:07:30.844 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:07:30.844 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:07:30.844 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:07:30.844 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:07:30.844 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:07:30.860 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:07:30.861 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:07:30.862 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:07:30.862 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:07:30.862 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:07:30.862 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:07:30.862 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:07:30.873 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:07:30.873 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:07:30.873 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:07:30.874 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:07:30.874 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:07:30.874 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:07:31.846 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 538 2024-09-12T13:07:31.851 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 541 2024-09-12T13:07:31.877 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:07:31.878 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:07:31.896 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 542 2024-09-12T13:07:31.910 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:07:31.912 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 543 2024-09-12T13:07:31.920 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:07:32.873 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:07:32.873 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:07:32.873 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:07:32.873 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:07:32.873 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:07:32.874 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:07:32.874 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:07:32.874 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:07:32.874 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:07:32.874 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:07:32.874 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:07:32.874 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:07:32.939 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:07:32.940 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:07:32.940 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:07:32.940 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:07:32.940 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:07:32.940 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:07:33.855 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 545 2024-09-12T13:07:33.863 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:07:33.863 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:07:33.863 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:07:33.863 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:07:33.863 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:07:33.863 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:07:33.872 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 544 2024-09-12T13:07:33.890 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:07:33.892 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:07:33.914 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 547 2024-09-12T13:07:33.917 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 546 2024-09-12T13:07:33.923 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:07:33.925 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:07:34.881 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:07:34.881 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:07:34.881 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:07:34.881 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:07:34.881 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:07:34.882 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:07:34.882 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:07:34.882 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:07:34.882 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:07:34.882 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:07:34.882 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:07:34.882 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:07:34.895 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 549 2024-09-12T13:07:34.903 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:07:34.913 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:07:34.913 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:07:34.913 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:07:34.913 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:07:34.913 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:07:34.913 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:07:34.922 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:07:34.922 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:07:34.922 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:07:34.923 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:07:34.923 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:07:34.923 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:07:35.857 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 548 2024-09-12T13:07:35.875 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:07:35.893 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 551 2024-09-12T13:07:35.904 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 550 2024-09-12T13:07:35.908 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:07:35.911 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:07:36.893 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:07:36.893 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:07:36.893 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:07:36.893 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:07:36.893 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:07:36.893 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:07:36.895 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:07:36.896 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:07:36.896 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:07:36.896 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:07:36.896 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:07:36.896 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:07:36.925 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:07:36.925 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:07:36.925 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:07:36.926 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:07:36.926 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:07:36.926 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:07:36.928 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:07:36.928 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:07:36.928 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:07:36.928 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:07:36.928 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:07:36.928 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:07:37.766 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 553 2024-09-12T13:07:37.781 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 552 2024-09-12T13:07:37.809 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:07:37.814 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:07:37.836 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 555 2024-09-12T13:07:37.839 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 554 2024-09-12T13:07:37.849 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:07:37.851 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:07:37.906 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:07:37.906 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:07:37.906 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:07:37.906 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:07:37.906 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:07:37.907 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:07:38.808 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 557 2024-09-12T13:07:38.811 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 556 2024-09-12T13:07:38.820 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:07:38.825 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:07:38.878 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:07:38.878 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:07:38.878 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:07:38.878 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:07:38.878 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:07:38.878 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:07:38.911 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:07:38.911 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:07:38.911 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:07:38.911 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:07:38.911 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:07:38.911 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:07:38.913 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:07:38.913 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:07:38.914 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:07:38.914 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:07:38.914 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:07:38.914 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:07:39.800 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 558 2024-09-12T13:07:39.809 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:07:39.816 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 559 2024-09-12T13:07:39.829 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:07:40.807 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 560 2024-09-12T13:07:40.813 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:07:40.814 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:07:40.814 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:07:40.814 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:07:40.814 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:07:40.814 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:07:40.818 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 561 2024-09-12T13:07:40.821 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:07:40.821 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:07:40.821 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:07:40.821 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:07:40.821 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:07:40.821 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:07:40.832 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:07:40.839 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:07:40.853 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:07:40.853 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:07:40.853 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:07:40.854 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:07:40.854 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:07:40.854 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:07:40.854 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:07:40.854 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:07:40.854 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:07:40.854 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:07:40.854 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:07:40.854 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:07:41.797 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 563 2024-09-12T13:07:41.814 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 562 2024-09-12T13:07:41.822 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:07:41.823 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:07:41.824 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:07:41.824 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:07:41.824 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:07:41.824 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:07:41.824 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:07:41.829 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:07:41.829 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:07:41.829 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:07:41.829 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:07:41.829 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:07:41.829 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:07:41.830 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:07:42.813 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:07:42.813 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:07:42.813 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:07:42.813 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:07:42.813 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:07:42.813 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:07:42.833 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:07:42.833 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:07:42.833 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:07:42.833 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:07:42.833 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:07:42.833 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:07:42.849 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 564 2024-09-12T13:07:42.853 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 565 2024-09-12T13:07:42.877 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:07:42.880 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:07:43.809 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 566 2024-09-12T13:07:43.829 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 567 2024-09-12T13:07:43.837 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:07:43.837 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:07:43.837 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:07:43.837 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:07:43.837 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:07:43.837 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:07:43.838 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:07:43.844 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:07:43.846 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:07:43.846 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:07:43.846 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:07:43.846 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:07:43.846 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:07:43.863 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:07:44.825 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:07:44.825 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:07:44.825 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:07:44.825 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:07:44.825 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:07:44.826 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:07:44.835 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:07:44.835 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:07:44.835 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:07:44.835 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:07:44.835 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:07:44.835 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:07:45.805 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 569 2024-09-12T13:07:45.811 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 568 2024-09-12T13:07:45.844 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:07:45.845 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:07:45.881 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:07:45.881 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:07:45.881 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:07:45.881 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:07:45.882 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:07:45.882 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:07:45.882 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:07:45.882 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:07:45.882 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:07:45.882 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:07:45.882 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:07:45.882 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:07:46.806 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 571 2024-09-12T13:07:46.816 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 570 2024-09-12T13:07:46.837 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:07:46.842 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:07:46.842 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:07:46.842 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:07:46.842 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:07:46.842 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:07:46.842 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:07:46.845 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:07:46.868 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:07:46.868 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:07:46.868 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:07:46.868 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:07:46.868 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:07:46.869 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:07:47.768 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 572 2024-09-12T13:07:47.784 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 573 2024-09-12T13:07:47.811 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:07:47.822 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:07:48.796 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 574 2024-09-12T13:07:48.817 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 575 2024-09-12T13:07:48.824 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:07:48.847 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:07:48.847 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:07:48.847 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:07:48.847 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:07:48.848 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:07:48.848 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:07:48.848 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:07:48.848 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:07:48.848 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:07:48.848 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:07:48.848 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:07:48.848 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:07:48.848 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:07:49.793 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 576 2024-09-12T13:07:49.799 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 577 2024-09-12T13:07:49.830 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:07:49.832 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:07:49.840 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:07:49.840 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:07:49.840 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:07:49.841 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:07:49.841 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:07:49.841 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:07:49.848 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:07:49.848 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:07:49.848 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:07:49.848 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:07:49.848 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:07:49.848 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:07:49.854 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 579 2024-09-12T13:07:49.857 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 578 2024-09-12T13:07:49.867 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:07:49.870 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:07:50.814 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:07:50.815 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:07:50.815 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:07:50.815 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:07:50.815 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:07:50.815 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:07:50.824 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 581 2024-09-12T13:07:50.825 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:07:50.825 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:07:50.825 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:07:50.825 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:07:50.825 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:07:50.825 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:07:50.826 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 580 2024-09-12T13:07:50.834 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:07:50.838 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:07:51.827 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:07:51.827 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:07:51.827 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:07:51.827 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:07:51.828 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:07:51.828 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:07:51.832 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 583 2024-09-12T13:07:51.841 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 582 2024-09-12T13:07:51.846 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:07:51.849 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:07:51.849 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:07:51.850 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:07:51.850 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:07:51.850 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:07:51.850 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:07:51.850 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:07:52.834 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:07:52.834 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:07:52.834 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:07:52.834 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:07:52.834 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:07:52.834 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:07:52.835 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:07:52.835 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:07:52.835 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:07:52.835 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:07:52.835 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:07:52.835 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:07:52.841 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 584 2024-09-12T13:07:52.843 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 585 2024-09-12T13:07:52.852 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:07:52.853 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:07:52.869 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:07:52.869 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:07:52.869 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:07:52.869 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:07:52.869 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:07:52.869 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:07:52.873 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:07:52.873 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:07:52.873 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:07:52.873 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:07:52.873 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:07:52.873 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:07:53.837 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:07:53.837 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:07:53.837 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:07:53.837 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:07:53.837 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:07:53.837 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:07:53.841 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:07:53.841 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:07:53.841 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:07:53.841 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:07:53.841 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:07:53.842 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:07:53.848 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 586 2024-09-12T13:07:53.850 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 587 2024-09-12T13:07:53.859 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:07:53.868 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:07:54.849 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:07:54.849 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:07:54.850 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:07:54.850 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:07:54.850 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:07:54.850 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:07:54.852 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:07:54.852 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:07:54.852 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:07:54.852 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:07:54.852 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:07:54.852 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:07:54.875 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:07:54.875+0000 7f67dd000640 -1 rbd::mirror::image_replayer::snapshot::Replayer: 0x560555cab400 handle_unlink_peer: failed to unlink local peer from remote image: (22) Invalid argument 2024-09-12T13:07:54.877 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:07:54.877+0000 7f67dd000640 -1 rbd::mirror::image_replayer::snapshot::Replayer: 0x56055c2b3c00 handle_unlink_peer: failed to unlink local peer from remote image: (22) Invalid argument 2024-09-12T13:07:55.839 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 588 2024-09-12T13:07:55.845 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 589 2024-09-12T13:07:55.854 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:07:55.855 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:07:55.855 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:07:55.855 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:07:55.855 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:07:55.855 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:07:55.856 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:07:55.856 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:07:55.856 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:07:55.856 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:07:55.856 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:07:55.856 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:07:55.881 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:07:55.889 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:07:56.861 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:07:56.862 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:07:56.862 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:07:56.862 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:07:56.862 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:07:56.862 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:07:56.871 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:07:56.871 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:07:56.871 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:07:56.871 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:07:56.871 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:07:56.871 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:07:56.900 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 591 2024-09-12T13:07:56.907 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 590 2024-09-12T13:07:56.934 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:07:56.935 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:07:57.777 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 592 2024-09-12T13:07:57.789 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 593 2024-09-12T13:07:57.824 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:07:57.832 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:07:57.838 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 595 2024-09-12T13:07:57.875 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:07:58.834 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 594 2024-09-12T13:07:58.864 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:07:58.883 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:07:58.883 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:07:58.884 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:07:58.884 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:07:58.884 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:07:58.884 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:07:58.892 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:07:58.892 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:07:58.892 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:07:58.892 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:07:58.892 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:07:58.892 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:07:59.851 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 596 2024-09-12T13:07:59.857 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 597 2024-09-12T13:07:59.876 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:07:59.883 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:07:59.893 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 598 2024-09-12T13:07:59.899 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 599 2024-09-12T13:07:59.907 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:07:59.912 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:07:59.938 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:07:59.938 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:07:59.938 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:07:59.938 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:07:59.938 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:07:59.938 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:07:59.938 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:07:59.939 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:07:59.939 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:07:59.939 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:07:59.939 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:07:59.939 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:08:00.827 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:08:00.827 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:08:00.827 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:08:00.827 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:08:00.828 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:08:00.828 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:08:00.835 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:08:00.835 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:08:00.835 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:08:00.835 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:08:00.835 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:08:00.835 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:08:00.867 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 601 2024-09-12T13:08:00.875 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 600 2024-09-12T13:08:00.878 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:08:00.878 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:08:00.878 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:08:00.878 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:08:00.878 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:08:00.878 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:08:00.882 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:08:00.887 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:08:01.867 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:08:01.868 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:08:01.868 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:08:01.868 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:08:01.868 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:08:01.868 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:08:01.871 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 603 2024-09-12T13:08:01.883 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 602 2024-09-12T13:08:01.886 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:08:01.891 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:08:02.880 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:08:02.880 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:08:02.880 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:08:02.880 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:08:02.880 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:08:02.881 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:08:02.886 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:08:02.886 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:08:02.886 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:08:02.886 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:08:02.886 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:08:02.887 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:08:02.892 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 605 2024-09-12T13:08:02.900 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:08:02.910 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:08:02.910 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:08:02.910 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:08:02.910 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:08:02.910 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:08:02.910 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:08:02.915 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:08:02.915 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:08:02.915 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:08:02.915 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:08:02.915 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:08:02.915 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:08:03.863 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 604 2024-09-12T13:08:03.886 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:08:03.886 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:08:03.886 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:08:03.886 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:08:03.886 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:08:03.886 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:08:03.886 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:08:03.890 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:08:03.890 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:08:03.890 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:08:03.890 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:08:03.890 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:08:03.890 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:08:03.902 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 607 2024-09-12T13:08:03.906 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 606 2024-09-12T13:08:03.910 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 608 2024-09-12T13:08:03.917 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 609 2024-09-12T13:08:03.918 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:08:03.918 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:08:03.925 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:08:03.932 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:08:04.889 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:08:04.890 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:08:04.890 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:08:04.890 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:08:04.890 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:08:04.890 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:08:04.894 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:08:04.894 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:08:04.894 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:08:04.894 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:08:04.894 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:08:04.894 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:08:05.903 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:08:05.904 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:08:05.904 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:08:05.904 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:08:05.904 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:08:05.904 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:08:06.873 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 611 2024-09-12T13:08:06.885 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 610 2024-09-12T13:08:06.889 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:08:06.889 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:08:06.889 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:08:06.889 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:08:06.889 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:08:06.889 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:08:06.907 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:08:06.917 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:08:06.921 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:08:06.922 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:08:06.922 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:08:06.922 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:08:06.922 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:08:06.922 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:08:06.922 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:08:06.922 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:08:06.922 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:08:06.922 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:08:06.922 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:08:06.922 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:08:06.928 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:08:06.928 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:08:06.928 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:08:06.928 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:08:06.928 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:08:06.928 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:08:06.934 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:08:06.934 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:08:06.934 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:08:06.935 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:08:06.935 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:08:06.935 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:08:07.773 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 612 2024-09-12T13:08:07.788 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 613 2024-09-12T13:08:07.824 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:08:07.825 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 614 2024-09-12T13:08:07.828 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:08:07.856 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:08:07.868 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 615 2024-09-12T13:08:07.869 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 618 2024-09-12T13:08:07.883 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 619 2024-09-12T13:08:07.893 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:08:07.894 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:08:07.900 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:08:09.880 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 617 2024-09-12T13:08:09.883 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 616 2024-09-12T13:08:09.910 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:08:09.910 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:08:09.910 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:08:09.910 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:08:09.910 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:08:09.911 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:08:09.911 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:08:09.916 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:08:09.919 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:08:09.919 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:08:09.919 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:08:09.919 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:08:09.919 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:08:09.919 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:08:10.827 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:08:10.827 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:08:10.828 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:08:10.828 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:08:10.828 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:08:10.828 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:08:10.831 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:08:10.831 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:08:10.831 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:08:10.831 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:08:10.832 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:08:10.832 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:08:10.859 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:08:10.859 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:08:10.859 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:08:10.859 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:08:10.859 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:08:10.859 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:08:10.897 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:08:10.897 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:08:10.897 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:08:10.897 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:08:10.897 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:08:10.897 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:08:10.898 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:08:10.898 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:08:10.898 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:08:10.898 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:08:10.898 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:08:10.898 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:08:10.903 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:08:10.903 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:08:10.903 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:08:10.903 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:08:10.903 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:08:10.904 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:08:10.916 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 620 2024-09-12T13:08:10.922 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 621 2024-09-12T13:08:10.934 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:08:10.934 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:08:11.930 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 622 2024-09-12T13:08:11.935 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 627 2024-09-12T13:08:11.939 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:08:11.942 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 623 2024-09-12T13:08:11.944 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 626 2024-09-12T13:08:11.957 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:08:11.969 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:08:11.969 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:08:12.914 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:08:12.914 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:08:12.914 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:08:12.915 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:08:12.915 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:08:12.915 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:08:12.919 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:08:12.919 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:08:12.919 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:08:12.919 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:08:12.919 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:08:12.920 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:08:12.944 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:08:12.943+0000 7f67dc600640 -1 rbd::mirror::image_replayer::snapshot::Replayer: 0x560556cdb400 handle_unlink_peer: failed to unlink local peer from remote image: (22) Invalid argument 2024-09-12T13:08:13.898 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 624 2024-09-12T13:08:13.905 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 625 2024-09-12T13:08:13.931 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:08:13.936 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:08:13.936 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:08:13.936 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:08:13.936 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:08:13.937 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:08:13.937 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:08:13.938 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:08:13.938 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:08:13.938 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:08:13.938 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:08:13.938 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:08:13.938 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:08:13.942 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:08:13.957 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 628 2024-09-12T13:08:13.959 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 629 2024-09-12T13:08:13.970 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:08:13.975 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:08:14.942 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:08:14.942 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:08:14.942 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:08:14.942 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:08:14.942 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:08:14.942 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:08:14.960 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:08:14.960 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:08:14.960 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:08:14.960 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:08:14.960 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:08:14.960 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:08:14.971 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:08:14.972 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:08:14.972 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:08:14.972 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:08:14.972 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:08:14.972 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:08:14.972 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${SNAP_PIDS[@]} 2024-09-12T13:08:14.972 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 47297 2024-09-12T13:08:16.905 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 631 2024-09-12T13:08:16.920 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 630 2024-09-12T13:08:16.928 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 633 2024-09-12T13:08:16.933 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:08:16.933 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:08:16.933 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:08:16.933 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:08:16.934 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:08:16.934 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:08:16.943 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:08:16.944 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:08:16.944 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:08:16.944 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:08:16.944 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:08:16.944 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:08:16.944 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:08:16.954 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:08:16.961 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:08:16.973 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:08:16.973 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:08:16.973 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:08:16.973 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:08:16.973 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:08:16.973 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:08:16.977 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:08:16.977 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:08:16.977 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:08:16.977 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:08:16.977 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:08:16.977 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:08:17.778 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 632 2024-09-12T13:08:17.790 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 634 2024-09-12T13:08:17.807 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:08:17.817 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:08:17.840 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 635 2024-09-12T13:08:17.842 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 636 2024-09-12T13:08:17.849 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:08:17.859 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:08:18.818 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:08:18.818+0000 7f67dc600640 -1 rbd::mirror::image_replayer::snapshot::Replayer: 0x56055b7dd800 handle_unlink_peer: failed to unlink local peer from remote image: (22) Invalid argument 2024-09-12T13:08:18.819 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:08:18.820+0000 7f67dd000640 -1 rbd::mirror::image_replayer::snapshot::Replayer: 0x56055b7d5800 handle_unlink_peer: failed to unlink local peer from remote image: (22) Invalid argument 2024-09-12T13:08:19.777 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 638 2024-09-12T13:08:19.777 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 637 2024-09-12T13:08:19.803 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:08:19.803 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:08:19.946 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${SNAP_PIDS[@]} 2024-09-12T13:08:19.946 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 47429 2024-09-12T13:08:19.957 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:08:19.957 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:08:19.957 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:08:19.957 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:08:19.957 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:08:19.957 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:08:20.789 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 639 2024-09-12T13:08:20.802 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:08:20.810 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${SNAP_PIDS[@]} 2024-09-12T13:08:20.810 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 47561 2024-09-12T13:08:20.820 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:08:20.820 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:08:20.820 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:08:20.820 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:08:20.820 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:08:20.820 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:08:20.852 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${SNAP_PIDS[@]} 2024-09-12T13:08:20.852 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 47713 2024-09-12T13:08:20.861 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:08:20.861 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:08:20.862 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:08:20.862 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:08:20.862 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:08:20.862 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:08:21.807 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 640 2024-09-12T13:08:21.820 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:08:22.806 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:08:22.806 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:08:22.807 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:08:22.807 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:08:22.807 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:08:22.807 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:08:22.807 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${SNAP_PIDS[@]} 2024-09-12T13:08:22.807 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 47866 2024-09-12T13:08:22.807 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${SNAP_PIDS[@]} 2024-09-12T13:08:22.807 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 48019 2024-09-12T13:08:23.787 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 641 2024-09-12T13:08:23.805 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${SNAP_PIDS[@]} 2024-09-12T13:08:23.805 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 48172 2024-09-12T13:08:23.815 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:08:23.825 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 642 2024-09-12T13:08:23.839 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:08:24.822 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${SNAP_PIDS[@]} 2024-09-12T13:08:24.822 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 48345 2024-09-12T13:08:26.818 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${SNAP_PIDS[@]} 2024-09-12T13:08:26.818 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 48519 2024-09-12T13:08:26.841 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ (( RET != 0 )) 2024-09-12T13:08:26.841 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${WORKLOAD_PIDS[@]} 2024-09-12T13:08:26.841 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 47166 2024-09-12T13:10:53.489 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ ret=124 2024-09-12T13:10:53.489 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ (( ret != 124 )) 2024-09-12T13:10:53.489 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${WORKLOAD_PIDS[@]} 2024-09-12T13:10:53.489 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 47298 2024-09-12T13:10:54.581 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ ret=124 2024-09-12T13:10:54.581 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ (( ret != 124 )) 2024-09-12T13:10:54.581 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${WORKLOAD_PIDS[@]} 2024-09-12T13:10:54.581 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 47430 2024-09-12T13:10:55.732 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ ret=124 2024-09-12T13:10:55.732 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ (( ret != 124 )) 2024-09-12T13:10:55.732 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${WORKLOAD_PIDS[@]} 2024-09-12T13:10:55.732 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 47562 2024-09-12T13:10:56.440 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ ret=124 2024-09-12T13:10:56.440 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ (( ret != 124 )) 2024-09-12T13:10:56.440 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${WORKLOAD_PIDS[@]} 2024-09-12T13:10:56.440 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 47714 2024-09-12T13:10:57.467 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ ret=124 2024-09-12T13:10:57.467 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ (( ret != 124 )) 2024-09-12T13:10:57.467 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${WORKLOAD_PIDS[@]} 2024-09-12T13:10:57.467 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 47867 2024-09-12T13:10:58.427 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ ret=124 2024-09-12T13:10:58.427 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ (( ret != 124 )) 2024-09-12T13:10:58.427 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${WORKLOAD_PIDS[@]} 2024-09-12T13:10:58.427 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 48020 2024-09-12T13:10:59.430 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ ret=124 2024-09-12T13:10:59.431 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ (( ret != 124 )) 2024-09-12T13:10:59.431 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${WORKLOAD_PIDS[@]} 2024-09-12T13:10:59.431 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 48173 2024-09-12T13:11:00.440 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ ret=124 2024-09-12T13:11:00.440 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ (( ret != 124 )) 2024-09-12T13:11:00.440 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${WORKLOAD_PIDS[@]} 2024-09-12T13:11:00.440 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 48346 2024-09-12T13:11:01.529 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ ret=124 2024-09-12T13:11:01.529 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ (( ret != 124 )) 2024-09-12T13:11:01.529 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${WORKLOAD_PIDS[@]} 2024-09-12T13:11:01.529 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 48520 2024-09-12T13:11:02.677 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ ret=124 2024-09-12T13:11:02.677 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ (( ret != 124 )) 2024-09-12T13:11:02.677 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ (( RET != 0 )) 2024-09-12T13:11:02.677 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:11:02.679 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ COMPARE_PIDS+=($!) 2024-09-12T13:11:02.679 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:11:02.679 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ compare_demoted_promoted_image 1 2024-09-12T13:11:02.679 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local dev=/dev/rbd0 2024-09-12T13:11:02.679 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local img=image-primary1 2024-09-12T13:11:02.679 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mntpt=test-primary1 2024-09-12T13:11:02.679 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local demote_md5 promote_md5 2024-09-12T13:11:02.679 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ COMPARE_PIDS+=($!) 2024-09-12T13:11:02.679 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo umount test-primary1 2024-09-12T13:11:02.679 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:11:02.679 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ compare_demoted_promoted_image 2 2024-09-12T13:11:02.680 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local dev=/dev/rbd1 2024-09-12T13:11:02.680 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local img=image-primary2 2024-09-12T13:11:02.680 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mntpt=test-primary2 2024-09-12T13:11:02.680 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local demote_md5 promote_md5 2024-09-12T13:11:02.680 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo umount test-primary2 2024-09-12T13:11:02.680 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ COMPARE_PIDS+=($!) 2024-09-12T13:11:02.680 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:11:02.681 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ compare_demoted_promoted_image 3 2024-09-12T13:11:02.682 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local dev=/dev/rbd2 2024-09-12T13:11:02.682 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ COMPARE_PIDS+=($!) 2024-09-12T13:11:02.682 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local img=image-primary3 2024-09-12T13:11:02.682 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mntpt=test-primary3 2024-09-12T13:11:02.682 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local demote_md5 promote_md5 2024-09-12T13:11:02.682 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo umount test-primary3 2024-09-12T13:11:02.682 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:11:02.682 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ compare_demoted_promoted_image 4 2024-09-12T13:11:02.682 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local dev=/dev/rbd3 2024-09-12T13:11:02.682 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local img=image-primary4 2024-09-12T13:11:02.682 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mntpt=test-primary4 2024-09-12T13:11:02.682 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local demote_md5 promote_md5 2024-09-12T13:11:02.682 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo umount test-primary4 2024-09-12T13:11:02.683 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ COMPARE_PIDS+=($!) 2024-09-12T13:11:02.683 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:11:02.683 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ compare_demoted_promoted_image 5 2024-09-12T13:11:02.683 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local dev=/dev/rbd4 2024-09-12T13:11:02.684 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local img=image-primary5 2024-09-12T13:11:02.684 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mntpt=test-primary5 2024-09-12T13:11:02.684 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local demote_md5 promote_md5 2024-09-12T13:11:02.684 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo umount test-primary5 2024-09-12T13:11:02.684 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ COMPARE_PIDS+=($!) 2024-09-12T13:11:02.685 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:11:02.685 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ compare_demoted_promoted_image 6 2024-09-12T13:11:02.685 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local dev=/dev/rbd5 2024-09-12T13:11:02.685 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local img=image-primary6 2024-09-12T13:11:02.685 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mntpt=test-primary6 2024-09-12T13:11:02.685 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local demote_md5 promote_md5 2024-09-12T13:11:02.685 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ COMPARE_PIDS+=($!) 2024-09-12T13:11:02.685 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo umount test-primary6 2024-09-12T13:11:02.685 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:11:02.685 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ compare_demoted_promoted_image 7 2024-09-12T13:11:02.685 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local dev=/dev/rbd6 2024-09-12T13:11:02.685 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local img=image-primary7 2024-09-12T13:11:02.685 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mntpt=test-primary7 2024-09-12T13:11:02.686 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local demote_md5 promote_md5 2024-09-12T13:11:02.686 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo umount test-primary7 2024-09-12T13:11:02.686 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ COMPARE_PIDS+=($!) 2024-09-12T13:11:02.686 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ compare_demoted_promoted_image 8 2024-09-12T13:11:02.686 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local dev=/dev/rbd7 2024-09-12T13:11:02.687 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local img=image-primary8 2024-09-12T13:11:02.687 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mntpt=test-primary8 2024-09-12T13:11:02.687 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local demote_md5 promote_md5 2024-09-12T13:11:02.687 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo umount test-primary8 2024-09-12T13:11:02.687 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:11:02.687 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ COMPARE_PIDS+=($!) 2024-09-12T13:11:02.687 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:11:02.687 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ compare_demoted_promoted_image 9 2024-09-12T13:11:02.687 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local dev=/dev/rbd8 2024-09-12T13:11:02.691 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local img=image-primary9 2024-09-12T13:11:02.691 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mntpt=test-primary9 2024-09-12T13:11:02.691 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local demote_md5 promote_md5 2024-09-12T13:11:02.691 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo umount test-primary9 2024-09-12T13:11:02.691 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ COMPARE_PIDS+=($!) 2024-09-12T13:11:02.691 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${COMPARE_PIDS[@]} 2024-09-12T13:11:02.692 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ compare_demoted_promoted_image 10 2024-09-12T13:11:02.692 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 54485 2024-09-12T13:11:02.692 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local dev=/dev/rbd9 2024-09-12T13:11:02.695 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local img=image-primary10 2024-09-12T13:11:02.696 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mntpt=test-primary10 2024-09-12T13:11:02.696 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local demote_md5 promote_md5 2024-09-12T13:11:02.696 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo umount test-primary10 2024-09-12T13:11:03.362 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ awk '{print $1}' 2024-09-12T13:11:03.364 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo md5sum /dev/rbd1 2024-09-12T13:11:03.442 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo md5sum /dev/rbd0 2024-09-12T13:11:03.444 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ awk '{print $1}' 2024-09-12T13:11:03.475 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo md5sum /dev/rbd2 2024-09-12T13:11:03.476 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo md5sum /dev/rbd4 2024-09-12T13:11:03.476 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ awk '{print $1}' 2024-09-12T13:11:03.479 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ awk '{print $1}' 2024-09-12T13:11:03.495 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo md5sum /dev/rbd6 2024-09-12T13:11:03.496 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ awk '{print $1}' 2024-09-12T13:11:03.499 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo md5sum /dev/rbd9 2024-09-12T13:11:03.502 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ awk '{print $1}' 2024-09-12T13:11:03.506 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo md5sum /dev/rbd7 2024-09-12T13:11:03.509 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ awk '{print $1}' 2024-09-12T13:11:03.609 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ awk '{print $1}' 2024-09-12T13:11:03.609 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo md5sum /dev/rbd3 2024-09-12T13:11:03.649 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ awk '{print $1}' 2024-09-12T13:11:03.650 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo md5sum /dev/rbd5 2024-09-12T13:11:03.719 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ awk '{print $1}' 2024-09-12T13:11:03.719 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo md5sum /dev/rbd8 2024-09-12T13:11:49.678 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ demote_md5=6081f378dfc29c81e4c3cd6d6202923e 2024-09-12T13:11:49.678 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo rbd --cluster cluster1 device unmap -t krbd mirror/image-primary6 2024-09-12T13:11:50.302 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ demote_image cluster1 mirror image-primary6 2024-09-12T13:11:50.302 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:11:50.302 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:11:50.302 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:11:50.302 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 mirror image demote mirror/image-primary6 2024-09-12T13:11:55.015 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ demote_md5=5f048362e14926082cdee95990e2db55 2024-09-12T13:11:55.015 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo rbd --cluster cluster1 device unmap -t krbd mirror/image-primary2 2024-09-12T13:11:55.105 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Image demoted to non-primary 2024-09-12T13:11:55.137 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary6 up+unknown 2024-09-12T13:11:55.139 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:11:55.139 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:11:55.139 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:11:55.139 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:11:55.140 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:11:55.140 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:11:55.140 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:11:55.140 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:11:55.289 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ demote_image cluster1 mirror image-primary2 2024-09-12T13:11:55.289 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:11:55.289 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:11:55.289 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:11:55.289 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 mirror image demote mirror/image-primary2 2024-09-12T13:11:56.109 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Image demoted to non-primary 2024-09-12T13:11:56.136 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary2 up+unknown 2024-09-12T13:11:56.136 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:11:56.136 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:11:56.136 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:11:56.136 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:11:56.137 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:11:56.137 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:11:56.137 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:11:56.137 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:11:56.163 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary6 up+unknown '' '' 2024-09-12T13:11:56.163 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:11:56.164 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:11:56.164 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:11:56.164 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:11:56.164 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:11:56.164 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:11:56.164 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:11:56.170 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:11:56.170 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary6 2024-09-12T13:11:56.352 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary6: 2024-09-12T13:11:56.352 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: b711ef4c-c601-46a4-a872-1c8abdd1ca45 2024-09-12T13:11:56.352 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:11:56.352 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:11:56.353 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:11:56.353 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:11:47' 2024-09-12T13:11:56.353 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:11:56.365 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:11:56.365 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:11:56.365 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 2 2024-09-12T13:11:57.162 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary2 up+unknown '' '' 2024-09-12T13:11:57.163 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:11:57.163 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:11:57.163 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:11:57.163 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:11:57.163 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:11:57.163 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:11:57.164 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:11:57.168 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:11:57.169 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary2 2024-09-12T13:11:57.209 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ demote_md5=30f1f17928cdcb4c7df6846597b60582 2024-09-12T13:11:57.210 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo rbd --cluster cluster1 device unmap -t krbd mirror/image-primary5 2024-09-12T13:11:57.469 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary2: 2024-09-12T13:11:57.470 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 23ee0857-2880-4d74-a09e-17095d384c6b 2024-09-12T13:11:57.470 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:11:57.470 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:11:57.470 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:11:57.470 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:11:47' 2024-09-12T13:11:57.470 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:11:57.479 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:11:57.479 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:11:57.480 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 2 2024-09-12T13:11:57.549 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ demote_image cluster1 mirror image-primary5 2024-09-12T13:11:57.550 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:11:57.550 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:11:57.550 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:11:57.550 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 mirror image demote mirror/image-primary5 2024-09-12T13:11:57.825 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ demote_md5=1bd72432f627efe6b6d65d64cf0cd64d 2024-09-12T13:11:57.825 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo rbd --cluster cluster1 device unmap -t krbd mirror/image-primary4 2024-09-12T13:11:57.838 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Image demoted to non-primary 2024-09-12T13:11:57.855 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary5 up+unknown 2024-09-12T13:11:57.855 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:11:57.855 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:11:57.856 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:11:57.856 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:11:57.856 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:11:57.856 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:11:57.856 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:11:57.856 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:11:57.995 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ demote_image cluster1 mirror image-primary4 2024-09-12T13:11:57.995 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:11:57.995 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:11:57.995 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:11:57.995 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 mirror image demote mirror/image-primary4 2024-09-12T13:11:58.231 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ demote_md5=6ae05fc69a7e23396e5919d64416a23a 2024-09-12T13:11:58.231 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo rbd --cluster cluster1 device unmap -t krbd mirror/image-primary10 2024-09-12T13:11:58.389 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary6 up+unknown '' '' 2024-09-12T13:11:58.389 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:11:58.389 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:11:58.389 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:11:58.389 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:11:58.389 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:11:58.390 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:11:58.390 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:11:58.391 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:11:58.394 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary6 2024-09-12T13:11:58.487 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ demote_image cluster1 mirror image-primary10 2024-09-12T13:11:58.487 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:11:58.487 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:11:58.487 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:11:58.487 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 mirror image demote mirror/image-primary10 2024-09-12T13:11:58.537 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary6: 2024-09-12T13:11:58.537 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: b711ef4c-c601-46a4-a872-1c8abdd1ca45 2024-09-12T13:11:58.537 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:11:58.537 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:11:58.537 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:11:58.537 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:11:47' 2024-09-12T13:11:58.537 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:11:58.540 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:11:58.540 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:11:58.540 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 4 2024-09-12T13:11:58.728 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ demote_md5=3b6e813ceb5ae83d16ef02b8346a6f8a 2024-09-12T13:11:58.728 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo rbd --cluster cluster1 device unmap -t krbd mirror/image-primary7 2024-09-12T13:11:58.859 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary5 up+unknown '' '' 2024-09-12T13:11:58.860 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:11:58.860 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:11:58.860 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:11:58.860 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:11:58.860 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:11:58.860 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:11:58.860 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:11:58.861 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:11:58.861 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary5 2024-09-12T13:11:58.909 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ demote_image cluster1 mirror image-primary7 2024-09-12T13:11:58.910 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:11:58.910 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:11:58.910 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:11:58.910 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 mirror image demote mirror/image-primary7 2024-09-12T13:11:58.923 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ demote_md5=5cce44a8cb4936d1c93e9789805e6a22 2024-09-12T13:11:58.924 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo rbd --cluster cluster1 device unmap -t krbd mirror/image-primary1 2024-09-12T13:11:58.941 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary5: 2024-09-12T13:11:58.942 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 261d375d-e3b2-4dc4-91bd-36dfede8b276 2024-09-12T13:11:58.942 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:11:58.942 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:11:58.943 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:11:58.943 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:11:47' 2024-09-12T13:11:58.943 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:11:58.945 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:11:58.945 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:11:58.945 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 2 2024-09-12T13:11:58.984 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ demote_md5=1b3a3de9819a5503a292e74cc2f0aecc 2024-09-12T13:11:58.984 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo rbd --cluster cluster1 device unmap -t krbd mirror/image-primary8 2024-09-12T13:11:59.109 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ demote_image cluster1 mirror image-primary1 2024-09-12T13:11:59.109 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:11:59.109 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:11:59.110 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:11:59.110 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 mirror image demote mirror/image-primary1 2024-09-12T13:11:59.119 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ demote_image cluster1 mirror image-primary8 2024-09-12T13:11:59.119 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:11:59.119 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:11:59.119 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:11:59.119 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 mirror image demote mirror/image-primary8 2024-09-12T13:11:59.484 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary2 up+unknown '' '' 2024-09-12T13:11:59.484 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:11:59.484 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:11:59.484 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:11:59.484 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:11:59.485 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:11:59.485 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:11:59.485 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:11:59.485 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:11:59.485 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary2 2024-09-12T13:11:59.562 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary2: 2024-09-12T13:11:59.563 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 23ee0857-2880-4d74-a09e-17095d384c6b 2024-09-12T13:11:59.563 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:11:59.563 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:11:59.563 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:11:59.563 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:11:47' 2024-09-12T13:11:59.563 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:11:59.565 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:11:59.565 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:11:59.565 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 4 2024-09-12T13:11:59.670 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ demote_md5=146b91e683fb47cb6ebca28406a46347 2024-09-12T13:11:59.670 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo rbd --cluster cluster1 device unmap -t krbd mirror/image-primary3 2024-09-12T13:11:59.783 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ demote_image cluster1 mirror image-primary3 2024-09-12T13:11:59.783 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:11:59.783 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:11:59.784 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:11:59.784 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 mirror image demote mirror/image-primary3 2024-09-12T13:12:00.141 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Image demoted to non-primary 2024-09-12T13:12:00.151 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Image demoted to non-primary 2024-09-12T13:12:00.158 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Image demoted to non-primary 2024-09-12T13:12:00.165 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary7 up+unknown 2024-09-12T13:12:00.165 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:12:00.165 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:12:00.165 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:12:00.165 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:12:00.165 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:12:00.165 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:12:00.166 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:12:00.166 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:12:00.173 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Image demoted to non-primary 2024-09-12T13:12:00.202 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary10 up+unknown 2024-09-12T13:12:00.204 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:12:00.204 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:12:00.204 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:12:00.205 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:12:00.205 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:12:00.205 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:12:00.205 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:12:00.206 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:12:00.209 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary4 up+unknown 2024-09-12T13:12:00.209 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:12:00.210 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:12:00.211 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:12:00.211 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:12:00.211 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:12:00.211 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:12:00.211 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:12:00.211 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:12:00.213 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary3 up+unknown 2024-09-12T13:12:00.216 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:12:00.216 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:12:00.216 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:12:00.217 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:12:00.217 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:12:00.217 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:12:00.218 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:12:00.218 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:12:00.262 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ demote_md5=fb01e2c493269b8b67426929cdfb8b12 2024-09-12T13:12:00.262 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo rbd --cluster cluster1 device unmap -t krbd mirror/image-primary9 2024-09-12T13:12:00.357 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ demote_image cluster1 mirror image-primary9 2024-09-12T13:12:00.357 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:12:00.357 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:12:00.357 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:12:00.357 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 mirror image demote mirror/image-primary9 2024-09-12T13:12:00.949 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary5 up+unknown '' '' 2024-09-12T13:12:00.949 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:12:00.949 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:12:00.949 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:12:00.949 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:12:00.950 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:12:00.950 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:12:00.950 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:12:00.950 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:12:00.950 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary5 2024-09-12T13:12:01.031 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary5: 2024-09-12T13:12:01.031 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 261d375d-e3b2-4dc4-91bd-36dfede8b276 2024-09-12T13:12:01.031 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:12:01.031 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:12:01.031 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:12:01.031 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:11:47' 2024-09-12T13:12:01.031 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:12:01.035 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:12:01.035 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:12:01.035 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 4 2024-09-12T13:12:01.150 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Image demoted to non-primary 2024-09-12T13:12:01.171 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary7 up+unknown '' '' 2024-09-12T13:12:01.171 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:12:01.172 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:12:01.172 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:12:01.172 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:12:01.172 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:12:01.172 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:12:01.172 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:12:01.174 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:12:01.174 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary7 2024-09-12T13:12:01.181 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary8 up+unknown 2024-09-12T13:12:01.182 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Image demoted to non-primary 2024-09-12T13:12:01.182 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:12:01.182 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:12:01.182 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:12:01.182 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:12:01.182 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:12:01.183 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:12:01.183 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:12:01.183 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:12:01.210 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary10 up+unknown '' '' 2024-09-12T13:12:01.211 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:12:01.211 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:12:01.211 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:12:01.211 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:12:01.211 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:12:01.211 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:12:01.211 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:12:01.216 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:12:01.216 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary10 2024-09-12T13:12:01.216 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary9 up+unknown 2024-09-12T13:12:01.216 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:12:01.217 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:12:01.217 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:12:01.217 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:12:01.217 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:12:01.218 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:12:01.218 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:12:01.218 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:12:01.223 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary4 up+unknown '' '' 2024-09-12T13:12:01.223 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:12:01.223 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:12:01.223 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:12:01.223 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:12:01.223 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:12:01.223 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:12:01.223 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:12:01.223 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:12:01.223 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary4 2024-09-12T13:12:01.229 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary3 up+unknown '' '' 2024-09-12T13:12:01.230 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:12:01.230 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:12:01.230 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:12:01.230 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:12:01.230 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:12:01.230 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:12:01.230 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:12:01.230 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:12:01.230 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary3 2024-09-12T13:12:01.402 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary7: 2024-09-12T13:12:01.402 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 73a79146-a78a-4543-843c-6cef99c271d3 2024-09-12T13:12:01.403 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:12:01.403 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:12:01.403 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:12:01.403 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:11:47' 2024-09-12T13:12:01.403 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:12:01.409 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:12:01.409 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:12:01.410 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 2 2024-09-12T13:12:01.432 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary10: 2024-09-12T13:12:01.432 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 2286adb1-ff3d-442a-9aac-6b38a8513aaa 2024-09-12T13:12:01.432 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:12:01.432 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:12:01.432 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:12:01.432 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:11:47' 2024-09-12T13:12:01.432 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:12:01.439 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:12:01.439 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:12:01.439 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 2 2024-09-12T13:12:01.459 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary3: 2024-09-12T13:12:01.459 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 92498ee6-eac1-4596-be69-c222174caeb6 2024-09-12T13:12:01.459 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:12:01.459 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:12:01.459 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:12:01.459 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:11:47' 2024-09-12T13:12:01.459 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:12:01.475 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:12:01.475 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:12:01.475 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 2 2024-09-12T13:12:01.479 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary4: 2024-09-12T13:12:01.479 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 11f002da-1dfb-4012-a01a-c5237417a099 2024-09-12T13:12:01.479 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:12:01.479 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:12:01.479 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:12:01.480 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:11:47' 2024-09-12T13:12:01.480 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:12:01.483 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:12:01.483 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:12:01.483 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 2 2024-09-12T13:12:02.185 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary8 up+unknown '' '' 2024-09-12T13:12:02.186 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:12:02.186 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:12:02.186 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:12:02.186 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:12:02.186 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:12:02.186 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:12:02.186 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:12:02.187 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:12:02.187 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary8 2024-09-12T13:12:02.223 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary9 up+unknown '' '' 2024-09-12T13:12:02.224 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:12:02.225 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:12:02.225 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:12:02.225 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:12:02.225 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:12:02.225 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:12:02.225 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:12:02.225 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:12:02.225 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary9 2024-09-12T13:12:02.324 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary8: 2024-09-12T13:12:02.324 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: fa7b2037-727d-4ca0-abbc-a51e250f5ecf 2024-09-12T13:12:02.324 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:12:02.325 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:12:02.325 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:12:02.325 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:11:47' 2024-09-12T13:12:02.325 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:12:02.329 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:12:02.329 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:12:02.329 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 2 2024-09-12T13:12:02.349 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary9: 2024-09-12T13:12:02.349 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 77edc192-6758-44c1-b93f-e8f7df97992a 2024-09-12T13:12:02.349 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:12:02.349 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:12:02.349 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:12:02.349 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:11:47' 2024-09-12T13:12:02.349 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:12:02.352 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:12:02.352 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:12:02.352 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 2 2024-09-12T13:12:02.542 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary6 up+unknown '' '' 2024-09-12T13:12:02.543 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:12:02.543 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:12:02.543 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:12:02.543 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:12:02.543 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:12:02.543 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:12:02.543 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:12:02.543 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:12:02.543 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary6 2024-09-12T13:12:02.623 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary6: 2024-09-12T13:12:02.623 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: b711ef4c-c601-46a4-a872-1c8abdd1ca45 2024-09-12T13:12:02.623 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:12:02.623 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:12:02.623 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:12:02.623 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:11:47' 2024-09-12T13:12:02.623 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:12:02.626 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:12:02.626 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:12:02.626 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 8 2024-09-12T13:12:03.417 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary7 up+unknown '' '' 2024-09-12T13:12:03.417 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:12:03.417 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:12:03.418 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:12:03.418 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:12:03.418 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:12:03.418 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:12:03.418 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:12:03.418 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:12:03.418 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary7 2024-09-12T13:12:03.442 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary10 up+unknown '' '' 2024-09-12T13:12:03.442 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:12:03.442 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:12:03.442 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:12:03.443 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:12:03.443 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:12:03.443 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:12:03.443 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:12:03.443 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:12:03.443 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary10 2024-09-12T13:12:03.479 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary3 up+unknown '' '' 2024-09-12T13:12:03.479 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:12:03.479 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:12:03.479 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:12:03.479 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:12:03.479 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:12:03.479 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:12:03.479 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:12:03.480 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:12:03.480 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary3 2024-09-12T13:12:03.487 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary4 up+unknown '' '' 2024-09-12T13:12:03.488 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:12:03.488 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:12:03.488 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:12:03.488 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:12:03.488 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:12:03.488 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:12:03.488 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:12:03.489 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:12:03.489 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary4 2024-09-12T13:12:03.494 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary7: 2024-09-12T13:12:03.494 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 73a79146-a78a-4543-843c-6cef99c271d3 2024-09-12T13:12:03.494 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:12:03.494 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:12:03.494 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:12:03.494 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:11:47' 2024-09-12T13:12:03.494 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:12:03.496 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:12:03.496 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:12:03.496 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 4 2024-09-12T13:12:03.522 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary10: 2024-09-12T13:12:03.523 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 2286adb1-ff3d-442a-9aac-6b38a8513aaa 2024-09-12T13:12:03.523 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:12:03.523 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:12:03.523 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:12:03.523 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:11:47' 2024-09-12T13:12:03.523 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:12:03.525 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:12:03.525 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:12:03.525 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 4 2024-09-12T13:12:03.559 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary3: 2024-09-12T13:12:03.559 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 92498ee6-eac1-4596-be69-c222174caeb6 2024-09-12T13:12:03.559 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:12:03.560 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:12:03.560 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:12:03.560 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:11:47' 2024-09-12T13:12:03.560 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:12:03.562 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:12:03.562 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:12:03.562 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 4 2024-09-12T13:12:03.565 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary4: 2024-09-12T13:12:03.565 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 11f002da-1dfb-4012-a01a-c5237417a099 2024-09-12T13:12:03.565 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:12:03.565 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:12:03.565 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:12:03.565 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:11:47' 2024-09-12T13:12:03.565 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:12:03.568 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:12:03.568 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:12:03.568 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 4 2024-09-12T13:12:03.568 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary2 up+unknown '' '' 2024-09-12T13:12:03.568 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:12:03.568 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:12:03.568 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:12:03.568 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:12:03.568 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:12:03.568 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:12:03.569 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:12:03.569 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:12:03.569 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary2 2024-09-12T13:12:03.647 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary2: 2024-09-12T13:12:03.647 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 23ee0857-2880-4d74-a09e-17095d384c6b 2024-09-12T13:12:03.647 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:12:03.647 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:12:03.647 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:12:03.647 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:11:47' 2024-09-12T13:12:03.647 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:12:03.649 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:12:03.649 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:12:03.649 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 8 2024-09-12T13:12:03.813 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Image demoted to non-primary 2024-09-12T13:12:03.823 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary1 up+unknown 2024-09-12T13:12:03.823 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:12:03.823 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:12:03.823 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:12:03.823 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:12:03.823 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:12:03.823 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:12:03.823 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:12:03.824 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:12:04.332 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary8 up+unknown '' '' 2024-09-12T13:12:04.332 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:12:04.333 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:12:04.333 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:12:04.333 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:12:04.333 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:12:04.333 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:12:04.333 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:12:04.333 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:12:04.333 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary8 2024-09-12T13:12:04.354 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary9 up+unknown '' '' 2024-09-12T13:12:04.355 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:12:04.355 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:12:04.355 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:12:04.355 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:12:04.355 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:12:04.355 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:12:04.355 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:12:04.356 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:12:04.356 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary9 2024-09-12T13:12:04.410 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary8: 2024-09-12T13:12:04.410 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: fa7b2037-727d-4ca0-abbc-a51e250f5ecf 2024-09-12T13:12:04.410 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:12:04.410 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:12:04.410 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:12:04.410 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:11:47' 2024-09-12T13:12:04.410 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:12:04.413 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:12:04.413 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:12:04.413 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 4 2024-09-12T13:12:04.433 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary9: 2024-09-12T13:12:04.433 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 77edc192-6758-44c1-b93f-e8f7df97992a 2024-09-12T13:12:04.434 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:12:04.434 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:12:04.434 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:12:04.434 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:11:47' 2024-09-12T13:12:04.434 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:12:04.436 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:12:04.436 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:12:04.436 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 4 2024-09-12T13:12:04.825 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary1 up+unknown '' '' 2024-09-12T13:12:04.825 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:12:04.826 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:12:04.826 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:12:04.826 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:12:04.826 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:12:04.826 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:12:04.826 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:12:04.826 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:12:04.826 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary1 2024-09-12T13:12:04.903 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary1: 2024-09-12T13:12:04.903 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 1df40b54-54f7-41fe-a2f4-c900ccf2e016 2024-09-12T13:12:04.903 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:12:04.903 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:12:04.903 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:12:04.903 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:11:47' 2024-09-12T13:12:04.903 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:12:04.905 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:12:04.905 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:12:04.906 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 2 2024-09-12T13:12:05.037 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary5 up+unknown '' '' 2024-09-12T13:12:05.037 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:12:05.037 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:12:05.037 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:12:05.037 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:12:05.037 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:12:05.037 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:12:05.037 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:12:05.038 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:12:05.038 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary5 2024-09-12T13:12:05.117 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary5: 2024-09-12T13:12:05.117 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 261d375d-e3b2-4dc4-91bd-36dfede8b276 2024-09-12T13:12:05.117 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:12:05.117 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:12:05.117 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:12:05.117 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:11:47' 2024-09-12T13:12:05.118 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:12:05.120 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:12:05.120 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:12:05.120 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 8 2024-09-12T13:12:06.908 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary1 up+unknown '' '' 2024-09-12T13:12:06.909 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:12:06.909 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:12:06.909 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:12:06.909 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:12:06.909 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:12:06.909 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:12:06.909 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:12:06.909 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:12:06.910 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary1 2024-09-12T13:12:07.003 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary1: 2024-09-12T13:12:07.003 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 1df40b54-54f7-41fe-a2f4-c900ccf2e016 2024-09-12T13:12:07.003 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:12:07.003 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:12:07.003 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:12:07.003 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:11:47' 2024-09-12T13:12:07.004 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:12:07.006 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:12:07.006 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:12:07.006 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 4 2024-09-12T13:12:07.499 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary7 up+unknown '' '' 2024-09-12T13:12:07.499 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:12:07.499 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:12:07.499 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:12:07.500 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:12:07.500 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:12:07.500 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:12:07.500 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:12:07.500 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:12:07.500 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary7 2024-09-12T13:12:07.528 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary10 up+unknown '' '' 2024-09-12T13:12:07.528 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:12:07.528 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:12:07.528 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:12:07.529 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:12:07.529 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:12:07.529 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:12:07.529 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:12:07.529 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:12:07.529 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary10 2024-09-12T13:12:07.565 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary3 up+unknown '' '' 2024-09-12T13:12:07.565 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:12:07.565 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:12:07.565 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:12:07.565 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:12:07.565 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:12:07.565 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:12:07.565 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:12:07.566 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:12:07.566 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary3 2024-09-12T13:12:07.570 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary4 up+unknown '' '' 2024-09-12T13:12:07.570 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:12:07.570 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:12:07.570 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:12:07.570 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:12:07.570 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:12:07.570 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:12:07.570 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:12:07.571 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:12:07.571 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary4 2024-09-12T13:12:07.586 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary7: 2024-09-12T13:12:07.586 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 73a79146-a78a-4543-843c-6cef99c271d3 2024-09-12T13:12:07.586 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:12:07.586 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:12:07.586 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:12:07.587 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:11:47' 2024-09-12T13:12:07.587 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:12:07.589 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:12:07.589 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:12:07.589 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 8 2024-09-12T13:12:07.624 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary10: 2024-09-12T13:12:07.625 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 2286adb1-ff3d-442a-9aac-6b38a8513aaa 2024-09-12T13:12:07.625 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:12:07.625 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:12:07.625 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:12:07.625 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:11:47' 2024-09-12T13:12:07.625 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:12:07.627 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:12:07.627 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:12:07.627 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 8 2024-09-12T13:12:07.658 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary4: 2024-09-12T13:12:07.659 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 11f002da-1dfb-4012-a01a-c5237417a099 2024-09-12T13:12:07.659 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:12:07.659 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:12:07.659 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:12:07.659 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:11:47' 2024-09-12T13:12:07.659 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:12:07.661 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:12:07.661 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:12:07.661 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 8 2024-09-12T13:12:07.664 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary3: 2024-09-12T13:12:07.665 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 92498ee6-eac1-4596-be69-c222174caeb6 2024-09-12T13:12:07.665 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:12:07.665 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:12:07.665 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:12:07.665 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:11:47' 2024-09-12T13:12:07.665 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:12:07.667 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:12:07.667 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:12:07.667 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 8 2024-09-12T13:12:08.417 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary8 up+unknown '' '' 2024-09-12T13:12:08.417 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:12:08.417 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:12:08.417 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:12:08.418 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:12:08.418 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:12:08.418 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:12:08.418 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:12:08.418 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:12:08.418 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary8 2024-09-12T13:12:08.439 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary9 up+unknown '' '' 2024-09-12T13:12:08.439 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:12:08.439 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:12:08.439 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:12:08.439 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:12:08.439 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:12:08.439 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:12:08.440 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:12:08.440 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:12:08.440 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary9 2024-09-12T13:12:08.501 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary8: 2024-09-12T13:12:08.501 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: fa7b2037-727d-4ca0-abbc-a51e250f5ecf 2024-09-12T13:12:08.502 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:12:08.502 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:12:08.502 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:12:08.502 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:11:47' 2024-09-12T13:12:08.502 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:12:08.504 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:12:08.504 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:12:08.504 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 8 2024-09-12T13:12:08.519 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary9: 2024-09-12T13:12:08.519 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 77edc192-6758-44c1-b93f-e8f7df97992a 2024-09-12T13:12:08.520 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:12:08.520 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:12:08.520 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:12:08.520 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:11:47' 2024-09-12T13:12:08.520 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:12:08.522 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:12:08.522 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:12:08.523 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 8 2024-09-12T13:12:10.628 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary6 up+unknown '' '' 2024-09-12T13:12:10.629 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:12:10.629 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:12:10.629 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:12:10.629 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:12:10.629 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:12:10.629 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:12:10.629 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:12:10.629 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:12:10.630 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary6 2024-09-12T13:12:10.713 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary6: 2024-09-12T13:12:10.713 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: b711ef4c-c601-46a4-a872-1c8abdd1ca45 2024-09-12T13:12:10.713 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:12:10.713 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:12:10.713 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:12:10.713 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:11:47' 2024-09-12T13:12:10.713 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:12:10.715 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:12:10.716 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:12:10.716 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 8 2024-09-12T13:12:11.009 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary1 up+unknown '' '' 2024-09-12T13:12:11.009 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:12:11.009 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:12:11.009 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:12:11.009 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:12:11.009 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:12:11.010 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:12:11.010 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:12:11.010 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:12:11.010 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary1 2024-09-12T13:12:11.093 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary1: 2024-09-12T13:12:11.093 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 1df40b54-54f7-41fe-a2f4-c900ccf2e016 2024-09-12T13:12:11.093 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:12:11.093 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:12:11.093 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:12:11.093 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:11:47' 2024-09-12T13:12:11.093 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:12:11.096 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:12:11.096 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:12:11.096 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 8 2024-09-12T13:12:11.652 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary2 up+unknown '' '' 2024-09-12T13:12:11.652 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:12:11.652 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:12:11.653 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:12:11.653 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:12:11.653 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:12:11.653 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:12:11.653 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:12:11.653 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:12:11.653 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary2 2024-09-12T13:12:11.733 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary2: 2024-09-12T13:12:11.733 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 23ee0857-2880-4d74-a09e-17095d384c6b 2024-09-12T13:12:11.733 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:12:11.733 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:12:11.733 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:12:11.733 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:11:47' 2024-09-12T13:12:11.733 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:12:11.735 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:12:11.736 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:12:11.736 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 8 2024-09-12T13:12:13.123 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary5 up+unknown '' '' 2024-09-12T13:12:13.123 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:12:13.123 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:12:13.123 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:12:13.123 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:12:13.123 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:12:13.123 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:12:13.123 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:12:13.123 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:12:13.124 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary5 2024-09-12T13:12:13.208 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary5: 2024-09-12T13:12:13.208 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 261d375d-e3b2-4dc4-91bd-36dfede8b276 2024-09-12T13:12:13.208 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:12:13.208 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:12:13.208 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:12:13.208 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:11:47' 2024-09-12T13:12:13.208 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:12:13.211 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:12:13.211 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:12:13.211 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 8 2024-09-12T13:12:15.592 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary7 up+unknown '' '' 2024-09-12T13:12:15.592 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:12:15.592 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:12:15.592 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:12:15.592 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:12:15.592 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:12:15.592 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:12:15.592 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:12:15.593 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:12:15.593 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary7 2024-09-12T13:12:15.630 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary10 up+unknown '' '' 2024-09-12T13:12:15.630 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:12:15.630 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:12:15.630 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:12:15.630 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:12:15.630 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:12:15.630 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:12:15.630 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:12:15.631 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:12:15.631 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary10 2024-09-12T13:12:15.664 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary4 up+unknown '' '' 2024-09-12T13:12:15.664 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:12:15.664 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:12:15.664 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:12:15.664 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:12:15.664 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:12:15.664 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:12:15.664 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:12:15.665 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:12:15.665 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary4 2024-09-12T13:12:15.670 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary3 up+unknown '' '' 2024-09-12T13:12:15.670 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:12:15.670 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:12:15.670 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:12:15.670 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:12:15.670 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:12:15.670 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:12:15.670 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:12:15.671 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:12:15.671 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary3 2024-09-12T13:12:15.677 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary7: 2024-09-12T13:12:15.677 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 73a79146-a78a-4543-843c-6cef99c271d3 2024-09-12T13:12:15.678 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:12:15.678 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:12:15.678 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:12:15.678 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:11:47' 2024-09-12T13:12:15.678 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:12:15.681 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:12:15.681 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:12:15.681 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 8 2024-09-12T13:12:15.717 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary10: 2024-09-12T13:12:15.717 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 2286adb1-ff3d-442a-9aac-6b38a8513aaa 2024-09-12T13:12:15.717 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:12:15.717 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:12:15.717 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:12:15.717 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:11:47' 2024-09-12T13:12:15.717 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:12:15.720 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:12:15.720 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:12:15.720 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 8 2024-09-12T13:12:15.759 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary3: 2024-09-12T13:12:15.759 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 92498ee6-eac1-4596-be69-c222174caeb6 2024-09-12T13:12:15.759 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:12:15.759 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:12:15.759 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:12:15.760 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:11:47' 2024-09-12T13:12:15.760 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:12:15.762 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:12:15.762 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:12:15.762 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 8 2024-09-12T13:12:15.953 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary4: 2024-09-12T13:12:15.954 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 11f002da-1dfb-4012-a01a-c5237417a099 2024-09-12T13:12:15.954 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:12:15.954 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:12:15.954 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:12:15.954 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:11:47' 2024-09-12T13:12:15.954 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:12:15.956 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:12:15.956 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:12:15.956 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 8 2024-09-12T13:12:16.507 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary8 up+unknown '' '' 2024-09-12T13:12:16.507 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:12:16.507 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:12:16.507 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:12:16.507 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:12:16.507 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:12:16.507 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:12:16.507 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:12:16.508 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:12:16.508 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary8 2024-09-12T13:12:16.525 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary9 up+unknown '' '' 2024-09-12T13:12:16.525 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:12:16.525 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:12:16.525 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:12:16.525 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:12:16.525 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:12:16.525 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:12:16.525 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:12:16.526 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:12:16.526 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary9 2024-09-12T13:12:16.586 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary8: 2024-09-12T13:12:16.586 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: fa7b2037-727d-4ca0-abbc-a51e250f5ecf 2024-09-12T13:12:16.587 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:12:16.587 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:12:16.587 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:12:16.587 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:11:47' 2024-09-12T13:12:16.587 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:12:16.589 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:12:16.589 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:12:16.589 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 8 2024-09-12T13:12:16.608 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary9: 2024-09-12T13:12:16.608 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 77edc192-6758-44c1-b93f-e8f7df97992a 2024-09-12T13:12:16.608 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:12:16.608 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:12:16.609 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:12:16.609 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:11:47' 2024-09-12T13:12:16.609 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:12:16.611 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:12:16.611 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:12:16.611 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 8 2024-09-12T13:12:18.718 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary6 up+unknown '' '' 2024-09-12T13:12:18.719 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:12:18.719 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:12:18.719 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:12:18.719 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:12:18.719 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:12:18.719 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:12:18.719 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:12:18.719 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:12:18.719 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary6 2024-09-12T13:12:18.803 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary6: 2024-09-12T13:12:18.804 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: b711ef4c-c601-46a4-a872-1c8abdd1ca45 2024-09-12T13:12:18.804 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+unknown 2024-09-12T13:12:18.804 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: remote image is not primary 2024-09-12T13:12:18.804 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:12:18.804 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:12:17' 2024-09-12T13:12:18.804 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:12:18.806 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+unknown 2024-09-12T13:12:18.806 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:12:18.809 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: remote image is not primary 2024-09-12T13:12:18.809 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:12:18.810 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:12:18.810 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:12:18.812 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:12:18.813 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:12:18.815 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:12:18.815 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:12:18.816 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:12:18.821 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:12:17' 2024-09-12T13:12:18.821 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary6 up+unknown '2024-09-12 13:12:17' 2024-09-12T13:12:18.821 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:12:18.821 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:12:18.821 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:12:18.821 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:12:18.821 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:12:17' 2024-09-12T13:12:18.821 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:12:18.822 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:12:18.822 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-09-12T13:12:18.931 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041934232mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary11df40b54-54f7-41fe-a2f4-c900ccf2e016up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:12:17image-primary102286adb1-ff3d-442a-9aac-6b38a8513aaaup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:12:17image-primary223ee0857-2880-4d74-a09e-17095d384c6bup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:12:17image-primary392498ee6-eac1-4596-be69-c222174caeb6up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:12:17image-primary411f002da-1dfb-4012-a01a-c5237417a099up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:12:17image-primary5261d375d-e3b2-4dc4-91bd-36dfede8b276up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:12:17image-primary6b711ef4c-c601-46a4-a872-1c8abdd1ca45up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:12:17image-primary773a79146-a78a-4543-843c-6cef99c271d3up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:12:17image-primary8fa7b2037-727d-4ca0-abbc-a51e250f5ecfup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:12:17image-primary977edc192-6758-44c1-b93f-e8f7df97992aup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:12:17' 2024-09-12T13:12:18.931 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:12:18.932 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary6'\'']/last_update' 2024-09-12T13:12:18.964 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:12:17' 2024-09-12T13:12:18.965 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary6'\'']/state' 2024-09-12T13:12:18.969 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+unknown 2024-09-12T13:12:18.969 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:12:18.970 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+unknown 2024-09-12T13:12:18.971 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:12:18.972 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:12:18.972 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:12:18.972 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster2 mirror image-primary6 up+unknown 2024-09-12T13:12:18.972 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:12:18.973 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:12:18.973 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:12:18.973 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:12:18.973 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:12:18.973 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:12:18.973 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:12:18.973 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:12:19.098 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary1 up+unknown '' '' 2024-09-12T13:12:19.099 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:12:19.099 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:12:19.099 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:12:19.099 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:12:19.099 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:12:19.099 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:12:19.099 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:12:19.099 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:12:19.100 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary1 2024-09-12T13:12:19.173 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary1: 2024-09-12T13:12:19.173 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 1df40b54-54f7-41fe-a2f4-c900ccf2e016 2024-09-12T13:12:19.173 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+unknown 2024-09-12T13:12:19.173 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: remote image is not primary 2024-09-12T13:12:19.173 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:12:19.173 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:12:17' 2024-09-12T13:12:19.173 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:12:19.175 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+unknown 2024-09-12T13:12:19.175 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:12:19.178 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: remote image is not primary 2024-09-12T13:12:19.178 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:12:19.179 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:12:19.179 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:12:19.181 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:12:19.181 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:12:19.184 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:12:19.184 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:12:19.185 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:12:19.189 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:12:17' 2024-09-12T13:12:19.189 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary1 up+unknown '2024-09-12 13:12:17' 2024-09-12T13:12:19.189 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:12:19.189 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:12:19.189 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:12:19.189 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:12:19.189 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:12:17' 2024-09-12T13:12:19.189 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:12:19.190 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:12:19.190 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-09-12T13:12:19.294 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041934232mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary11df40b54-54f7-41fe-a2f4-c900ccf2e016up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:12:17image-primary102286adb1-ff3d-442a-9aac-6b38a8513aaaup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:12:17image-primary223ee0857-2880-4d74-a09e-17095d384c6bup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:12:17image-primary392498ee6-eac1-4596-be69-c222174caeb6up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:12:17image-primary411f002da-1dfb-4012-a01a-c5237417a099up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:12:17image-primary5261d375d-e3b2-4dc4-91bd-36dfede8b276up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:12:17image-primary6b711ef4c-c601-46a4-a872-1c8abdd1ca45up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:12:17image-primary773a79146-a78a-4543-843c-6cef99c271d3up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:12:17image-primary8fa7b2037-727d-4ca0-abbc-a51e250f5ecfup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:12:17image-primary977edc192-6758-44c1-b93f-e8f7df97992aup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:12:17' 2024-09-12T13:12:19.294 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:12:19.295 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary1'\'']/last_update' 2024-09-12T13:12:19.299 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:12:17' 2024-09-12T13:12:19.300 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary1'\'']/state' 2024-09-12T13:12:19.303 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+unknown 2024-09-12T13:12:19.304 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:12:19.304 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+unknown 2024-09-12T13:12:19.306 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:12:19.308 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:12:19.308 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:12:19.308 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster2 mirror image-primary1 up+unknown 2024-09-12T13:12:19.308 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:12:19.308 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:12:19.308 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:12:19.308 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:12:19.309 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:12:19.309 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:12:19.309 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:12:19.309 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:12:19.739 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary2 up+unknown '' '' 2024-09-12T13:12:19.739 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:12:19.739 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:12:19.739 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:12:19.739 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:12:19.739 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:12:19.739 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:12:19.739 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:12:19.740 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:12:19.740 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary2 2024-09-12T13:12:19.816 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary2: 2024-09-12T13:12:19.817 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 23ee0857-2880-4d74-a09e-17095d384c6b 2024-09-12T13:12:19.817 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+unknown 2024-09-12T13:12:19.817 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: remote image is not primary 2024-09-12T13:12:19.817 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:12:19.817 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:12:17' 2024-09-12T13:12:19.817 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:12:19.819 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+unknown 2024-09-12T13:12:19.819 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:12:19.822 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: remote image is not primary 2024-09-12T13:12:19.822 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:12:19.823 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:12:19.823 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:12:19.825 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:12:19.825 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:12:19.828 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:12:19.828 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:12:19.829 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:12:19.833 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:12:17' 2024-09-12T13:12:19.833 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary2 up+unknown '2024-09-12 13:12:17' 2024-09-12T13:12:19.833 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:12:19.833 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:12:19.833 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:12:19.833 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:12:19.833 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:12:17' 2024-09-12T13:12:19.833 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:12:19.834 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:12:19.834 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-09-12T13:12:19.938 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041934232mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary11df40b54-54f7-41fe-a2f4-c900ccf2e016up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:12:17image-primary102286adb1-ff3d-442a-9aac-6b38a8513aaaup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:12:17image-primary223ee0857-2880-4d74-a09e-17095d384c6bup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:12:17image-primary392498ee6-eac1-4596-be69-c222174caeb6up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:12:17image-primary411f002da-1dfb-4012-a01a-c5237417a099up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:12:17image-primary5261d375d-e3b2-4dc4-91bd-36dfede8b276up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:12:17image-primary6b711ef4c-c601-46a4-a872-1c8abdd1ca45up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:12:17image-primary773a79146-a78a-4543-843c-6cef99c271d3up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:12:17image-primary8fa7b2037-727d-4ca0-abbc-a51e250f5ecfup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:12:17image-primary977edc192-6758-44c1-b93f-e8f7df97992aup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:12:17' 2024-09-12T13:12:19.939 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:12:19.939 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary2'\'']/last_update' 2024-09-12T13:12:19.943 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:12:17' 2024-09-12T13:12:19.944 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary2'\'']/state' 2024-09-12T13:12:19.947 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+unknown 2024-09-12T13:12:19.948 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:12:19.948 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+unknown 2024-09-12T13:12:19.950 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:12:19.951 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:12:19.951 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:12:19.951 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster2 mirror image-primary2 up+unknown 2024-09-12T13:12:19.951 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:12:19.951 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:12:19.951 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:12:19.952 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:12:19.952 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:12:19.952 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:12:19.952 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:12:19.952 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:12:19.974 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster2 mirror image-primary6 up+unknown '' '' 2024-09-12T13:12:19.974 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:12:19.975 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:12:19.975 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:12:19.975 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:12:19.975 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:12:19.975 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:12:19.975 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:12:19.975 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:12:19.975 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster2 mirror image status mirror/image-primary6 2024-09-12T13:12:20.038 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary6: 2024-09-12T13:12:20.038 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: b711ef4c-c601-46a4-a872-1c8abdd1ca45 2024-09-12T13:12:20.038 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+unknown 2024-09-12T13:12:20.038 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: remote image is not primary 2024-09-12T13:12:20.038 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:12:20.038 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:12:17' 2024-09-12T13:12:20.039 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:12:20.040 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+unknown 2024-09-12T13:12:20.041 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:12:20.043 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: remote image is not primary 2024-09-12T13:12:20.044 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:12:20.044 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:12:20.044 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:12:20.046 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:12:20.047 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:12:20.049 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:12:20.049 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:12:20.050 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:12:20.054 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:12:17' 2024-09-12T13:12:20.054 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster2 mirror image-primary6 up+unknown '2024-09-12 13:12:17' 2024-09-12T13:12:20.054 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:12:20.054 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:12:20.054 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:12:20.054 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:12:20.054 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:12:17' 2024-09-12T13:12:20.054 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:12:20.055 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:12:20.055 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2024-09-12T13:12:20.151 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041924231mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary11df40b54-54f7-41fe-a2f4-c900ccf2e016up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:12:17image-primary102286adb1-ff3d-442a-9aac-6b38a8513aaaup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:12:17image-primary223ee0857-2880-4d74-a09e-17095d384c6bup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:12:17image-primary392498ee6-eac1-4596-be69-c222174caeb6up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:12:17image-primary411f002da-1dfb-4012-a01a-c5237417a099up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:12:17image-primary5261d375d-e3b2-4dc4-91bd-36dfede8b276up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:12:17image-primary6b711ef4c-c601-46a4-a872-1c8abdd1ca45up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:12:17image-primary773a79146-a78a-4543-843c-6cef99c271d3up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:12:17image-primary8fa7b2037-727d-4ca0-abbc-a51e250f5ecfup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:12:17image-primary977edc192-6758-44c1-b93f-e8f7df97992aup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:12:17' 2024-09-12T13:12:20.151 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:12:20.152 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary6'\'']/last_update' 2024-09-12T13:12:20.156 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:12:17' 2024-09-12T13:12:20.156 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary6'\'']/state' 2024-09-12T13:12:20.160 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+unknown 2024-09-12T13:12:20.161 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:12:20.161 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+unknown 2024-09-12T13:12:20.163 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:12:20.164 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:12:20.164 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:12:20.164 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ promote_image cluster2 mirror image-primary6 2024-09-12T13:12:20.164 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:12:20.164 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:12:20.164 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:12:20.165 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local force= 2024-09-12T13:12:20.165 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 mirror image promote mirror/image-primary6 2024-09-12T13:12:20.309 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster2 mirror image-primary1 up+unknown '' '' 2024-09-12T13:12:20.309 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:12:20.310 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:12:20.310 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:12:20.310 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:12:20.310 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:12:20.310 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:12:20.310 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:12:20.310 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:12:20.310 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster2 mirror image status mirror/image-primary1 2024-09-12T13:12:20.373 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary1: 2024-09-12T13:12:20.373 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 1df40b54-54f7-41fe-a2f4-c900ccf2e016 2024-09-12T13:12:20.373 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+unknown 2024-09-12T13:12:20.373 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: remote image is not primary 2024-09-12T13:12:20.373 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:12:20.374 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:12:17' 2024-09-12T13:12:20.374 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:12:20.376 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+unknown 2024-09-12T13:12:20.376 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:12:20.378 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: remote image is not primary 2024-09-12T13:12:20.378 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:12:20.379 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:12:20.379 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:12:20.381 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:12:20.382 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:12:20.384 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:12:20.385 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:12:20.385 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:12:20.390 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:12:17' 2024-09-12T13:12:20.390 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster2 mirror image-primary1 up+unknown '2024-09-12 13:12:17' 2024-09-12T13:12:20.390 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:12:20.390 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:12:20.390 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:12:20.390 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:12:20.390 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:12:17' 2024-09-12T13:12:20.391 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:12:20.391 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:12:20.391 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2024-09-12T13:12:20.489 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041924231mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary11df40b54-54f7-41fe-a2f4-c900ccf2e016up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:12:17image-primary102286adb1-ff3d-442a-9aac-6b38a8513aaaup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:12:17image-primary223ee0857-2880-4d74-a09e-17095d384c6bup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:12:17image-primary392498ee6-eac1-4596-be69-c222174caeb6up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:12:17image-primary411f002da-1dfb-4012-a01a-c5237417a099up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:12:17image-primary5261d375d-e3b2-4dc4-91bd-36dfede8b276up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:12:17image-primary6b711ef4c-c601-46a4-a872-1c8abdd1ca45up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:12:17image-primary773a79146-a78a-4543-843c-6cef99c271d3up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:12:17image-primary8fa7b2037-727d-4ca0-abbc-a51e250f5ecfup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:12:17image-primary977edc192-6758-44c1-b93f-e8f7df97992aup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:12:17' 2024-09-12T13:12:20.489 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:12:20.489 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary1'\'']/last_update' 2024-09-12T13:12:20.492 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:12:17' 2024-09-12T13:12:20.493 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary1'\'']/state' 2024-09-12T13:12:20.496 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+unknown 2024-09-12T13:12:20.497 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:12:20.497 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+unknown 2024-09-12T13:12:20.499 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:12:20.500 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:12:20.500 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:12:20.500 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ promote_image cluster2 mirror image-primary1 2024-09-12T13:12:20.500 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:12:20.500 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:12:20.500 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:12:20.501 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local force= 2024-09-12T13:12:20.501 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 mirror image promote mirror/image-primary1 2024-09-12T13:12:20.844 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Image promoted to primary 2024-09-12T13:12:20.846 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Image promoted to primary 2024-09-12T13:12:20.855 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \n\b\d ]] 2024-09-12T13:12:20.856 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \k\r\b\d ]] 2024-09-12T13:12:20.856 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo rbd --cluster cluster2 device map -t krbd mirror/image-primary1 2024-09-12T13:12:20.856 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \n\b\d ]] 2024-09-12T13:12:20.857 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \k\r\b\d ]] 2024-09-12T13:12:20.857 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo rbd --cluster cluster2 device map -t krbd mirror/image-primary6 2024-09-12T13:12:20.953 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster2 mirror image-primary2 up+unknown '' '' 2024-09-12T13:12:20.954 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:12:20.954 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:12:20.954 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:12:20.954 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:12:20.954 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:12:20.954 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:12:20.954 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:12:20.954 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:12:20.955 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster2 mirror image status mirror/image-primary2 2024-09-12T13:12:20.996 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ dev=/dev/rbd0 2024-09-12T13:12:20.997 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo md5sum /dev/rbd0 2024-09-12T13:12:20.997 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ awk '{print $1}' 2024-09-12T13:12:21.001 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ dev=/dev/rbd1 2024-09-12T13:12:21.002 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo md5sum /dev/rbd1 2024-09-12T13:12:21.002 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ awk '{print $1}' 2024-09-12T13:12:21.036 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary2: 2024-09-12T13:12:21.036 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 23ee0857-2880-4d74-a09e-17095d384c6b 2024-09-12T13:12:21.036 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+unknown 2024-09-12T13:12:21.036 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: remote image is not primary 2024-09-12T13:12:21.036 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:12:21.036 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:12:17' 2024-09-12T13:12:21.036 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:12:21.037 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+unknown 2024-09-12T13:12:21.037 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:12:21.040 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: remote image is not primary 2024-09-12T13:12:21.041 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:12:21.042 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:12:21.042 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:12:21.044 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:12:21.045 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:12:21.047 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:12:21.048 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:12:21.049 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:12:21.053 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:12:17' 2024-09-12T13:12:21.053 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster2 mirror image-primary2 up+unknown '2024-09-12 13:12:17' 2024-09-12T13:12:21.053 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:12:21.054 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:12:21.054 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:12:21.054 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:12:21.054 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:12:17' 2024-09-12T13:12:21.054 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:12:21.054 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:12:21.054 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2024-09-12T13:12:21.173 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041924231mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary11df40b54-54f7-41fe-a2f4-c900ccf2e016up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:12:17image-primary102286adb1-ff3d-442a-9aac-6b38a8513aaaup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:12:17image-primary223ee0857-2880-4d74-a09e-17095d384c6bup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:12:17image-primary392498ee6-eac1-4596-be69-c222174caeb6up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:12:17image-primary411f002da-1dfb-4012-a01a-c5237417a099up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:12:17image-primary5261d375d-e3b2-4dc4-91bd-36dfede8b276up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:12:17image-primary6b711ef4c-c601-46a4-a872-1c8abdd1ca45up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:12:17image-primary773a79146-a78a-4543-843c-6cef99c271d3up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:12:17image-primary8fa7b2037-727d-4ca0-abbc-a51e250f5ecfup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:12:17image-primary977edc192-6758-44c1-b93f-e8f7df97992aup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:12:17' 2024-09-12T13:12:21.173 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:12:21.173 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary2'\'']/last_update' 2024-09-12T13:12:21.177 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:12:17' 2024-09-12T13:12:21.178 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary2'\'']/state' 2024-09-12T13:12:21.182 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+unknown 2024-09-12T13:12:21.182 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:12:21.183 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+unknown 2024-09-12T13:12:21.185 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:12:21.186 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:12:21.186 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:12:21.186 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ promote_image cluster2 mirror image-primary2 2024-09-12T13:12:21.187 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:12:21.187 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:12:21.187 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:12:21.187 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local force= 2024-09-12T13:12:21.187 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 mirror image promote mirror/image-primary2 2024-09-12T13:12:21.214 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary5 up+unknown '' '' 2024-09-12T13:12:21.214 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:12:21.214 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:12:21.214 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:12:21.214 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:12:21.214 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:12:21.214 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:12:21.214 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:12:21.215 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:12:21.215 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary5 2024-09-12T13:12:21.299 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary5: 2024-09-12T13:12:21.299 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 261d375d-e3b2-4dc4-91bd-36dfede8b276 2024-09-12T13:12:21.299 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+unknown 2024-09-12T13:12:21.299 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: remote image is not primary 2024-09-12T13:12:21.299 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:12:21.299 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:12:17' 2024-09-12T13:12:21.299 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:12:21.300 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+unknown 2024-09-12T13:12:21.301 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:12:21.304 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: remote image is not primary 2024-09-12T13:12:21.304 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:12:21.304 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:12:21.305 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:12:21.307 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:12:21.307 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:12:21.311 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:12:21.311 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:12:21.311 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:12:21.315 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:12:17' 2024-09-12T13:12:21.315 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary5 up+unknown '2024-09-12 13:12:17' 2024-09-12T13:12:21.315 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:12:21.316 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:12:21.316 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:12:21.316 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:12:21.316 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:12:17' 2024-09-12T13:12:21.316 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:12:21.316 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:12:21.316 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-09-12T13:12:21.435 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041934232mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary11df40b54-54f7-41fe-a2f4-c900ccf2e016up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:12:17image-primary102286adb1-ff3d-442a-9aac-6b38a8513aaaup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:12:17image-primary223ee0857-2880-4d74-a09e-17095d384c6bup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:12:17image-primary392498ee6-eac1-4596-be69-c222174caeb6up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:12:17image-primary411f002da-1dfb-4012-a01a-c5237417a099up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:12:17image-primary5261d375d-e3b2-4dc4-91bd-36dfede8b276up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:12:17image-primary6b711ef4c-c601-46a4-a872-1c8abdd1ca45up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:12:17image-primary773a79146-a78a-4543-843c-6cef99c271d3up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:12:17image-primary8fa7b2037-727d-4ca0-abbc-a51e250f5ecfup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:12:17image-primary977edc192-6758-44c1-b93f-e8f7df97992aup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:12:17' 2024-09-12T13:12:21.435 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:12:21.435 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary5'\'']/last_update' 2024-09-12T13:12:21.439 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:12:17' 2024-09-12T13:12:21.440 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary5'\'']/state' 2024-09-12T13:12:21.443 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+unknown 2024-09-12T13:12:21.444 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:12:21.444 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+unknown 2024-09-12T13:12:21.446 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:12:21.447 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:12:21.447 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:12:21.447 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster2 mirror image-primary5 up+unknown 2024-09-12T13:12:21.447 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:12:21.447 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:12:21.448 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:12:21.448 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:12:21.448 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:12:21.448 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:12:21.448 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:12:21.448 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:12:21.857 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Image promoted to primary 2024-09-12T13:12:21.872 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \n\b\d ]] 2024-09-12T13:12:21.872 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \k\r\b\d ]] 2024-09-12T13:12:21.872 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo rbd --cluster cluster2 device map -t krbd mirror/image-primary2 2024-09-12T13:12:22.002 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ dev=/dev/rbd2 2024-09-12T13:12:22.004 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo md5sum /dev/rbd2 2024-09-12T13:12:22.004 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ awk '{print $1}' 2024-09-12T13:12:22.450 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster2 mirror image-primary5 up+unknown '' '' 2024-09-12T13:12:22.450 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:12:22.450 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:12:22.450 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:12:22.450 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:12:22.451 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:12:22.451 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:12:22.451 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:12:22.451 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:12:22.451 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster2 mirror image status mirror/image-primary5 2024-09-12T13:12:22.523 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary5: 2024-09-12T13:12:22.523 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 261d375d-e3b2-4dc4-91bd-36dfede8b276 2024-09-12T13:12:22.523 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+unknown 2024-09-12T13:12:22.523 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: remote image is not primary 2024-09-12T13:12:22.523 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:12:22.523 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:12:17' 2024-09-12T13:12:22.523 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:12:22.525 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+unknown 2024-09-12T13:12:22.525 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:12:22.528 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: remote image is not primary 2024-09-12T13:12:22.528 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:12:22.533 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:12:22.533 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:12:22.533 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:12:22.534 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:12:22.535 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:12:22.535 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:12:22.536 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:12:22.540 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:12:17' 2024-09-12T13:12:22.540 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster2 mirror image-primary5 up+unknown '2024-09-12 13:12:17' 2024-09-12T13:12:22.541 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:12:22.541 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:12:22.541 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:12:22.541 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:12:22.541 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:12:17' 2024-09-12T13:12:22.541 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:12:22.541 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:12:22.541 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2024-09-12T13:12:22.669 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041924231mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary11df40b54-54f7-41fe-a2f4-c900ccf2e016up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:12:17image-primary102286adb1-ff3d-442a-9aac-6b38a8513aaaup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:12:17image-primary223ee0857-2880-4d74-a09e-17095d384c6bup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:12:17image-primary392498ee6-eac1-4596-be69-c222174caeb6up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:12:17image-primary411f002da-1dfb-4012-a01a-c5237417a099up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:12:17image-primary5261d375d-e3b2-4dc4-91bd-36dfede8b276up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:12:17image-primary6b711ef4c-c601-46a4-a872-1c8abdd1ca45up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:12:17image-primary773a79146-a78a-4543-843c-6cef99c271d3up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:12:17image-primary8fa7b2037-727d-4ca0-abbc-a51e250f5ecfup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:12:17image-primary977edc192-6758-44c1-b93f-e8f7df97992aup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:12:17' 2024-09-12T13:12:22.670 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:12:22.670 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary5'\'']/last_update' 2024-09-12T13:12:22.674 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:12:17' 2024-09-12T13:12:22.675 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary5'\'']/state' 2024-09-12T13:12:22.679 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+unknown 2024-09-12T13:12:22.680 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:12:22.681 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+unknown 2024-09-12T13:12:22.683 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:12:22.684 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:12:22.684 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:12:22.684 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ promote_image cluster2 mirror image-primary5 2024-09-12T13:12:22.684 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:12:22.684 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:12:22.684 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:12:22.684 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local force= 2024-09-12T13:12:22.684 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 mirror image promote mirror/image-primary5 2024-09-12T13:12:22.863 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Image promoted to primary 2024-09-12T13:12:22.875 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \n\b\d ]] 2024-09-12T13:12:22.876 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \k\r\b\d ]] 2024-09-12T13:12:22.876 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo rbd --cluster cluster2 device map -t krbd mirror/image-primary5 2024-09-12T13:12:23.010 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ dev=/dev/rbd3 2024-09-12T13:12:23.012 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo md5sum /dev/rbd3 2024-09-12T13:12:23.012 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ awk '{print $1}' 2024-09-12T13:12:23.684 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary7 up+unknown '' '' 2024-09-12T13:12:23.684 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:12:23.684 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:12:23.684 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:12:23.684 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:12:23.684 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:12:23.684 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:12:23.684 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:12:23.685 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:12:23.685 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary7 2024-09-12T13:12:23.723 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary10 up+unknown '' '' 2024-09-12T13:12:23.723 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:12:23.723 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:12:23.723 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:12:23.723 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:12:23.723 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:12:23.723 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:12:23.723 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:12:23.724 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:12:23.724 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary10 2024-09-12T13:12:23.765 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary3 up+unknown '' '' 2024-09-12T13:12:23.765 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:12:23.765 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:12:23.765 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:12:23.765 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:12:23.765 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:12:23.765 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:12:23.765 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:12:23.766 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:12:23.766 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary3 2024-09-12T13:12:23.791 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary7: 2024-09-12T13:12:23.791 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 73a79146-a78a-4543-843c-6cef99c271d3 2024-09-12T13:12:23.791 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+unknown 2024-09-12T13:12:23.791 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: remote image is not primary 2024-09-12T13:12:23.791 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:12:23.791 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:12:17' 2024-09-12T13:12:23.791 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:12:23.795 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+unknown 2024-09-12T13:12:23.797 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:12:23.801 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: remote image is not primary 2024-09-12T13:12:23.802 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:12:23.803 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:12:23.803 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:12:23.807 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:12:23.808 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:12:23.812 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:12:23.813 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:12:23.814 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:12:23.820 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:12:17' 2024-09-12T13:12:23.821 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary7 up+unknown '2024-09-12 13:12:17' 2024-09-12T13:12:23.821 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:12:23.821 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:12:23.821 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:12:23.821 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:12:23.821 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:12:17' 2024-09-12T13:12:23.821 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:12:23.821 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:12:23.821 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-09-12T13:12:23.841 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary10: 2024-09-12T13:12:23.841 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 2286adb1-ff3d-442a-9aac-6b38a8513aaa 2024-09-12T13:12:23.842 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+unknown 2024-09-12T13:12:23.842 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: remote image is not primary 2024-09-12T13:12:23.842 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:12:23.842 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:12:17' 2024-09-12T13:12:23.842 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:12:23.845 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+unknown 2024-09-12T13:12:23.846 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:12:23.851 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: remote image is not primary 2024-09-12T13:12:23.852 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:12:23.853 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:12:23.853 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:12:23.856 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:12:23.856 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:12:23.860 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:12:23.860 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:12:23.862 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:12:23.866 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary3: 2024-09-12T13:12:23.866 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 92498ee6-eac1-4596-be69-c222174caeb6 2024-09-12T13:12:23.866 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+unknown 2024-09-12T13:12:23.866 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: remote image is not primary 2024-09-12T13:12:23.866 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:12:23.866 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:12:17' 2024-09-12T13:12:23.867 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:12:23.868 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:12:17' 2024-09-12T13:12:23.868 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary10 up+unknown '2024-09-12 13:12:17' 2024-09-12T13:12:23.868 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:12:23.870 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:12:23.870 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:12:23.870 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:12:23.870 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:12:17' 2024-09-12T13:12:23.870 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:12:23.871 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:12:23.871 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-09-12T13:12:23.871 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+unknown 2024-09-12T13:12:23.871 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:12:23.876 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: remote image is not primary 2024-09-12T13:12:23.878 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:12:23.879 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:12:23.879 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:12:23.883 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:12:23.883 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:12:23.888 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:12:23.889 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:12:23.890 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:12:23.898 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:12:17' 2024-09-12T13:12:23.898 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary3 up+unknown '2024-09-12 13:12:17' 2024-09-12T13:12:23.898 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:12:23.898 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:12:23.898 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:12:23.898 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:12:23.898 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:12:17' 2024-09-12T13:12:23.898 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:12:23.899 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:12:23.899 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-09-12T13:12:23.960 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary4 up+unknown '' '' 2024-09-12T13:12:23.961 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:12:23.961 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:12:23.961 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:12:23.961 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:12:23.961 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:12:23.961 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:12:23.961 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:12:23.961 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:12:23.961 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary4 2024-09-12T13:12:23.984 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041934232mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary11df40b54-54f7-41fe-a2f4-c900ccf2e016up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:12:17image-primary102286adb1-ff3d-442a-9aac-6b38a8513aaaup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:12:17image-primary223ee0857-2880-4d74-a09e-17095d384c6bup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:12:17image-primary392498ee6-eac1-4596-be69-c222174caeb6up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:12:17image-primary411f002da-1dfb-4012-a01a-c5237417a099up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:12:17image-primary5261d375d-e3b2-4dc4-91bd-36dfede8b276up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:12:17image-primary6b711ef4c-c601-46a4-a872-1c8abdd1ca45up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:12:17image-primary773a79146-a78a-4543-843c-6cef99c271d3up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:12:17image-primary8fa7b2037-727d-4ca0-abbc-a51e250f5ecfup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:12:17image-primary977edc192-6758-44c1-b93f-e8f7df97992aup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:12:17' 2024-09-12T13:12:23.984 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:12:23.985 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary7'\'']/last_update' 2024-09-12T13:12:23.992 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:12:17' 2024-09-12T13:12:23.994 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary7'\'']/state' 2024-09-12T13:12:24.001 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+unknown 2024-09-12T13:12:24.002 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:12:24.004 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+unknown 2024-09-12T13:12:24.008 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:12:24.010 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:12:24.010 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:12:24.010 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster2 mirror image-primary7 up+unknown 2024-09-12T13:12:24.010 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:12:24.010 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:12:24.010 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:12:24.010 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:12:24.010 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:12:24.010 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:12:24.010 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:12:24.010 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:12:24.053 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041934232mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary11df40b54-54f7-41fe-a2f4-c900ccf2e016up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:12:17image-primary102286adb1-ff3d-442a-9aac-6b38a8513aaaup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:12:17image-primary223ee0857-2880-4d74-a09e-17095d384c6bup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:12:17image-primary392498ee6-eac1-4596-be69-c222174caeb6up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:12:17image-primary411f002da-1dfb-4012-a01a-c5237417a099up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:12:17image-primary5261d375d-e3b2-4dc4-91bd-36dfede8b276up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:12:17image-primary6b711ef4c-c601-46a4-a872-1c8abdd1ca45up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:12:17image-primary773a79146-a78a-4543-843c-6cef99c271d3up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:12:17image-primary8fa7b2037-727d-4ca0-abbc-a51e250f5ecfup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:12:17image-primary977edc192-6758-44c1-b93f-e8f7df97992aup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:12:17' 2024-09-12T13:12:24.053 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:12:24.054 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary10'\'']/last_update' 2024-09-12T13:12:24.057 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041934232mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary11df40b54-54f7-41fe-a2f4-c900ccf2e016up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:12:17image-primary102286adb1-ff3d-442a-9aac-6b38a8513aaaup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:12:17image-primary223ee0857-2880-4d74-a09e-17095d384c6bup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:12:17image-primary392498ee6-eac1-4596-be69-c222174caeb6up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:12:17image-primary411f002da-1dfb-4012-a01a-c5237417a099up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:12:17image-primary5261d375d-e3b2-4dc4-91bd-36dfede8b276up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:12:17image-primary6b711ef4c-c601-46a4-a872-1c8abdd1ca45up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:12:17image-primary773a79146-a78a-4543-843c-6cef99c271d3up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:12:17image-primary8fa7b2037-727d-4ca0-abbc-a51e250f5ecfup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:12:17image-primary977edc192-6758-44c1-b93f-e8f7df97992aup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:12:17' 2024-09-12T13:12:24.057 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:12:24.058 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary3'\'']/last_update' 2024-09-12T13:12:24.060 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:12:17' 2024-09-12T13:12:24.061 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary10'\'']/state' 2024-09-12T13:12:24.064 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:12:17' 2024-09-12T13:12:24.065 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary3'\'']/state' 2024-09-12T13:12:24.066 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+unknown 2024-09-12T13:12:24.067 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:12:24.068 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+unknown 2024-09-12T13:12:24.071 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:12:24.072 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+unknown 2024-09-12T13:12:24.072 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:12:24.072 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:12:24.072 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster2 mirror image-primary10 up+unknown 2024-09-12T13:12:24.072 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:12:24.073 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:12:24.073 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:12:24.073 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:12:24.073 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:12:24.073 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:12:24.073 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:12:24.073 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:12:24.073 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:12:24.073 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+unknown 2024-09-12T13:12:24.075 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:12:24.076 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:12:24.076 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:12:24.076 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster2 mirror image-primary3 up+unknown 2024-09-12T13:12:24.076 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:12:24.077 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:12:24.077 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:12:24.077 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:12:24.077 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:12:24.077 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:12:24.077 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:12:24.077 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:12:24.077 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary4: 2024-09-12T13:12:24.078 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 11f002da-1dfb-4012-a01a-c5237417a099 2024-09-12T13:12:24.078 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+unknown 2024-09-12T13:12:24.078 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: remote image is not primary 2024-09-12T13:12:24.078 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:12:24.078 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:12:17' 2024-09-12T13:12:24.078 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:12:24.080 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+unknown 2024-09-12T13:12:24.080 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:12:24.083 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: remote image is not primary 2024-09-12T13:12:24.083 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:12:24.084 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:12:24.085 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:12:24.087 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:12:24.087 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:12:24.090 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:12:24.090 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:12:24.091 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:12:24.095 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:12:17' 2024-09-12T13:12:24.095 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary4 up+unknown '2024-09-12 13:12:17' 2024-09-12T13:12:24.095 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:12:24.095 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:12:24.095 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:12:24.095 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:12:24.095 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:12:17' 2024-09-12T13:12:24.096 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:12:24.096 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:12:24.096 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-09-12T13:12:24.231 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041934232mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary11df40b54-54f7-41fe-a2f4-c900ccf2e016up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:12:17image-primary102286adb1-ff3d-442a-9aac-6b38a8513aaaup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:12:17image-primary223ee0857-2880-4d74-a09e-17095d384c6bup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:12:17image-primary392498ee6-eac1-4596-be69-c222174caeb6up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:12:17image-primary411f002da-1dfb-4012-a01a-c5237417a099up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:12:17image-primary5261d375d-e3b2-4dc4-91bd-36dfede8b276up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:12:17image-primary6b711ef4c-c601-46a4-a872-1c8abdd1ca45up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:12:17image-primary773a79146-a78a-4543-843c-6cef99c271d3up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:12:17image-primary8fa7b2037-727d-4ca0-abbc-a51e250f5ecfup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:12:17image-primary977edc192-6758-44c1-b93f-e8f7df97992aup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:12:17' 2024-09-12T13:12:24.231 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:12:24.232 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary4'\'']/last_update' 2024-09-12T13:12:24.236 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:12:17' 2024-09-12T13:12:24.237 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary4'\'']/state' 2024-09-12T13:12:24.242 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+unknown 2024-09-12T13:12:24.242 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:12:24.243 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+unknown 2024-09-12T13:12:24.245 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:12:24.246 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:12:24.246 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:12:24.246 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster2 mirror image-primary4 up+unknown 2024-09-12T13:12:24.246 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:12:24.247 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:12:24.247 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:12:24.247 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:12:24.247 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:12:24.247 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:12:24.247 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:12:24.247 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:12:24.592 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary8 up+unknown '' '' 2024-09-12T13:12:24.592 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:12:24.592 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:12:24.593 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:12:24.593 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:12:24.593 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:12:24.593 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:12:24.593 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:12:24.593 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:12:24.594 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary8 2024-09-12T13:12:24.613 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary9 up+unknown '' '' 2024-09-12T13:12:24.614 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:12:24.614 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:12:24.614 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:12:24.614 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:12:24.614 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:12:24.614 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:12:24.614 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:12:24.615 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:12:24.615 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary9 2024-09-12T13:12:24.695 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary8: 2024-09-12T13:12:24.695 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: fa7b2037-727d-4ca0-abbc-a51e250f5ecf 2024-09-12T13:12:24.695 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+unknown 2024-09-12T13:12:24.695 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: remote image is not primary 2024-09-12T13:12:24.695 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:12:24.696 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:12:17' 2024-09-12T13:12:24.696 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:12:24.700 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+unknown 2024-09-12T13:12:24.700 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:12:24.705 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary9: 2024-09-12T13:12:24.705 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 77edc192-6758-44c1-b93f-e8f7df97992a 2024-09-12T13:12:24.705 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+unknown 2024-09-12T13:12:24.705 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: remote image is not primary 2024-09-12T13:12:24.705 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:12:24.705 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:12:17' 2024-09-12T13:12:24.706 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: remote image is not primary 2024-09-12T13:12:24.707 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:12:24.707 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:12:24.707 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:12:24.708 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:12:24.709 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+unknown 2024-09-12T13:12:24.710 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:12:24.710 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:12:24.711 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:12:24.714 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: remote image is not primary 2024-09-12T13:12:24.715 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:12:24.715 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:12:24.716 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:12:24.716 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:12:24.716 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:12:24.717 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:12:24.719 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:12:24.721 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:12:24.724 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:12:17' 2024-09-12T13:12:24.724 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary8 up+unknown '2024-09-12 13:12:17' 2024-09-12T13:12:24.724 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:12:24.724 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:12:24.724 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:12:24.725 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:12:24.725 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:12:17' 2024-09-12T13:12:24.725 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:12:24.725 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:12:24.725 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-09-12T13:12:24.726 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:12:24.726 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:12:24.726 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:12:24.733 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:12:17' 2024-09-12T13:12:24.733 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary9 up+unknown '2024-09-12 13:12:17' 2024-09-12T13:12:24.733 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:12:24.734 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:12:24.734 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:12:24.734 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:12:24.734 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:12:17' 2024-09-12T13:12:24.734 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:12:24.734 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:12:24.734 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-09-12T13:12:24.867 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041934232mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary11df40b54-54f7-41fe-a2f4-c900ccf2e016up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:12:17image-primary102286adb1-ff3d-442a-9aac-6b38a8513aaaup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:12:17image-primary223ee0857-2880-4d74-a09e-17095d384c6bup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:12:17image-primary392498ee6-eac1-4596-be69-c222174caeb6up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:12:17image-primary411f002da-1dfb-4012-a01a-c5237417a099up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:12:17image-primary5261d375d-e3b2-4dc4-91bd-36dfede8b276up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:12:17image-primary6b711ef4c-c601-46a4-a872-1c8abdd1ca45up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:12:17image-primary773a79146-a78a-4543-843c-6cef99c271d3up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:12:17image-primary8fa7b2037-727d-4ca0-abbc-a51e250f5ecfup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:12:17image-primary977edc192-6758-44c1-b93f-e8f7df97992aup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:12:17' 2024-09-12T13:12:24.868 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:12:24.868 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary8'\'']/last_update' 2024-09-12T13:12:24.873 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:12:17' 2024-09-12T13:12:24.873 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary8'\'']/state' 2024-09-12T13:12:24.877 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+unknown 2024-09-12T13:12:24.878 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:12:24.879 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+unknown 2024-09-12T13:12:24.881 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041934232mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary11df40b54-54f7-41fe-a2f4-c900ccf2e016up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:12:17image-primary102286adb1-ff3d-442a-9aac-6b38a8513aaaup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:12:17image-primary223ee0857-2880-4d74-a09e-17095d384c6bup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:12:17image-primary392498ee6-eac1-4596-be69-c222174caeb6up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:12:17image-primary411f002da-1dfb-4012-a01a-c5237417a099up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:12:17image-primary5261d375d-e3b2-4dc4-91bd-36dfede8b276up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:12:17image-primary6b711ef4c-c601-46a4-a872-1c8abdd1ca45up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:12:17image-primary773a79146-a78a-4543-843c-6cef99c271d3up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:12:17image-primary8fa7b2037-727d-4ca0-abbc-a51e250f5ecfup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:12:17image-primary977edc192-6758-44c1-b93f-e8f7df97992aup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:12:17' 2024-09-12T13:12:24.881 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:12:24.881 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:12:24.882 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary9'\'']/last_update' 2024-09-12T13:12:24.882 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:12:24.882 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:12:24.883 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster2 mirror image-primary8 up+unknown 2024-09-12T13:12:24.883 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:12:24.883 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:12:24.883 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:12:24.883 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:12:24.883 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:12:24.883 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:12:24.883 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:12:24.883 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:12:24.887 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:12:17' 2024-09-12T13:12:24.889 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary9'\'']/state' 2024-09-12T13:12:24.893 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+unknown 2024-09-12T13:12:24.894 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:12:24.894 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+unknown 2024-09-12T13:12:24.896 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:12:24.897 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:12:24.898 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:12:24.898 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster2 mirror image-primary9 up+unknown 2024-09-12T13:12:24.898 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:12:24.898 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:12:24.898 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:12:24.898 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:12:24.898 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:12:24.898 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:12:24.898 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:12:24.898 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:12:25.014 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster2 mirror image-primary7 up+unknown '' '' 2024-09-12T13:12:25.014 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:12:25.015 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:12:25.015 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:12:25.015 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:12:25.015 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:12:25.015 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:12:25.015 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:12:25.016 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:12:25.016 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster2 mirror image status mirror/image-primary7 2024-09-12T13:12:25.077 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster2 mirror image-primary10 up+unknown '' '' 2024-09-12T13:12:25.077 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:12:25.077 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:12:25.077 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:12:25.078 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:12:25.078 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:12:25.078 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:12:25.078 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:12:25.078 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:12:25.079 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster2 mirror image status mirror/image-primary10 2024-09-12T13:12:25.080 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster2 mirror image-primary3 up+unknown '' '' 2024-09-12T13:12:25.080 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:12:25.080 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:12:25.080 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:12:25.080 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:12:25.080 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:12:25.080 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:12:25.080 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:12:25.081 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:12:25.081 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster2 mirror image status mirror/image-primary3 2024-09-12T13:12:25.133 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary7: 2024-09-12T13:12:25.133 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 73a79146-a78a-4543-843c-6cef99c271d3 2024-09-12T13:12:25.133 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+unknown 2024-09-12T13:12:25.134 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: remote image is not primary 2024-09-12T13:12:25.134 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:12:25.134 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:12:17' 2024-09-12T13:12:25.134 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:12:25.136 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+unknown 2024-09-12T13:12:25.137 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:12:25.141 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: remote image is not primary 2024-09-12T13:12:25.142 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:12:25.144 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:12:25.144 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:12:25.147 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:12:25.147 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:12:25.152 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:12:25.152 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:12:25.153 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:12:25.160 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:12:17' 2024-09-12T13:12:25.160 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster2 mirror image-primary7 up+unknown '2024-09-12 13:12:17' 2024-09-12T13:12:25.160 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:12:25.160 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:12:25.160 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:12:25.160 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:12:25.160 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:12:17' 2024-09-12T13:12:25.161 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:12:25.161 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:12:25.161 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2024-09-12T13:12:25.190 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary10: 2024-09-12T13:12:25.191 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 2286adb1-ff3d-442a-9aac-6b38a8513aaa 2024-09-12T13:12:25.191 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+unknown 2024-09-12T13:12:25.191 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: remote image is not primary 2024-09-12T13:12:25.191 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:12:25.191 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:12:17' 2024-09-12T13:12:25.192 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:12:25.194 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+unknown 2024-09-12T13:12:25.195 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:12:25.199 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: remote image is not primary 2024-09-12T13:12:25.200 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:12:25.202 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:12:25.204 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:12:25.208 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:12:25.208 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:12:25.212 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:12:25.214 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:12:25.215 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:12:25.217 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary3: 2024-09-12T13:12:25.217 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 92498ee6-eac1-4596-be69-c222174caeb6 2024-09-12T13:12:25.217 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+unknown 2024-09-12T13:12:25.217 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: remote image is not primary 2024-09-12T13:12:25.217 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:12:25.217 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:12:17' 2024-09-12T13:12:25.217 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:12:25.221 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+unknown 2024-09-12T13:12:25.222 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:12:25.222 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:12:17' 2024-09-12T13:12:25.222 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster2 mirror image-primary10 up+unknown '2024-09-12 13:12:17' 2024-09-12T13:12:25.222 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:12:25.222 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:12:25.222 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:12:25.222 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:12:25.222 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:12:17' 2024-09-12T13:12:25.222 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:12:25.226 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: remote image is not primary 2024-09-12T13:12:25.226 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:12:25.226 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2024-09-12T13:12:25.226 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:12:25.227 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:12:25.228 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:12:25.231 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:12:25.232 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:12:25.236 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:12:25.237 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:12:25.238 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:12:25.244 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:12:17' 2024-09-12T13:12:25.244 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster2 mirror image-primary3 up+unknown '2024-09-12 13:12:17' 2024-09-12T13:12:25.245 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:12:25.245 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:12:25.245 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:12:25.245 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:12:25.245 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:12:17' 2024-09-12T13:12:25.245 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:12:25.246 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:12:25.246 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2024-09-12T13:12:25.258 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster2 mirror image-primary4 up+unknown '' '' 2024-09-12T13:12:25.258 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:12:25.258 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:12:25.258 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:12:25.258 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:12:25.258 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:12:25.258 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:12:25.258 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:12:25.263 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:12:25.263 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster2 mirror image status mirror/image-primary4 2024-09-12T13:12:25.346 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041924231mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary11df40b54-54f7-41fe-a2f4-c900ccf2e016up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:12:17image-primary102286adb1-ff3d-442a-9aac-6b38a8513aaaup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:12:17image-primary223ee0857-2880-4d74-a09e-17095d384c6bup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:12:17image-primary392498ee6-eac1-4596-be69-c222174caeb6up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:12:17image-primary411f002da-1dfb-4012-a01a-c5237417a099up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:12:17image-primary5261d375d-e3b2-4dc4-91bd-36dfede8b276up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:12:17image-primary6b711ef4c-c601-46a4-a872-1c8abdd1ca45up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:12:17image-primary773a79146-a78a-4543-843c-6cef99c271d3up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:12:17image-primary8fa7b2037-727d-4ca0-abbc-a51e250f5ecfup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:12:17image-primary977edc192-6758-44c1-b93f-e8f7df97992aup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:12:17' 2024-09-12T13:12:25.346 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:12:25.347 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary7'\'']/last_update' 2024-09-12T13:12:25.355 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:12:17' 2024-09-12T13:12:25.357 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary7'\'']/state' 2024-09-12T13:12:25.365 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+unknown 2024-09-12T13:12:25.366 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:12:25.366 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+unknown 2024-09-12T13:12:25.369 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:12:25.371 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:12:25.371 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:12:25.371 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ promote_image cluster2 mirror image-primary7 2024-09-12T13:12:25.371 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:12:25.371 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:12:25.372 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:12:25.372 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local force= 2024-09-12T13:12:25.372 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 mirror image promote mirror/image-primary7 2024-09-12T13:12:25.489 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary4: 2024-09-12T13:12:25.489 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 11f002da-1dfb-4012-a01a-c5237417a099 2024-09-12T13:12:25.489 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+unknown 2024-09-12T13:12:25.489 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: remote image is not primary 2024-09-12T13:12:25.489 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:12:25.489 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:12:17' 2024-09-12T13:12:25.489 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:12:25.497 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+unknown 2024-09-12T13:12:25.501 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:12:25.508 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: remote image is not primary 2024-09-12T13:12:25.510 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041924231mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary11df40b54-54f7-41fe-a2f4-c900ccf2e016up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:12:17image-primary102286adb1-ff3d-442a-9aac-6b38a8513aaaup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:12:17image-primary223ee0857-2880-4d74-a09e-17095d384c6bup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:12:17image-primary392498ee6-eac1-4596-be69-c222174caeb6up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:12:17image-primary411f002da-1dfb-4012-a01a-c5237417a099up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:12:17image-primary5261d375d-e3b2-4dc4-91bd-36dfede8b276up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:12:17image-primary6b711ef4c-c601-46a4-a872-1c8abdd1ca45up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:12:17image-primary773a79146-a78a-4543-843c-6cef99c271d3up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:12:17image-primary8fa7b2037-727d-4ca0-abbc-a51e250f5ecfup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:12:17image-primary977edc192-6758-44c1-b93f-e8f7df97992aup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:12:17' 2024-09-12T13:12:25.510 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:12:25.511 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:12:25.511 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary10'\'']/last_update' 2024-09-12T13:12:25.513 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:12:25.513 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:12:25.517 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:12:25.518 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:12:25.519 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:12:17' 2024-09-12T13:12:25.520 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary10'\'']/state' 2024-09-12T13:12:25.523 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:12:25.523 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:12:25.525 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:12:25.528 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+unknown 2024-09-12T13:12:25.529 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041924231mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary11df40b54-54f7-41fe-a2f4-c900ccf2e016up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:12:17image-primary102286adb1-ff3d-442a-9aac-6b38a8513aaaup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:12:17image-primary223ee0857-2880-4d74-a09e-17095d384c6bup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:12:17image-primary392498ee6-eac1-4596-be69-c222174caeb6up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:12:17image-primary411f002da-1dfb-4012-a01a-c5237417a099up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:12:17image-primary5261d375d-e3b2-4dc4-91bd-36dfede8b276up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:12:17image-primary6b711ef4c-c601-46a4-a872-1c8abdd1ca45up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:12:17image-primary773a79146-a78a-4543-843c-6cef99c271d3up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:12:17image-primary8fa7b2037-727d-4ca0-abbc-a51e250f5ecfup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:12:17image-primary977edc192-6758-44c1-b93f-e8f7df97992aup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:12:17' 2024-09-12T13:12:25.530 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:12:25.530 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:12:25.530 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+unknown 2024-09-12T13:12:25.532 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:12:17' 2024-09-12T13:12:25.532 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster2 mirror image-primary4 up+unknown '2024-09-12 13:12:17' 2024-09-12T13:12:25.533 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:12:25.533 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:12:25.533 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:12:25.533 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:12:25.533 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:12:17' 2024-09-12T13:12:25.533 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:12:25.534 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:12:25.534 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2024-09-12T13:12:25.534 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary3'\'']/last_update' 2024-09-12T13:12:25.536 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:12:25.537 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:12:25.537 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:12:25.538 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ promote_image cluster2 mirror image-primary10 2024-09-12T13:12:25.538 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:12:25.538 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:12:25.538 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:12:25.538 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local force= 2024-09-12T13:12:25.538 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 mirror image promote mirror/image-primary10 2024-09-12T13:12:25.542 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:12:17' 2024-09-12T13:12:25.546 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary3'\'']/state' 2024-09-12T13:12:25.553 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+unknown 2024-09-12T13:12:25.554 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:12:25.554 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+unknown 2024-09-12T13:12:25.557 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:12:25.559 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:12:25.559 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:12:25.559 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ promote_image cluster2 mirror image-primary3 2024-09-12T13:12:25.559 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:12:25.559 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:12:25.559 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:12:25.560 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local force= 2024-09-12T13:12:25.560 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 mirror image promote mirror/image-primary3 2024-09-12T13:12:25.726 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041924231mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary11df40b54-54f7-41fe-a2f4-c900ccf2e016up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:12:17image-primary102286adb1-ff3d-442a-9aac-6b38a8513aaaup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:12:17image-primary223ee0857-2880-4d74-a09e-17095d384c6bup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:12:17image-primary392498ee6-eac1-4596-be69-c222174caeb6up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:12:17image-primary411f002da-1dfb-4012-a01a-c5237417a099up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:12:17image-primary5261d375d-e3b2-4dc4-91bd-36dfede8b276up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:12:17image-primary6b711ef4c-c601-46a4-a872-1c8abdd1ca45up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:12:17image-primary773a79146-a78a-4543-843c-6cef99c271d3up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:12:17image-primary8fa7b2037-727d-4ca0-abbc-a51e250f5ecfup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:12:17image-primary977edc192-6758-44c1-b93f-e8f7df97992aup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:12:17' 2024-09-12T13:12:25.726 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:12:25.726 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary4'\'']/last_update' 2024-09-12T13:12:25.731 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:12:17' 2024-09-12T13:12:25.732 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary4'\'']/state' 2024-09-12T13:12:25.736 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+unknown 2024-09-12T13:12:25.736 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:12:25.737 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+unknown 2024-09-12T13:12:25.739 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:12:25.740 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:12:25.740 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:12:25.740 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ promote_image cluster2 mirror image-primary4 2024-09-12T13:12:25.740 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:12:25.740 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:12:25.740 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:12:25.740 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local force= 2024-09-12T13:12:25.740 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 mirror image promote mirror/image-primary4 2024-09-12T13:12:25.890 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster2 mirror image-primary8 up+unknown '' '' 2024-09-12T13:12:25.890 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:12:25.890 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:12:25.891 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:12:25.891 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:12:25.891 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:12:25.891 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:12:25.891 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:12:25.891 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:12:25.895 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster2 mirror image status mirror/image-primary8 2024-09-12T13:12:25.899 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster2 mirror image-primary9 up+unknown '' '' 2024-09-12T13:12:25.901 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:12:25.901 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:12:25.901 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:12:25.901 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:12:25.901 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:12:25.901 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:12:25.902 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:12:25.903 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:12:25.903 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster2 mirror image status mirror/image-primary9 2024-09-12T13:12:25.907 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Image promoted to primary 2024-09-12T13:12:25.916 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Image promoted to primary 2024-09-12T13:12:25.921 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Image promoted to primary 2024-09-12T13:12:25.923 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Image promoted to primary 2024-09-12T13:12:25.953 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \n\b\d ]] 2024-09-12T13:12:25.953 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \k\r\b\d ]] 2024-09-12T13:12:25.953 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \n\b\d ]] 2024-09-12T13:12:25.953 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \k\r\b\d ]] 2024-09-12T13:12:25.955 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo rbd --cluster cluster2 device map -t krbd mirror/image-primary10 2024-09-12T13:12:25.957 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo rbd --cluster cluster2 device map -t krbd mirror/image-primary7 2024-09-12T13:12:25.967 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \n\b\d ]] 2024-09-12T13:12:25.967 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \k\r\b\d ]] 2024-09-12T13:12:25.969 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo rbd --cluster cluster2 device map -t krbd mirror/image-primary4 2024-09-12T13:12:25.978 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \n\b\d ]] 2024-09-12T13:12:25.978 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \k\r\b\d ]] 2024-09-12T13:12:25.979 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo rbd --cluster cluster2 device map -t krbd mirror/image-primary3 2024-09-12T13:12:26.063 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary8: 2024-09-12T13:12:26.063 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: fa7b2037-727d-4ca0-abbc-a51e250f5ecf 2024-09-12T13:12:26.064 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+unknown 2024-09-12T13:12:26.064 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: remote image is not primary 2024-09-12T13:12:26.064 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:12:26.064 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:12:17' 2024-09-12T13:12:26.064 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:12:26.069 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+unknown 2024-09-12T13:12:26.073 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:12:26.080 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary9: 2024-09-12T13:12:26.080 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 77edc192-6758-44c1-b93f-e8f7df97992a 2024-09-12T13:12:26.080 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+unknown 2024-09-12T13:12:26.080 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: remote image is not primary 2024-09-12T13:12:26.080 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:12:26.080 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:12:17' 2024-09-12T13:12:26.081 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:12:26.082 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: remote image is not primary 2024-09-12T13:12:26.084 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:12:26.085 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+unknown 2024-09-12T13:12:26.086 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:12:26.086 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:12:26.087 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:12:26.092 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: remote image is not primary 2024-09-12T13:12:26.094 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:12:26.096 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:12:26.099 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:12:26.099 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:12:26.099 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:12:26.111 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:12:26.111 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:12:26.114 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:12:26.115 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:12:26.116 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:12:26.120 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:12:26.121 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:12:26.122 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:12:26.124 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:12:17' 2024-09-12T13:12:26.126 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster2 mirror image-primary8 up+unknown '2024-09-12 13:12:17' 2024-09-12T13:12:26.126 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:12:26.126 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:12:26.126 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:12:26.126 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:12:26.126 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:12:17' 2024-09-12T13:12:26.127 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:12:26.128 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:12:26.129 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2024-09-12T13:12:26.136 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:12:17' 2024-09-12T13:12:26.137 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster2 mirror image-primary9 up+unknown '2024-09-12 13:12:17' 2024-09-12T13:12:26.137 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:12:26.137 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:12:26.137 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:12:26.137 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:12:26.138 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:12:17' 2024-09-12T13:12:26.138 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:12:26.139 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:12:26.139 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2024-09-12T13:12:26.326 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ dev=/dev/rbd5 2024-09-12T13:12:26.329 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ awk '{print $1}' 2024-09-12T13:12:26.331 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo md5sum /dev/rbd5 2024-09-12T13:12:26.366 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ dev=/dev/rbd6 2024-09-12T13:12:26.368 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ dev=/dev/rbd7 2024-09-12T13:12:26.372 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo md5sum /dev/rbd6 2024-09-12T13:12:26.373 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ awk '{print $1}' 2024-09-12T13:12:26.382 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ awk '{print $1}' 2024-09-12T13:12:26.382 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo md5sum /dev/rbd7 2024-09-12T13:12:26.394 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ dev=/dev/rbd4 2024-09-12T13:12:26.399 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo md5sum /dev/rbd4 2024-09-12T13:12:26.401 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ awk '{print $1}' 2024-09-12T13:12:26.463 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041924231mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary11df40b54-54f7-41fe-a2f4-c900ccf2e016up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:12:17image-primary102286adb1-ff3d-442a-9aac-6b38a8513aaaup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:12:17image-primary223ee0857-2880-4d74-a09e-17095d384c6bup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:12:17image-primary392498ee6-eac1-4596-be69-c222174caeb6up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:12:17image-primary411f002da-1dfb-4012-a01a-c5237417a099up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:12:17image-primary5261d375d-e3b2-4dc4-91bd-36dfede8b276up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:12:17image-primary6b711ef4c-c601-46a4-a872-1c8abdd1ca45up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:12:17image-primary773a79146-a78a-4543-843c-6cef99c271d3up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:12:17image-primary8fa7b2037-727d-4ca0-abbc-a51e250f5ecfup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:12:17image-primary977edc192-6758-44c1-b93f-e8f7df97992aup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:12:17' 2024-09-12T13:12:26.463 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:12:26.467 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary8'\'']/last_update' 2024-09-12T13:12:26.478 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:12:17' 2024-09-12T13:12:26.483 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary8'\'']/state' 2024-09-12T13:12:26.499 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+unknown 2024-09-12T13:12:26.500 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:12:26.502 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+unknown 2024-09-12T13:12:26.509 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:12:26.513 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:12:26.513 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:12:26.513 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ promote_image cluster2 mirror image-primary8 2024-09-12T13:12:26.513 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:12:26.513 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:12:26.513 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:12:26.513 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local force= 2024-09-12T13:12:26.513 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 mirror image promote mirror/image-primary8 2024-09-12T13:12:26.519 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041924231mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary11df40b54-54f7-41fe-a2f4-c900ccf2e016up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:12:17image-primary102286adb1-ff3d-442a-9aac-6b38a8513aaaup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:12:17image-primary223ee0857-2880-4d74-a09e-17095d384c6bup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:12:17image-primary392498ee6-eac1-4596-be69-c222174caeb6up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:12:17image-primary411f002da-1dfb-4012-a01a-c5237417a099up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:12:17image-primary5261d375d-e3b2-4dc4-91bd-36dfede8b276up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:12:17image-primary6b711ef4c-c601-46a4-a872-1c8abdd1ca45up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:12:17image-primary773a79146-a78a-4543-843c-6cef99c271d3up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:12:17image-primary8fa7b2037-727d-4ca0-abbc-a51e250f5ecfup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:12:17image-primary977edc192-6758-44c1-b93f-e8f7df97992aup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:12:17' 2024-09-12T13:12:26.519 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:12:26.521 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary9'\'']/last_update' 2024-09-12T13:12:26.530 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:12:17' 2024-09-12T13:12:26.531 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary9'\'']/state' 2024-09-12T13:12:26.538 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+unknown 2024-09-12T13:12:26.540 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+unknown 2024-09-12T13:12:26.540 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:12:26.551 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:12:26.551 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:12:26.557 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:12:26.557 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ promote_image cluster2 mirror image-primary9 2024-09-12T13:12:26.557 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:12:26.557 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:12:26.557 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:12:26.557 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local force= 2024-09-12T13:12:26.557 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 mirror image promote mirror/image-primary9 2024-09-12T13:12:26.923 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Image promoted to primary 2024-09-12T13:12:26.946 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Image promoted to primary 2024-09-12T13:12:26.962 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \n\b\d ]] 2024-09-12T13:12:26.962 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \k\r\b\d ]] 2024-09-12T13:12:26.969 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo rbd --cluster cluster2 device map -t krbd mirror/image-primary8 2024-09-12T13:12:26.984 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \n\b\d ]] 2024-09-12T13:12:26.985 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \k\r\b\d ]] 2024-09-12T13:12:26.985 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo rbd --cluster cluster2 device map -t krbd mirror/image-primary9 2024-09-12T13:12:27.432 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ dev=/dev/rbd8 2024-09-12T13:12:27.445 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo md5sum /dev/rbd8 2024-09-12T13:12:27.448 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ awk '{print $1}' 2024-09-12T13:12:27.477 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ dev=/dev/rbd9 2024-09-12T13:12:27.480 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ awk '{print $1}' 2024-09-12T13:12:27.483 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo md5sum /dev/rbd9 2024-09-12T13:13:08.625 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ promote_md5=6081f378dfc29c81e4c3cd6d6202923e 2024-09-12T13:13:08.626 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo rbd --cluster cluster2 device unmap -t krbd /dev/rbd1 2024-09-12T13:13:09.273 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ 6081f378dfc29c81e4c3cd6d6202923e != \6\0\8\1\f\3\7\8\d\f\c\2\9\c\8\1\e\4\c\3\c\d\6\d\6\2\0\2\9\2\3\e ]] 2024-09-12T13:13:12.010 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ promote_md5=5f048362e14926082cdee95990e2db55 2024-09-12T13:13:12.010 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo rbd --cluster cluster2 device unmap -t krbd /dev/rbd2 2024-09-12T13:13:12.267 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ 5f048362e14926082cdee95990e2db55 != \5\f\0\4\8\3\6\2\e\1\4\9\2\6\0\8\2\c\d\e\e\9\5\9\9\0\e\2\d\b\5\5 ]] 2024-09-12T13:13:13.397 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ promote_md5=5cce44a8cb4936d1c93e9789805e6a22 2024-09-12T13:13:13.398 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo rbd --cluster cluster2 device unmap -t krbd /dev/rbd0 2024-09-12T13:13:13.584 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ 5cce44a8cb4936d1c93e9789805e6a22 != \5\c\c\e\4\4\a\8\c\b\4\9\3\6\d\1\c\9\3\e\9\7\8\9\8\0\5\e\6\a\2\2 ]] 2024-09-12T13:13:13.584 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${COMPARE_PIDS[@]} 2024-09-12T13:13:13.585 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 54486 2024-09-12T13:13:13.585 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${COMPARE_PIDS[@]} 2024-09-12T13:13:13.585 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 54488 2024-09-12T13:13:14.054 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ promote_md5=30f1f17928cdcb4c7df6846597b60582 2024-09-12T13:13:14.054 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo rbd --cluster cluster2 device unmap -t krbd /dev/rbd3 2024-09-12T13:13:14.233 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ 30f1f17928cdcb4c7df6846597b60582 != \3\0\f\1\f\1\7\9\2\8\c\d\c\b\4\c\7\d\f\6\8\4\6\5\9\7\b\6\0\5\8\2 ]] 2024-09-12T13:13:18.786 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ promote_md5=1b3a3de9819a5503a292e74cc2f0aecc 2024-09-12T13:13:18.787 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo rbd --cluster cluster2 device unmap -t krbd /dev/rbd8 2024-09-12T13:13:18.825 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ promote_md5=6ae05fc69a7e23396e5919d64416a23a 2024-09-12T13:13:18.825 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo rbd --cluster cluster2 device unmap -t krbd /dev/rbd5 2024-09-12T13:13:18.943 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ 1b3a3de9819a5503a292e74cc2f0aecc != \1\b\3\a\3\d\e\9\8\1\9\a\5\5\0\3\a\2\9\2\e\7\4\c\c\2\f\0\a\e\c\c ]] 2024-09-12T13:13:19.025 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ promote_md5=3b6e813ceb5ae83d16ef02b8346a6f8a 2024-09-12T13:13:19.026 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo rbd --cluster cluster2 device unmap -t krbd /dev/rbd6 2024-09-12T13:13:19.094 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ 6ae05fc69a7e23396e5919d64416a23a != \6\a\e\0\5\f\c\6\9\a\7\e\2\3\3\9\6\e\5\9\1\9\d\6\4\4\1\6\a\2\3\a ]] 2024-09-12T13:13:19.204 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ 3b6e813ceb5ae83d16ef02b8346a6f8a != \3\b\6\e\8\1\3\c\e\b\5\a\e\8\3\d\1\6\e\f\0\2\b\8\3\4\6\a\6\f\8\a ]] 2024-09-12T13:13:19.245 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ promote_md5=fb01e2c493269b8b67426929cdfb8b12 2024-09-12T13:13:19.245 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo rbd --cluster cluster2 device unmap -t krbd /dev/rbd9 2024-09-12T13:13:19.359 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ fb01e2c493269b8b67426929cdfb8b12 != \f\b\0\1\e\2\c\4\9\3\2\6\9\b\8\b\6\7\4\2\6\9\2\9\c\d\f\b\8\b\1\2 ]] 2024-09-12T13:13:19.638 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ promote_md5=1bd72432f627efe6b6d65d64cf0cd64d 2024-09-12T13:13:19.638 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo rbd --cluster cluster2 device unmap -t krbd /dev/rbd4 2024-09-12T13:13:19.735 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ 1bd72432f627efe6b6d65d64cf0cd64d != \1\b\d\7\2\4\3\2\f\6\2\7\e\f\e\6\b\6\d\6\5\d\6\4\c\f\0\c\d\6\4\d ]] 2024-09-12T13:13:20.207 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ promote_md5=146b91e683fb47cb6ebca28406a46347 2024-09-12T13:13:20.207 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo rbd --cluster cluster2 device unmap -t krbd /dev/rbd7 2024-09-12T13:13:20.290 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ 146b91e683fb47cb6ebca28406a46347 != \1\4\6\b\9\1\e\6\8\3\f\b\4\7\c\b\6\e\b\c\a\2\8\4\0\6\a\4\6\3\4\7 ]] 2024-09-12T13:13:20.291 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${COMPARE_PIDS[@]} 2024-09-12T13:13:20.292 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 54490 2024-09-12T13:13:20.292 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${COMPARE_PIDS[@]} 2024-09-12T13:13:20.292 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 54492 2024-09-12T13:13:20.292 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${COMPARE_PIDS[@]} 2024-09-12T13:13:20.292 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 54494 2024-09-12T13:13:20.292 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${COMPARE_PIDS[@]} 2024-09-12T13:13:20.292 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 54496 2024-09-12T13:13:20.292 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${COMPARE_PIDS[@]} 2024-09-12T13:13:20.292 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 54498 2024-09-12T13:13:20.292 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${COMPARE_PIDS[@]} 2024-09-12T13:13:20.292 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 54501 2024-09-12T13:13:20.292 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${COMPARE_PIDS[@]} 2024-09-12T13:13:20.292 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 54502 2024-09-12T13:13:20.293 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ (( RET != 0 )) 2024-09-12T13:13:20.293 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:13:20.293 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ IMG=image-primary1 2024-09-12T13:13:20.293 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_replaying_status_in_pool_dir cluster1 mirror image-primary1 2024-09-12T13:13:20.293 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:13:20.293 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:13:20.293 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:13:20.293 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' snapshot = journal ']' 2024-09-12T13:13:20.293 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary1 up+replaying 2024-09-12T13:13:20.293 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:13:20.293 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:13:20.293 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:13:20.293 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:13:20.294 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:13:20.294 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:13:20.294 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:13:20.294 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:13:21.304 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary1 up+replaying '' '' 2024-09-12T13:13:21.304 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:13:21.305 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:13:21.305 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:13:21.305 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:13:21.305 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:13:21.305 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:13:21.305 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:13:21.305 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:13:21.305 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary1 2024-09-12T13:13:21.390 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary1: 2024-09-12T13:13:21.390 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 1df40b54-54f7-41fe-a2f4-c900ccf2e016 2024-09-12T13:13:21.390 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+replaying 2024-09-12T13:13:21.391 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: replaying, {"bytes_per_second":3174195.2,"bytes_per_snapshot":63483904.0,"last_snapshot_bytes":63483904,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726146740,"remote_snapshot_timestamp":1726146740,"replay_state":"idle"} 2024-09-12T13:13:21.391 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:13:21.391 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:13:17' 2024-09-12T13:13:21.391 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+replaying' 2024-09-12T13:13:21.392 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+replaying 2024-09-12T13:13:21.393 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:13:21.395 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: replaying, {"bytes_per_second":3174195.2,"bytes_per_snapshot":63483904.0,"last_snapshot_bytes":63483904,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726146740,"remote_snapshot_timestamp":1726146740,"replay_state":"idle"} 2024-09-12T13:13:21.396 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:13:21.396 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+replaying 2024-09-12T13:13:21.397 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:13:21.399 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+replaying 2024-09-12T13:13:21.399 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:13:21.402 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:13:21.402 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:13:21.403 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:13:21.407 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:13:17' 2024-09-12T13:13:21.407 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary1 up+replaying '2024-09-12 13:13:17' 2024-09-12T13:13:21.407 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:13:21.407 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:13:21.407 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:13:21.407 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:13:21.407 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:13:17' 2024-09-12T13:13:21.407 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:13:21.408 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:13:21.408 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-09-12T13:13:21.514 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041934232mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary11df40b54-54f7-41fe-a2f4-c900ccf2e016up+replayingreplaying, {"bytes_per_second":3174195.2,"bytes_per_snapshot":63483904.0,"last_snapshot_bytes":63483904,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726146740,"remote_snapshot_timestamp":1726146740,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:13:17image-primary102286adb1-ff3d-442a-9aac-6b38a8513aaaup+replayingreplaying, {"bytes_per_second":3120332.8,"bytes_per_snapshot":62406656.0,"last_snapshot_bytes":62406656,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726146745,"remote_snapshot_timestamp":1726146745,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:13:17image-primary223ee0857-2880-4d74-a09e-17095d384c6bup+replayingreplaying, {"bytes_per_second":3253452.8,"bytes_per_snapshot":65069056.0,"last_snapshot_bytes":65069056,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726146741,"remote_snapshot_timestamp":1726146741,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:13:17image-primary392498ee6-eac1-4596-be69-c222174caeb6up+replayingreplaying, {"bytes_per_second":3255705.6,"bytes_per_snapshot":65114112.0,"last_snapshot_bytes":65114112,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726146745,"remote_snapshot_timestamp":1726146745,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:13:17image-primary411f002da-1dfb-4012-a01a-c5237417a099up+replayingreplaying, {"bytes_per_second":3653017.6,"bytes_per_snapshot":73060352.0,"last_snapshot_bytes":73060352,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726146745,"remote_snapshot_timestamp":1726146745,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:13:17image-primary5261d375d-e3b2-4dc4-91bd-36dfede8b276up+replayingreplaying, {"bytes_per_second":3169280.0,"bytes_per_snapshot":63385600.0,"last_snapshot_bytes":63385600,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726146742,"remote_snapshot_timestamp":1726146742,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:13:17image-primary6b711ef4c-c601-46a4-a872-1c8abdd1ca45up+replayingreplaying, {"bytes_per_second":3268608.0,"bytes_per_snapshot":65372160.0,"last_snapshot_bytes":65372160,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726146740,"remote_snapshot_timestamp":1726146740,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:13:17image-primary773a79146-a78a-4543-843c-6cef99c271d3up+replayingreplaying, {"bytes_per_second":3253452.8,"bytes_per_snapshot":65069056.0,"last_snapshot_bytes":65069056,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726146745,"remote_snapshot_timestamp":1726146745,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:13:17image-primary8fa7b2037-727d-4ca0-abbc-a51e250f5ecfup+replayingreplaying, {"bytes_per_second":3252428.8,"bytes_per_snapshot":65048576.0,"last_snapshot_bytes":65048576,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726146746,"remote_snapshot_timestamp":1726146746,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:13:17image-primary977edc192-6758-44c1-b93f-e8f7df97992aup+replayingreplaying, {"bytes_per_second":3260211.2,"bytes_per_snapshot":65204224.0,"last_snapshot_bytes":65204224,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726146746,"remote_snapshot_timestamp":1726146746,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:13:17' 2024-09-12T13:13:21.514 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:13:21.514 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary1'\'']/last_update' 2024-09-12T13:13:21.546 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:13:17' 2024-09-12T13:13:21.546 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary1'\'']/state' 2024-09-12T13:13:21.550 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+replaying 2024-09-12T13:13:21.551 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+replaying 2024-09-12T13:13:21.551 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+replaying 2024-09-12T13:13:21.553 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+replaying 2024-09-12T13:13:21.554 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:13:21.554 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:13:21.554 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ remove_image cluster2 mirror image-primary1 2024-09-12T13:13:21.554 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:13:21.554 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:13:21.554 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:13:21.554 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 snap purge mirror/image-primary1 2024-09-12T13:13:21.614 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 rm mirror/image-primary1 2024-09-12T13:13:21.756 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:13:21.755+0000 7f67e5200640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x560555824ea0 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-09-12T13:13:21.759 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:13:21.755+0000 7f67ed400640 -1 rbd::mirror::ImageReplayer: 0x5605561ac000 [3/1df40b54-54f7-41fe-a2f4-c900ccf2e016] operator(): start failed: (85) Interrupted system call should be restarted 2024-09-12T13:13:23.099 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-09-12T13:13:23.105 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_image_removal cluster1 mirror image-primary1 2024-09-12T13:13:23.105 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:13:23.105 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:13:23.105 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:13:23.105 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:13:23.106 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 ls mirror 2024-09-12T13:13:23.106 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep -wq image-primary1 2024-09-12T13:13:23.155 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:13:23.156 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rm -rf test-primary1 2024-09-12T13:13:23.169 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:13:23.169 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ IMG=image-primary2 2024-09-12T13:13:23.169 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_replaying_status_in_pool_dir cluster1 mirror image-primary2 2024-09-12T13:13:23.169 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:13:23.169 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:13:23.169 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:13:23.169 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' snapshot = journal ']' 2024-09-12T13:13:23.169 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary2 up+replaying 2024-09-12T13:13:23.170 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:13:23.170 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:13:23.170 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:13:23.170 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:13:23.170 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:13:23.170 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:13:23.170 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:13:23.170 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:13:24.171 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary2 up+replaying '' '' 2024-09-12T13:13:24.171 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:13:24.171 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:13:24.171 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:13:24.171 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:13:24.171 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:13:24.172 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:13:24.172 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:13:24.172 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:13:24.172 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary2 2024-09-12T13:13:24.263 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary2: 2024-09-12T13:13:24.263 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 23ee0857-2880-4d74-a09e-17095d384c6b 2024-09-12T13:13:24.263 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+replaying 2024-09-12T13:13:24.263 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: replaying, {"bytes_per_second":3253452.8,"bytes_per_snapshot":65069056.0,"last_snapshot_bytes":65069056,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726146741,"remote_snapshot_timestamp":1726146741,"replay_state":"idle"} 2024-09-12T13:13:24.263 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:13:24.263 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:13:17' 2024-09-12T13:13:24.263 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+replaying' 2024-09-12T13:13:24.265 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+replaying 2024-09-12T13:13:24.265 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:13:24.268 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: replaying, {"bytes_per_second":3253452.8,"bytes_per_snapshot":65069056.0,"last_snapshot_bytes":65069056,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726146741,"remote_snapshot_timestamp":1726146741,"replay_state":"idle"} 2024-09-12T13:13:24.268 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:13:24.269 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+replaying 2024-09-12T13:13:24.269 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:13:24.271 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+replaying 2024-09-12T13:13:24.271 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:13:24.274 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:13:24.274 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:13:24.275 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:13:24.279 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:13:17' 2024-09-12T13:13:24.279 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary2 up+replaying '2024-09-12 13:13:17' 2024-09-12T13:13:24.279 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:13:24.279 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:13:24.279 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:13:24.279 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:13:24.280 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:13:17' 2024-09-12T13:13:24.280 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:13:24.280 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:13:24.280 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-09-12T13:13:24.382 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING941934232mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary102286adb1-ff3d-442a-9aac-6b38a8513aaaup+replayingreplaying, {"bytes_per_second":3120332.8,"bytes_per_snapshot":62406656.0,"last_snapshot_bytes":62406656,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726146745,"remote_snapshot_timestamp":1726146745,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:13:17image-primary223ee0857-2880-4d74-a09e-17095d384c6bup+replayingreplaying, {"bytes_per_second":3253452.8,"bytes_per_snapshot":65069056.0,"last_snapshot_bytes":65069056,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726146741,"remote_snapshot_timestamp":1726146741,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:13:17image-primary392498ee6-eac1-4596-be69-c222174caeb6up+replayingreplaying, {"bytes_per_second":3255705.6,"bytes_per_snapshot":65114112.0,"last_snapshot_bytes":65114112,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726146745,"remote_snapshot_timestamp":1726146745,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:13:17image-primary411f002da-1dfb-4012-a01a-c5237417a099up+replayingreplaying, {"bytes_per_second":3653017.6,"bytes_per_snapshot":73060352.0,"last_snapshot_bytes":73060352,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726146745,"remote_snapshot_timestamp":1726146745,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:13:17image-primary5261d375d-e3b2-4dc4-91bd-36dfede8b276up+replayingreplaying, {"bytes_per_second":3169280.0,"bytes_per_snapshot":63385600.0,"last_snapshot_bytes":63385600,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726146742,"remote_snapshot_timestamp":1726146742,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:13:17image-primary6b711ef4c-c601-46a4-a872-1c8abdd1ca45up+replayingreplaying, {"bytes_per_second":3268608.0,"bytes_per_snapshot":65372160.0,"last_snapshot_bytes":65372160,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726146740,"remote_snapshot_timestamp":1726146740,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:13:17image-primary773a79146-a78a-4543-843c-6cef99c271d3up+replayingreplaying, {"bytes_per_second":3253452.8,"bytes_per_snapshot":65069056.0,"last_snapshot_bytes":65069056,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726146745,"remote_snapshot_timestamp":1726146745,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:13:17image-primary8fa7b2037-727d-4ca0-abbc-a51e250f5ecfup+replayingreplaying, {"bytes_per_second":3252428.8,"bytes_per_snapshot":65048576.0,"last_snapshot_bytes":65048576,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726146746,"remote_snapshot_timestamp":1726146746,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:13:17image-primary977edc192-6758-44c1-b93f-e8f7df97992aup+replayingreplaying, {"bytes_per_second":3260211.2,"bytes_per_snapshot":65204224.0,"last_snapshot_bytes":65204224,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726146746,"remote_snapshot_timestamp":1726146746,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:13:17' 2024-09-12T13:13:24.383 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:13:24.383 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary2'\'']/last_update' 2024-09-12T13:13:24.387 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:13:17' 2024-09-12T13:13:24.388 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary2'\'']/state' 2024-09-12T13:13:24.391 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+replaying 2024-09-12T13:13:24.392 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+replaying 2024-09-12T13:13:24.392 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+replaying 2024-09-12T13:13:24.394 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+replaying 2024-09-12T13:13:24.395 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:13:24.395 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:13:24.395 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ remove_image cluster2 mirror image-primary2 2024-09-12T13:13:24.395 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:13:24.395 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:13:24.396 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:13:24.396 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 snap purge mirror/image-primary2 2024-09-12T13:13:24.455 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 rm mirror/image-primary2 2024-09-12T13:13:24.588 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:13:24.587+0000 7f67e5200640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x5605557781a0 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-09-12T13:13:24.588 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:13:24.587+0000 7f67eca00640 -1 rbd::mirror::ImageReplayer: 0x56055c199180 [3/23ee0857-2880-4d74-a09e-17095d384c6b] operator(): start failed: (85) Interrupted system call should be restarted 2024-09-12T13:13:26.099 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-09-12T13:13:26.105 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_image_removal cluster1 mirror image-primary2 2024-09-12T13:13:26.105 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:13:26.105 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:13:26.105 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:13:26.106 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:13:26.106 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 ls mirror 2024-09-12T13:13:26.106 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep -wq image-primary2 2024-09-12T13:13:26.155 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:13:26.155 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rm -rf test-primary2 2024-09-12T13:13:26.158 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:13:26.158 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ IMG=image-primary3 2024-09-12T13:13:26.158 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_replaying_status_in_pool_dir cluster1 mirror image-primary3 2024-09-12T13:13:26.158 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:13:26.158 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:13:26.159 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:13:26.159 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' snapshot = journal ']' 2024-09-12T13:13:26.159 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary3 up+replaying 2024-09-12T13:13:26.159 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:13:26.159 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:13:26.159 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:13:26.159 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:13:26.159 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:13:26.159 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:13:26.159 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:13:26.159 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:13:27.160 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary3 up+replaying '' '' 2024-09-12T13:13:27.160 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:13:27.161 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:13:27.161 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:13:27.161 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:13:27.161 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:13:27.161 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:13:27.161 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:13:27.161 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:13:27.162 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary3 2024-09-12T13:13:27.244 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary3: 2024-09-12T13:13:27.244 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 92498ee6-eac1-4596-be69-c222174caeb6 2024-09-12T13:13:27.244 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+replaying 2024-09-12T13:13:27.244 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: replaying, {"bytes_per_second":3255705.6,"bytes_per_snapshot":65114112.0,"last_snapshot_bytes":65114112,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726146745,"remote_snapshot_timestamp":1726146745,"replay_state":"idle"} 2024-09-12T13:13:27.244 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:13:27.244 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:13:17' 2024-09-12T13:13:27.244 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+replaying' 2024-09-12T13:13:27.246 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+replaying 2024-09-12T13:13:27.246 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:13:27.249 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: replaying, {"bytes_per_second":3255705.6,"bytes_per_snapshot":65114112.0,"last_snapshot_bytes":65114112,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726146745,"remote_snapshot_timestamp":1726146745,"replay_state":"idle"} 2024-09-12T13:13:27.249 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:13:27.250 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+replaying 2024-09-12T13:13:27.250 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:13:27.252 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+replaying 2024-09-12T13:13:27.253 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:13:27.255 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:13:27.255 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:13:27.256 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:13:27.260 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:13:17' 2024-09-12T13:13:27.260 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary3 up+replaying '2024-09-12 13:13:17' 2024-09-12T13:13:27.260 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:13:27.260 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:13:27.260 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:13:27.260 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:13:27.260 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:13:17' 2024-09-12T13:13:27.261 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:13:27.261 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:13:27.261 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-09-12T13:13:27.367 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING841934232mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary102286adb1-ff3d-442a-9aac-6b38a8513aaaup+replayingreplaying, {"bytes_per_second":3120332.8,"bytes_per_snapshot":62406656.0,"last_snapshot_bytes":62406656,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726146745,"remote_snapshot_timestamp":1726146745,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:13:17image-primary392498ee6-eac1-4596-be69-c222174caeb6up+replayingreplaying, {"bytes_per_second":3255705.6,"bytes_per_snapshot":65114112.0,"last_snapshot_bytes":65114112,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726146745,"remote_snapshot_timestamp":1726146745,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:13:17image-primary411f002da-1dfb-4012-a01a-c5237417a099up+replayingreplaying, {"bytes_per_second":3653017.6,"bytes_per_snapshot":73060352.0,"last_snapshot_bytes":73060352,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726146745,"remote_snapshot_timestamp":1726146745,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:13:17image-primary5261d375d-e3b2-4dc4-91bd-36dfede8b276up+replayingreplaying, {"bytes_per_second":3169280.0,"bytes_per_snapshot":63385600.0,"last_snapshot_bytes":63385600,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726146742,"remote_snapshot_timestamp":1726146742,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:13:17image-primary6b711ef4c-c601-46a4-a872-1c8abdd1ca45up+replayingreplaying, {"bytes_per_second":3268608.0,"bytes_per_snapshot":65372160.0,"last_snapshot_bytes":65372160,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726146740,"remote_snapshot_timestamp":1726146740,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:13:17image-primary773a79146-a78a-4543-843c-6cef99c271d3up+replayingreplaying, {"bytes_per_second":3253452.8,"bytes_per_snapshot":65069056.0,"last_snapshot_bytes":65069056,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726146745,"remote_snapshot_timestamp":1726146745,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:13:17image-primary8fa7b2037-727d-4ca0-abbc-a51e250f5ecfup+replayingreplaying, {"bytes_per_second":3252428.8,"bytes_per_snapshot":65048576.0,"last_snapshot_bytes":65048576,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726146746,"remote_snapshot_timestamp":1726146746,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:13:17image-primary977edc192-6758-44c1-b93f-e8f7df97992aup+replayingreplaying, {"bytes_per_second":3260211.2,"bytes_per_snapshot":65204224.0,"last_snapshot_bytes":65204224,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726146746,"remote_snapshot_timestamp":1726146746,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:13:17' 2024-09-12T13:13:27.367 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:13:27.367 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary3'\'']/last_update' 2024-09-12T13:13:27.371 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:13:17' 2024-09-12T13:13:27.372 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary3'\'']/state' 2024-09-12T13:13:27.376 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+replaying 2024-09-12T13:13:27.376 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+replaying 2024-09-12T13:13:27.377 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+replaying 2024-09-12T13:13:27.379 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+replaying 2024-09-12T13:13:27.379 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:13:27.380 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:13:27.380 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ remove_image cluster2 mirror image-primary3 2024-09-12T13:13:27.380 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:13:27.380 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:13:27.380 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:13:27.380 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 snap purge mirror/image-primary3 2024-09-12T13:13:27.439 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 rm mirror/image-primary3 2024-09-12T13:13:27.563 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:13:27.563+0000 7f67e5200640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x56055b9724e0 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-09-12T13:13:27.563 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:13:27.563+0000 7f67ed400640 -1 rbd::mirror::ImageReplayer: 0x56055c199b80 [3/92498ee6-eac1-4596-be69-c222174caeb6] operator(): start failed: (85) Interrupted system call should be restarted 2024-09-12T13:13:29.132 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-09-12T13:13:29.138 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_image_removal cluster1 mirror image-primary3 2024-09-12T13:13:29.138 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:13:29.138 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:13:29.138 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:13:29.138 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:13:29.139 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 ls mirror 2024-09-12T13:13:29.139 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep -wq image-primary3 2024-09-12T13:13:29.189 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:13:29.189 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rm -rf test-primary3 2024-09-12T13:13:29.192 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:13:29.192 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ IMG=image-primary4 2024-09-12T13:13:29.192 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_replaying_status_in_pool_dir cluster1 mirror image-primary4 2024-09-12T13:13:29.192 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:13:29.192 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:13:29.193 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:13:29.193 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' snapshot = journal ']' 2024-09-12T13:13:29.193 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary4 up+replaying 2024-09-12T13:13:29.193 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:13:29.193 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:13:29.193 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:13:29.193 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:13:29.193 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:13:29.193 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:13:29.193 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:13:29.193 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:13:30.194 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary4 up+replaying '' '' 2024-09-12T13:13:30.194 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:13:30.194 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:13:30.194 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:13:30.194 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:13:30.194 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:13:30.194 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:13:30.194 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:13:30.195 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:13:30.195 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary4 2024-09-12T13:13:30.277 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary4: 2024-09-12T13:13:30.277 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 11f002da-1dfb-4012-a01a-c5237417a099 2024-09-12T13:13:30.277 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+replaying 2024-09-12T13:13:30.277 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: replaying, {"bytes_per_second":3653017.6,"bytes_per_snapshot":73060352.0,"last_snapshot_bytes":73060352,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726146745,"remote_snapshot_timestamp":1726146745,"replay_state":"idle"} 2024-09-12T13:13:30.277 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:13:30.277 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:13:17' 2024-09-12T13:13:30.277 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+replaying' 2024-09-12T13:13:30.279 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+replaying 2024-09-12T13:13:30.280 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:13:30.282 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: replaying, {"bytes_per_second":3653017.6,"bytes_per_snapshot":73060352.0,"last_snapshot_bytes":73060352,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726146745,"remote_snapshot_timestamp":1726146745,"replay_state":"idle"} 2024-09-12T13:13:30.282 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:13:30.283 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+replaying 2024-09-12T13:13:30.283 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:13:30.285 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+replaying 2024-09-12T13:13:30.285 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:13:30.288 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:13:30.288 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:13:30.289 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:13:30.293 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:13:17' 2024-09-12T13:13:30.293 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary4 up+replaying '2024-09-12 13:13:17' 2024-09-12T13:13:30.293 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:13:30.293 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:13:30.293 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:13:30.293 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:13:30.293 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:13:17' 2024-09-12T13:13:30.293 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:13:30.294 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:13:30.294 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-09-12T13:13:30.398 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING741934232mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary102286adb1-ff3d-442a-9aac-6b38a8513aaaup+replayingreplaying, {"bytes_per_second":3120332.8,"bytes_per_snapshot":62406656.0,"last_snapshot_bytes":62406656,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726146745,"remote_snapshot_timestamp":1726146745,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:13:17image-primary411f002da-1dfb-4012-a01a-c5237417a099up+replayingreplaying, {"bytes_per_second":3653017.6,"bytes_per_snapshot":73060352.0,"last_snapshot_bytes":73060352,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726146745,"remote_snapshot_timestamp":1726146745,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:13:17image-primary5261d375d-e3b2-4dc4-91bd-36dfede8b276up+replayingreplaying, {"bytes_per_second":3169280.0,"bytes_per_snapshot":63385600.0,"last_snapshot_bytes":63385600,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726146742,"remote_snapshot_timestamp":1726146742,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:13:17image-primary6b711ef4c-c601-46a4-a872-1c8abdd1ca45up+replayingreplaying, {"bytes_per_second":3268608.0,"bytes_per_snapshot":65372160.0,"last_snapshot_bytes":65372160,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726146740,"remote_snapshot_timestamp":1726146740,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:13:17image-primary773a79146-a78a-4543-843c-6cef99c271d3up+replayingreplaying, {"bytes_per_second":3253452.8,"bytes_per_snapshot":65069056.0,"last_snapshot_bytes":65069056,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726146745,"remote_snapshot_timestamp":1726146745,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:13:17image-primary8fa7b2037-727d-4ca0-abbc-a51e250f5ecfup+replayingreplaying, {"bytes_per_second":3252428.8,"bytes_per_snapshot":65048576.0,"last_snapshot_bytes":65048576,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726146746,"remote_snapshot_timestamp":1726146746,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:13:17image-primary977edc192-6758-44c1-b93f-e8f7df97992aup+replayingreplaying, {"bytes_per_second":3260211.2,"bytes_per_snapshot":65204224.0,"last_snapshot_bytes":65204224,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726146746,"remote_snapshot_timestamp":1726146746,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:13:17' 2024-09-12T13:13:30.398 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:13:30.399 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary4'\'']/last_update' 2024-09-12T13:13:30.402 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:13:17' 2024-09-12T13:13:30.403 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary4'\'']/state' 2024-09-12T13:13:30.407 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+replaying 2024-09-12T13:13:30.408 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+replaying 2024-09-12T13:13:30.408 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+replaying 2024-09-12T13:13:30.410 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+replaying 2024-09-12T13:13:30.411 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:13:30.411 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:13:30.411 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ remove_image cluster2 mirror image-primary4 2024-09-12T13:13:30.411 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:13:30.411 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:13:30.411 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:13:30.411 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 snap purge mirror/image-primary4 2024-09-12T13:13:30.470 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 rm mirror/image-primary4 2024-09-12T13:13:30.599 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:13:30.598+0000 7f67e4800640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x56055bf601a0 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-09-12T13:13:30.599 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:13:30.598+0000 7f67ed400640 -1 rbd::mirror::ImageReplayer: 0x56055c199900 [3/11f002da-1dfb-4012-a01a-c5237417a099] operator(): start failed: (85) Interrupted system call should be restarted 2024-09-12T13:13:32.134 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-09-12T13:13:32.139 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_image_removal cluster1 mirror image-primary4 2024-09-12T13:13:32.139 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:13:32.139 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:13:32.139 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:13:32.139 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:13:32.140 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 ls mirror 2024-09-12T13:13:32.140 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep -wq image-primary4 2024-09-12T13:13:32.191 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:13:32.191 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rm -rf test-primary4 2024-09-12T13:13:32.194 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:13:32.194 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ IMG=image-primary5 2024-09-12T13:13:32.194 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_replaying_status_in_pool_dir cluster1 mirror image-primary5 2024-09-12T13:13:32.194 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:13:32.194 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:13:32.195 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:13:32.195 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' snapshot = journal ']' 2024-09-12T13:13:32.195 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary5 up+replaying 2024-09-12T13:13:32.195 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:13:32.195 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:13:32.195 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:13:32.196 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:13:32.196 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:13:32.196 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:13:32.196 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:13:32.196 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:13:33.195 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary5 up+replaying '' '' 2024-09-12T13:13:33.195 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:13:33.195 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:13:33.195 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:13:33.195 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:13:33.196 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:13:33.196 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:13:33.196 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:13:33.196 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:13:33.196 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary5 2024-09-12T13:13:33.278 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary5: 2024-09-12T13:13:33.278 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 261d375d-e3b2-4dc4-91bd-36dfede8b276 2024-09-12T13:13:33.278 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+replaying 2024-09-12T13:13:33.278 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: replaying, {"bytes_per_second":3169280.0,"bytes_per_snapshot":63385600.0,"last_snapshot_bytes":63385600,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726146742,"remote_snapshot_timestamp":1726146742,"replay_state":"idle"} 2024-09-12T13:13:33.278 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:13:33.279 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:13:17' 2024-09-12T13:13:33.279 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+replaying' 2024-09-12T13:13:33.280 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+replaying 2024-09-12T13:13:33.281 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:13:33.283 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: replaying, {"bytes_per_second":3169280.0,"bytes_per_snapshot":63385600.0,"last_snapshot_bytes":63385600,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726146742,"remote_snapshot_timestamp":1726146742,"replay_state":"idle"} 2024-09-12T13:13:33.283 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:13:33.285 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+replaying 2024-09-12T13:13:33.285 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:13:33.286 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+replaying 2024-09-12T13:13:33.287 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:13:33.289 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:13:33.290 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:13:33.290 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:13:33.295 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:13:17' 2024-09-12T13:13:33.295 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary5 up+replaying '2024-09-12 13:13:17' 2024-09-12T13:13:33.295 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:13:33.295 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:13:33.296 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:13:33.296 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:13:33.296 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:13:17' 2024-09-12T13:13:33.296 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:13:33.296 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:13:33.297 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-09-12T13:13:33.395 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING641934232mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary102286adb1-ff3d-442a-9aac-6b38a8513aaaup+replayingreplaying, {"bytes_per_second":3120332.8,"bytes_per_snapshot":62406656.0,"last_snapshot_bytes":62406656,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726146745,"remote_snapshot_timestamp":1726146745,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:13:17image-primary5261d375d-e3b2-4dc4-91bd-36dfede8b276up+replayingreplaying, {"bytes_per_second":3169280.0,"bytes_per_snapshot":63385600.0,"last_snapshot_bytes":63385600,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726146742,"remote_snapshot_timestamp":1726146742,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:13:17image-primary6b711ef4c-c601-46a4-a872-1c8abdd1ca45up+replayingreplaying, {"bytes_per_second":3268608.0,"bytes_per_snapshot":65372160.0,"last_snapshot_bytes":65372160,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726146740,"remote_snapshot_timestamp":1726146740,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:13:17image-primary773a79146-a78a-4543-843c-6cef99c271d3up+replayingreplaying, {"bytes_per_second":3253452.8,"bytes_per_snapshot":65069056.0,"last_snapshot_bytes":65069056,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726146745,"remote_snapshot_timestamp":1726146745,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:13:17image-primary8fa7b2037-727d-4ca0-abbc-a51e250f5ecfup+replayingreplaying, {"bytes_per_second":3252428.8,"bytes_per_snapshot":65048576.0,"last_snapshot_bytes":65048576,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726146746,"remote_snapshot_timestamp":1726146746,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:13:17image-primary977edc192-6758-44c1-b93f-e8f7df97992aup+replayingreplaying, {"bytes_per_second":3260211.2,"bytes_per_snapshot":65204224.0,"last_snapshot_bytes":65204224,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726146746,"remote_snapshot_timestamp":1726146746,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:13:17' 2024-09-12T13:13:33.395 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:13:33.395 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary5'\'']/last_update' 2024-09-12T13:13:33.399 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:13:17' 2024-09-12T13:13:33.399 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary5'\'']/state' 2024-09-12T13:13:33.403 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+replaying 2024-09-12T13:13:33.404 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+replaying 2024-09-12T13:13:33.404 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+replaying 2024-09-12T13:13:33.406 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+replaying 2024-09-12T13:13:33.407 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:13:33.407 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:13:33.408 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ remove_image cluster2 mirror image-primary5 2024-09-12T13:13:33.408 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:13:33.409 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:13:33.409 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:13:33.409 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 snap purge mirror/image-primary5 2024-09-12T13:13:33.467 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 rm mirror/image-primary5 2024-09-12T13:13:33.609 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:13:33.609+0000 7f67e4800640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x56055b931380 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-09-12T13:13:33.609 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:13:33.609+0000 7f67ed400640 -1 rbd::mirror::ImageReplayer: 0x5605561ac500 [3/261d375d-e3b2-4dc4-91bd-36dfede8b276] operator(): start failed: (85) Interrupted system call should be restarted 2024-09-12T13:13:34.887 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-09-12T13:13:34.894 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_image_removal cluster1 mirror image-primary5 2024-09-12T13:13:34.894 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:13:34.894 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:13:34.894 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:13:34.894 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:13:34.895 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 ls mirror 2024-09-12T13:13:34.895 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep -wq image-primary5 2024-09-12T13:13:34.946 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:13:34.946 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rm -rf test-primary5 2024-09-12T13:13:34.949 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:13:34.949 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ IMG=image-primary6 2024-09-12T13:13:34.949 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_replaying_status_in_pool_dir cluster1 mirror image-primary6 2024-09-12T13:13:34.949 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:13:34.949 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:13:34.949 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:13:34.949 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' snapshot = journal ']' 2024-09-12T13:13:34.949 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary6 up+replaying 2024-09-12T13:13:34.949 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:13:34.949 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:13:34.950 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:13:34.950 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:13:34.950 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:13:34.950 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:13:34.950 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:13:34.950 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:13:35.951 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary6 up+replaying '' '' 2024-09-12T13:13:35.951 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:13:35.951 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:13:35.951 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:13:35.951 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:13:35.951 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:13:35.951 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:13:35.951 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:13:35.952 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:13:35.952 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary6 2024-09-12T13:13:36.065 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary6: 2024-09-12T13:13:36.065 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: b711ef4c-c601-46a4-a872-1c8abdd1ca45 2024-09-12T13:13:36.066 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+replaying 2024-09-12T13:13:36.066 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: replaying, {"bytes_per_second":3268608.0,"bytes_per_snapshot":65372160.0,"last_snapshot_bytes":65372160,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726146740,"remote_snapshot_timestamp":1726146740,"replay_state":"idle"} 2024-09-12T13:13:36.066 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:13:36.066 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:13:17' 2024-09-12T13:13:36.066 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+replaying' 2024-09-12T13:13:36.068 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+replaying 2024-09-12T13:13:36.068 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:13:36.071 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: replaying, {"bytes_per_second":3268608.0,"bytes_per_snapshot":65372160.0,"last_snapshot_bytes":65372160,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726146740,"remote_snapshot_timestamp":1726146740,"replay_state":"idle"} 2024-09-12T13:13:36.072 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:13:36.072 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+replaying 2024-09-12T13:13:36.073 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:13:36.075 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+replaying 2024-09-12T13:13:36.075 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:13:36.078 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:13:36.078 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:13:36.079 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:13:36.083 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:13:17' 2024-09-12T13:13:36.083 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary6 up+replaying '2024-09-12 13:13:17' 2024-09-12T13:13:36.083 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:13:36.083 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:13:36.083 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:13:36.083 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:13:36.083 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:13:17' 2024-09-12T13:13:36.083 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:13:36.084 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:13:36.084 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-09-12T13:13:36.213 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING541934232mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary102286adb1-ff3d-442a-9aac-6b38a8513aaaup+replayingreplaying, {"bytes_per_second":3120332.8,"bytes_per_snapshot":62406656.0,"last_snapshot_bytes":62406656,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726146745,"remote_snapshot_timestamp":1726146745,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:13:17image-primary6b711ef4c-c601-46a4-a872-1c8abdd1ca45up+replayingreplaying, {"bytes_per_second":3268608.0,"bytes_per_snapshot":65372160.0,"last_snapshot_bytes":65372160,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726146740,"remote_snapshot_timestamp":1726146740,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:13:17image-primary773a79146-a78a-4543-843c-6cef99c271d3up+replayingreplaying, {"bytes_per_second":3253452.8,"bytes_per_snapshot":65069056.0,"last_snapshot_bytes":65069056,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726146745,"remote_snapshot_timestamp":1726146745,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:13:17image-primary8fa7b2037-727d-4ca0-abbc-a51e250f5ecfup+replayingreplaying, {"bytes_per_second":3252428.8,"bytes_per_snapshot":65048576.0,"last_snapshot_bytes":65048576,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726146746,"remote_snapshot_timestamp":1726146746,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:13:17image-primary977edc192-6758-44c1-b93f-e8f7df97992aup+replayingreplaying, {"bytes_per_second":3260211.2,"bytes_per_snapshot":65204224.0,"last_snapshot_bytes":65204224,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726146746,"remote_snapshot_timestamp":1726146746,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:13:17' 2024-09-12T13:13:36.213 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:13:36.214 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary6'\'']/last_update' 2024-09-12T13:13:36.217 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:13:17' 2024-09-12T13:13:36.218 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary6'\'']/state' 2024-09-12T13:13:36.222 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+replaying 2024-09-12T13:13:36.222 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+replaying 2024-09-12T13:13:36.223 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+replaying 2024-09-12T13:13:36.225 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+replaying 2024-09-12T13:13:36.225 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:13:36.226 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:13:36.226 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ remove_image cluster2 mirror image-primary6 2024-09-12T13:13:36.226 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:13:36.226 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:13:36.226 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:13:36.226 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 snap purge mirror/image-primary6 2024-09-12T13:13:36.289 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 rm mirror/image-primary6 2024-09-12T13:13:36.480 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:13:36.480+0000 7f67e5200640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x5605566e5380 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-09-12T13:13:36.481 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:13:36.480+0000 7f67eca00640 -1 rbd::mirror::ImageReplayer: 0x5605561ad400 [3/b711ef4c-c601-46a4-a872-1c8abdd1ca45] operator(): start failed: (85) Interrupted system call should be restarted 2024-09-12T13:13:38.164 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-09-12T13:13:38.170 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_image_removal cluster1 mirror image-primary6 2024-09-12T13:13:38.171 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:13:38.171 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:13:38.171 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:13:38.171 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:13:38.171 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 ls mirror 2024-09-12T13:13:38.171 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep -wq image-primary6 2024-09-12T13:13:38.225 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:13:38.225 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rm -rf test-primary6 2024-09-12T13:13:38.228 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:13:38.228 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ IMG=image-primary7 2024-09-12T13:13:38.228 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_replaying_status_in_pool_dir cluster1 mirror image-primary7 2024-09-12T13:13:38.229 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:13:38.229 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:13:38.229 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:13:38.229 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' snapshot = journal ']' 2024-09-12T13:13:38.229 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary7 up+replaying 2024-09-12T13:13:38.229 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:13:38.229 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:13:38.229 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:13:38.229 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:13:38.229 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:13:38.229 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:13:38.229 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:13:38.229 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:13:39.230 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary7 up+replaying '' '' 2024-09-12T13:13:39.230 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:13:39.230 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:13:39.230 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:13:39.231 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:13:39.231 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:13:39.231 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:13:39.231 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:13:39.231 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:13:39.231 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary7 2024-09-12T13:13:39.316 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary7: 2024-09-12T13:13:39.317 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 73a79146-a78a-4543-843c-6cef99c271d3 2024-09-12T13:13:39.317 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+replaying 2024-09-12T13:13:39.317 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: replaying, {"bytes_per_second":3253452.8,"bytes_per_snapshot":65069056.0,"last_snapshot_bytes":65069056,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726146745,"remote_snapshot_timestamp":1726146745,"replay_state":"idle"} 2024-09-12T13:13:39.317 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:13:39.317 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:13:17' 2024-09-12T13:13:39.317 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+replaying' 2024-09-12T13:13:39.319 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+replaying 2024-09-12T13:13:39.320 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:13:39.322 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: replaying, {"bytes_per_second":3253452.8,"bytes_per_snapshot":65069056.0,"last_snapshot_bytes":65069056,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726146745,"remote_snapshot_timestamp":1726146745,"replay_state":"idle"} 2024-09-12T13:13:39.322 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:13:39.323 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+replaying 2024-09-12T13:13:39.323 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:13:39.325 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+replaying 2024-09-12T13:13:39.325 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:13:39.328 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:13:39.328 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:13:39.329 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:13:39.333 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:13:17' 2024-09-12T13:13:39.333 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary7 up+replaying '2024-09-12 13:13:17' 2024-09-12T13:13:39.333 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:13:39.333 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:13:39.333 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:13:39.333 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:13:39.333 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:13:17' 2024-09-12T13:13:39.334 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:13:39.334 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:13:39.334 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-09-12T13:13:39.428 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING441934232mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary102286adb1-ff3d-442a-9aac-6b38a8513aaaup+replayingreplaying, {"bytes_per_second":3120332.8,"bytes_per_snapshot":62406656.0,"last_snapshot_bytes":62406656,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726146745,"remote_snapshot_timestamp":1726146745,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:13:17image-primary773a79146-a78a-4543-843c-6cef99c271d3up+replayingreplaying, {"bytes_per_second":3253452.8,"bytes_per_snapshot":65069056.0,"last_snapshot_bytes":65069056,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726146745,"remote_snapshot_timestamp":1726146745,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:13:17image-primary8fa7b2037-727d-4ca0-abbc-a51e250f5ecfup+replayingreplaying, {"bytes_per_second":3252428.8,"bytes_per_snapshot":65048576.0,"last_snapshot_bytes":65048576,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726146746,"remote_snapshot_timestamp":1726146746,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:13:17image-primary977edc192-6758-44c1-b93f-e8f7df97992aup+replayingreplaying, {"bytes_per_second":3260211.2,"bytes_per_snapshot":65204224.0,"last_snapshot_bytes":65204224,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726146746,"remote_snapshot_timestamp":1726146746,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:13:17' 2024-09-12T13:13:39.428 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:13:39.428 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary7'\'']/last_update' 2024-09-12T13:13:39.432 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:13:17' 2024-09-12T13:13:39.433 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary7'\'']/state' 2024-09-12T13:13:39.437 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+replaying 2024-09-12T13:13:39.437 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+replaying 2024-09-12T13:13:39.438 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+replaying 2024-09-12T13:13:39.440 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+replaying 2024-09-12T13:13:39.440 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:13:39.440 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:13:39.441 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ remove_image cluster2 mirror image-primary7 2024-09-12T13:13:39.441 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:13:39.441 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:13:39.441 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:13:39.441 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 snap purge mirror/image-primary7 2024-09-12T13:13:39.501 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 rm mirror/image-primary7 2024-09-12T13:13:39.634 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:13:39.633+0000 7f67e5200640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x56055bce9380 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-09-12T13:13:39.634 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:13:39.633+0000 7f67ed400640 -1 rbd::mirror::ImageReplayer: 0x56055c198c80 [3/73a79146-a78a-4543-843c-6cef99c271d3] operator(): start failed: (85) Interrupted system call should be restarted 2024-09-12T13:13:40.899 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-09-12T13:13:40.905 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_image_removal cluster1 mirror image-primary7 2024-09-12T13:13:40.905 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:13:40.905 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:13:40.905 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:13:40.905 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:13:40.906 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 ls mirror 2024-09-12T13:13:40.906 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep -wq image-primary7 2024-09-12T13:13:40.954 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:13:40.954 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rm -rf test-primary7 2024-09-12T13:13:40.958 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:13:40.958 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ IMG=image-primary8 2024-09-12T13:13:40.958 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_replaying_status_in_pool_dir cluster1 mirror image-primary8 2024-09-12T13:13:40.958 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:13:40.958 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:13:40.958 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:13:40.959 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' snapshot = journal ']' 2024-09-12T13:13:40.959 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary8 up+replaying 2024-09-12T13:13:40.959 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:13:40.959 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:13:40.959 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:13:40.960 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:13:40.960 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:13:40.960 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:13:40.960 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:13:40.960 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:13:41.959 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary8 up+replaying '' '' 2024-09-12T13:13:41.959 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:13:41.959 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:13:41.960 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:13:41.960 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:13:41.960 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:13:41.960 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:13:41.960 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:13:41.960 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:13:41.960 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary8 2024-09-12T13:13:42.049 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary8: 2024-09-12T13:13:42.049 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: fa7b2037-727d-4ca0-abbc-a51e250f5ecf 2024-09-12T13:13:42.049 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+replaying 2024-09-12T13:13:42.049 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: replaying, {"bytes_per_second":3252428.8,"bytes_per_snapshot":65048576.0,"last_snapshot_bytes":65048576,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726146746,"remote_snapshot_timestamp":1726146746,"replay_state":"idle"} 2024-09-12T13:13:42.049 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:13:42.050 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:13:17' 2024-09-12T13:13:42.050 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+replaying' 2024-09-12T13:13:42.052 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+replaying 2024-09-12T13:13:42.052 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:13:42.055 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: replaying, {"bytes_per_second":3252428.8,"bytes_per_snapshot":65048576.0,"last_snapshot_bytes":65048576,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726146746,"remote_snapshot_timestamp":1726146746,"replay_state":"idle"} 2024-09-12T13:13:42.055 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:13:42.055 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+replaying 2024-09-12T13:13:42.056 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:13:42.058 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+replaying 2024-09-12T13:13:42.058 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:13:42.061 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:13:42.061 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:13:42.062 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:13:42.066 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:13:17' 2024-09-12T13:13:42.066 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary8 up+replaying '2024-09-12 13:13:17' 2024-09-12T13:13:42.066 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:13:42.066 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:13:42.066 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:13:42.066 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:13:42.066 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:13:17' 2024-09-12T13:13:42.066 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:13:42.067 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:13:42.067 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-09-12T13:13:42.154 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING341934232mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary102286adb1-ff3d-442a-9aac-6b38a8513aaaup+replayingreplaying, {"bytes_per_second":3120332.8,"bytes_per_snapshot":62406656.0,"last_snapshot_bytes":62406656,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726146745,"remote_snapshot_timestamp":1726146745,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:13:17image-primary8fa7b2037-727d-4ca0-abbc-a51e250f5ecfup+replayingreplaying, {"bytes_per_second":3252428.8,"bytes_per_snapshot":65048576.0,"last_snapshot_bytes":65048576,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726146746,"remote_snapshot_timestamp":1726146746,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:13:17image-primary977edc192-6758-44c1-b93f-e8f7df97992aup+replayingreplaying, {"bytes_per_second":3260211.2,"bytes_per_snapshot":65204224.0,"last_snapshot_bytes":65204224,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726146746,"remote_snapshot_timestamp":1726146746,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:13:17' 2024-09-12T13:13:42.154 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:13:42.154 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary8'\'']/last_update' 2024-09-12T13:13:42.158 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:13:17' 2024-09-12T13:13:42.159 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary8'\'']/state' 2024-09-12T13:13:42.163 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+replaying 2024-09-12T13:13:42.163 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+replaying 2024-09-12T13:13:42.163 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+replaying 2024-09-12T13:13:42.165 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+replaying 2024-09-12T13:13:42.166 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:13:42.166 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:13:42.167 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ remove_image cluster2 mirror image-primary8 2024-09-12T13:13:42.167 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:13:42.167 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:13:42.167 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:13:42.167 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 snap purge mirror/image-primary8 2024-09-12T13:13:42.227 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 rm mirror/image-primary8 2024-09-12T13:13:42.368 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:13:42.368+0000 7f67e4800640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x560556582820 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-09-12T13:13:42.369 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:13:42.368+0000 7f67eca00640 -1 rbd::mirror::ImageReplayer: 0x5605561acf00 [3/fa7b2037-727d-4ca0-abbc-a51e250f5ecf] operator(): start failed: (85) Interrupted system call should be restarted 2024-09-12T13:13:43.899 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-09-12T13:13:43.905 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_image_removal cluster1 mirror image-primary8 2024-09-12T13:13:43.906 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:13:43.906 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:13:43.906 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:13:43.906 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:13:43.906 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 ls mirror 2024-09-12T13:13:43.906 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep -wq image-primary8 2024-09-12T13:13:43.963 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:13:43.963 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rm -rf test-primary8 2024-09-12T13:13:43.966 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:13:43.966 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ IMG=image-primary9 2024-09-12T13:13:43.966 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_replaying_status_in_pool_dir cluster1 mirror image-primary9 2024-09-12T13:13:43.966 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:13:43.967 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:13:43.967 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:13:43.967 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' snapshot = journal ']' 2024-09-12T13:13:43.967 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary9 up+replaying 2024-09-12T13:13:43.967 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:13:43.967 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:13:43.967 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:13:43.967 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:13:43.968 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:13:43.968 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:13:43.968 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:13:43.968 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:13:44.968 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary9 up+replaying '' '' 2024-09-12T13:13:44.969 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:13:44.969 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:13:44.969 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:13:44.969 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:13:44.969 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:13:44.969 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:13:44.969 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:13:44.969 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:13:44.970 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary9 2024-09-12T13:13:45.064 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary9: 2024-09-12T13:13:45.064 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 77edc192-6758-44c1-b93f-e8f7df97992a 2024-09-12T13:13:45.064 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+replaying 2024-09-12T13:13:45.064 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: replaying, {"bytes_per_second":3260211.2,"bytes_per_snapshot":65204224.0,"last_snapshot_bytes":65204224,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726146746,"remote_snapshot_timestamp":1726146746,"replay_state":"idle"} 2024-09-12T13:13:45.065 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:13:45.065 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:13:17' 2024-09-12T13:13:45.065 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+replaying' 2024-09-12T13:13:45.066 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+replaying 2024-09-12T13:13:45.067 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:13:45.069 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: replaying, {"bytes_per_second":3260211.2,"bytes_per_snapshot":65204224.0,"last_snapshot_bytes":65204224,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726146746,"remote_snapshot_timestamp":1726146746,"replay_state":"idle"} 2024-09-12T13:13:45.070 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:13:45.070 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+replaying 2024-09-12T13:13:45.071 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:13:45.072 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+replaying 2024-09-12T13:13:45.073 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:13:45.075 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:13:45.076 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:13:45.076 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:13:45.080 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:13:17' 2024-09-12T13:13:45.080 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary9 up+replaying '2024-09-12 13:13:17' 2024-09-12T13:13:45.081 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:13:45.081 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:13:45.081 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:13:45.081 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:13:45.081 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:13:17' 2024-09-12T13:13:45.081 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:13:45.081 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:13:45.081 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-09-12T13:13:45.185 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING241934232mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary102286adb1-ff3d-442a-9aac-6b38a8513aaaup+replayingreplaying, {"bytes_per_second":3120332.8,"bytes_per_snapshot":62406656.0,"last_snapshot_bytes":62406656,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726146745,"remote_snapshot_timestamp":1726146745,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:13:17image-primary977edc192-6758-44c1-b93f-e8f7df97992aup+replayingreplaying, {"bytes_per_second":3260211.2,"bytes_per_snapshot":65204224.0,"last_snapshot_bytes":65204224,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726146746,"remote_snapshot_timestamp":1726146746,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:13:17' 2024-09-12T13:13:45.185 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:13:45.186 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary9'\'']/last_update' 2024-09-12T13:13:45.190 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:13:17' 2024-09-12T13:13:45.191 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary9'\'']/state' 2024-09-12T13:13:45.194 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+replaying 2024-09-12T13:13:45.195 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+replaying 2024-09-12T13:13:45.195 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+replaying 2024-09-12T13:13:45.197 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+replaying 2024-09-12T13:13:45.198 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:13:45.198 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:13:45.198 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ remove_image cluster2 mirror image-primary9 2024-09-12T13:13:45.198 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:13:45.198 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:13:45.199 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:13:45.199 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 snap purge mirror/image-primary9 2024-09-12T13:13:45.270 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 rm mirror/image-primary9 2024-09-12T13:13:45.397 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:13:45.396+0000 7f67e5200640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x56055bf60340 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-09-12T13:13:45.397 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:13:45.396+0000 7f67ed400640 -1 rbd::mirror::ImageReplayer: 0x56055c198500 [3/77edc192-6758-44c1-b93f-e8f7df97992a] operator(): start failed: (85) Interrupted system call should be restarted 2024-09-12T13:13:46.901 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-09-12T13:13:46.907 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_image_removal cluster1 mirror image-primary9 2024-09-12T13:13:46.907 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:13:46.907 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:13:46.907 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:13:46.907 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:13:46.907 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 ls mirror 2024-09-12T13:13:46.908 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep -wq image-primary9 2024-09-12T13:13:46.957 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:13:46.957 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rm -rf test-primary9 2024-09-12T13:13:46.959 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:13:46.959 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ IMG=image-primary10 2024-09-12T13:13:46.959 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_replaying_status_in_pool_dir cluster1 mirror image-primary10 2024-09-12T13:13:46.960 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:13:46.960 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:13:46.960 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:13:46.960 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' snapshot = journal ']' 2024-09-12T13:13:46.960 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary10 up+replaying 2024-09-12T13:13:46.960 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:13:46.960 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:13:46.960 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:13:46.960 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:13:46.960 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:13:46.960 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:13:46.960 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:13:46.960 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:13:47.962 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary10 up+replaying '' '' 2024-09-12T13:13:47.962 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:13:47.962 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:13:47.962 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:13:47.962 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:13:47.962 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:13:47.963 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:13:47.963 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:13:47.963 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:13:47.963 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary10 2024-09-12T13:13:48.077 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary10: 2024-09-12T13:13:48.078 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 2286adb1-ff3d-442a-9aac-6b38a8513aaa 2024-09-12T13:13:48.078 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+replaying 2024-09-12T13:13:48.078 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: replaying, {"bytes_per_second":0.0,"bytes_per_snapshot":62406656.0,"last_snapshot_bytes":62406656,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726146745,"remote_snapshot_timestamp":1726146745,"replay_state":"idle"} 2024-09-12T13:13:48.078 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:13:48.078 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:13:47' 2024-09-12T13:13:48.078 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+replaying' 2024-09-12T13:13:48.080 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+replaying 2024-09-12T13:13:48.080 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:13:48.084 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: replaying, {"bytes_per_second":0.0,"bytes_per_snapshot":62406656.0,"last_snapshot_bytes":62406656,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726146745,"remote_snapshot_timestamp":1726146745,"replay_state":"idle"} 2024-09-12T13:13:48.085 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:13:48.085 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+replaying 2024-09-12T13:13:48.085 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:13:48.086 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+replaying 2024-09-12T13:13:48.087 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:13:48.089 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:13:48.090 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:13:48.091 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:13:48.094 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:13:47' 2024-09-12T13:13:48.095 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary10 up+replaying '2024-09-12 13:13:47' 2024-09-12T13:13:48.095 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:13:48.095 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:13:48.095 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:13:48.095 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:13:48.095 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:13:47' 2024-09-12T13:13:48.095 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:13:48.095 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:13:48.096 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-09-12T13:13:48.206 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING141934232mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary102286adb1-ff3d-442a-9aac-6b38a8513aaaup+replayingreplaying, {"bytes_per_second":0.0,"bytes_per_snapshot":62406656.0,"last_snapshot_bytes":62406656,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726146745,"remote_snapshot_timestamp":1726146745,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:13:47' 2024-09-12T13:13:48.206 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:13:48.206 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary10'\'']/last_update' 2024-09-12T13:13:48.210 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:13:47' 2024-09-12T13:13:48.211 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary10'\'']/state' 2024-09-12T13:13:48.215 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+replaying 2024-09-12T13:13:48.215 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+replaying 2024-09-12T13:13:48.215 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+replaying 2024-09-12T13:13:48.217 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+replaying 2024-09-12T13:13:48.218 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:13:48.218 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:13:48.218 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ remove_image cluster2 mirror image-primary10 2024-09-12T13:13:48.218 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:13:48.218 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:13:48.219 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:13:48.219 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 snap purge mirror/image-primary10 2024-09-12T13:13:48.279 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 rm mirror/image-primary10 2024-09-12T13:13:48.412 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:13:48.413+0000 7f67e5200640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x56055b973860 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-09-12T13:13:48.412 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:13:48.413+0000 7f67ed400640 -1 rbd::mirror::ImageReplayer: 0x56055c198780 [3/2286adb1-ff3d-442a-9aac-6b38a8513aaa] operator(): start failed: (85) Interrupted system call should be restarted 2024-09-12T13:13:49.905 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-09-12T13:13:49.911 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_image_removal cluster1 mirror image-primary10 2024-09-12T13:13:49.911 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:13:49.912 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:13:49.912 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:13:49.912 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:13:49.912 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 ls mirror 2024-09-12T13:13:49.912 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep -wq image-primary10 2024-09-12T13:13:49.962 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:13:49.962 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rm -rf test-primary10 2024-09-12T13:13:49.965 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..10} 2024-09-12T13:13:49.965 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEVS=() 2024-09-12T13:13:49.965 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ SNAP_PIDS=() 2024-09-12T13:13:49.965 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ COMPARE_PIDS=() 2024-09-12T13:13:49.965 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ WORKLOAD_PIDS=() 2024-09-12T13:13:49.965 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ RET=0 2024-09-12T13:13:49.965 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:13:49.965 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ IMG=image-primary1 2024-09-12T13:13:49.966 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ MNTPT=test-primary1 2024-09-12T13:13:49.966 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ create_image_and_enable_mirror cluster1 mirror image-primary1 snapshot 10G 2024-09-12T13:13:49.966 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:13:49.966 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:13:49.966 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:13:49.966 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:13:49.966 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:13:49.966 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:13:49.966 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mode=snapshot 2024-09-12T13:13:49.966 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n snapshot ']' 2024-09-12T13:13:49.966 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:13:49.966 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ create_image cluster1 mirror image-primary1 10G 2024-09-12T13:13:49.966 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:13:49.967 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:13:49.967 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:13:49.967 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:13:49.967 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:13:49.967 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:13:49.967 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local size=128 2024-09-12T13:13:49.967 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n 10G ']' 2024-09-12T13:13:49.967 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ size=10G 2024-09-12T13:13:49.967 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:13:49.967 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 create --size 10G --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image-primary1 2024-09-12T13:13:50.034 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ is_pool_mirror_mode_image mirror 2024-09-12T13:13:50.034 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:13:50.034 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' image = image ']' 2024-09-12T13:13:50.034 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:13:50.034 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ enable_mirror cluster1 mirror image-primary1 snapshot 2024-09-12T13:13:50.034 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:13:50.034 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:13:50.034 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:13:50.034 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mode=snapshot 2024-09-12T13:13:50.034 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 mirror image enable mirror/image-primary1 snapshot 2024-09-12T13:13:50.860 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Mirroring enabled 2024-09-12T13:13:50.872 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 info mirror/image-primary1 2024-09-12T13:13:50.934 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:rbd image 'image-primary1': 2024-09-12T13:13:50.934 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: size 10 GiB in 2560 objects 2024-09-12T13:13:50.934 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: order 22 (4 MiB objects) 2024-09-12T13:13:50.934 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: snapshot_count: 1 2024-09-12T13:13:50.934 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: id: 1a72a6a7c6e7 2024-09-12T13:13:50.934 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: block_name_prefix: rbd_data.1a72a6a7c6e7 2024-09-12T13:13:50.935 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: format: 2 2024-09-12T13:13:50.935 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: features: layering, exclusive-lock, object-map, fast-diff 2024-09-12T13:13:50.935 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: op_features: 2024-09-12T13:13:50.935 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: flags: 2024-09-12T13:13:50.935 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: create_timestamp: Thu Sep 12 13:13:50 2024 2024-09-12T13:13:50.935 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: access_timestamp: Thu Sep 12 13:13:50 2024 2024-09-12T13:13:50.935 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: modify_timestamp: Thu Sep 12 13:13:50 2024 2024-09-12T13:13:50.935 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring state: enabled 2024-09-12T13:13:50.935 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring mode: snapshot 2024-09-12T13:13:50.935 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring global id: 96ef9ec0-0a45-4e60-bcf0-79d28853db5e 2024-09-12T13:13:50.935 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring primary: true 2024-09-12T13:13:50.940 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \n\b\d ]] 2024-09-12T13:13:50.940 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \k\r\b\d ]] 2024-09-12T13:13:50.941 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo rbd --cluster cluster1 device map -t krbd mirror/image-primary1 2024-09-12T13:13:51.098 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEV=/dev/rbd0 2024-09-12T13:13:51.098 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEVS+=($DEV) 2024-09-12T13:13:51.098 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo mkfs.ext4 /dev/rbd0 2024-09-12T13:13:51.149 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:mke2fs 1.46.5 (30-Dec-2021) 2024-09-12T13:13:51.161 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Discarding device blocks: 0/2621440 done 2024-09-12T13:13:51.161 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Creating filesystem with 2621440 4k blocks and 655360 inodes 2024-09-12T13:13:51.161 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Filesystem UUID: 53e32fc3-a928-43a9-8aa3-7745ec89afc9 2024-09-12T13:13:51.161 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Superblock backups stored on blocks: 2024-09-12T13:13:51.161 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 2024-09-12T13:13:51.161 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 2024-09-12T13:13:51.162 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Allocating group tables: 0/80 done 2024-09-12T13:13:51.162 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Writing inode tables: 0/80 done 2024-09-12T13:13:51.166 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Creating journal (16384 blocks): done 2024-09-12T13:13:51.211 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Writing superblocks and filesystem accounting information: 0/80 done 2024-09-12T13:13:51.212 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 2024-09-12T13:13:51.214 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mkdir test-primary1 2024-09-12T13:13:51.217 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo mount /dev/rbd0 test-primary1 2024-09-12T13:13:51.273 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ whoami 2024-09-12T13:13:51.301 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo chown ubuntu test-primary1 2024-09-12T13:13:51.332 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ SNAP_PIDS+=($!) 2024-09-12T13:13:51.332 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ take_mirror_snapshots cluster1 mirror image-primary1 2024-09-12T13:13:51.332 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:13:51.332 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:13:51.332 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:13:51.333 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:13:51.333 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:13:51.333 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:13:51.333 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:13:51.333 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:13:51.333 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ WORKLOAD_PIDS+=($!) 2024-09-12T13:13:51.333 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:13:51.333 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:13:51.333 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ IMG=image-primary2 2024-09-12T13:13:51.333 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ MNTPT=test-primary2 2024-09-12T13:13:51.333 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ create_image_and_enable_mirror cluster1 mirror image-primary2 snapshot 10G 2024-09-12T13:13:51.333 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:13:51.333 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:13:51.333 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:13:51.334 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:13:51.334 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:13:51.334 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:13:51.334 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mode=snapshot 2024-09-12T13:13:51.334 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n snapshot ']' 2024-09-12T13:13:51.334 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:13:51.334 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ create_image cluster1 mirror image-primary2 10G 2024-09-12T13:13:51.334 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:13:51.334 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:13:51.334 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:13:51.334 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:13:51.334 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ slow_untar_workload test-primary1 2024-09-12T13:13:51.334 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:13:51.335 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:13:51.335 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local size=128 2024-09-12T13:13:51.335 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n 10G ']' 2024-09-12T13:13:51.335 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ size=10G 2024-09-12T13:13:51.335 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:13:51.335 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 create --size 10G --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image-primary2 2024-09-12T13:13:51.335 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mountpt=test-primary1 2024-09-12T13:13:51.335 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ cp linux-5.4.tar.gz test-primary1 2024-09-12T13:13:51.403 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ is_pool_mirror_mode_image mirror 2024-09-12T13:13:51.403 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:13:51.403 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' image = image ']' 2024-09-12T13:13:51.404 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:13:51.404 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ enable_mirror cluster1 mirror image-primary2 snapshot 2024-09-12T13:13:51.404 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:13:51.404 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:13:51.404 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:13:51.404 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mode=snapshot 2024-09-12T13:13:51.404 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 mirror image enable mirror/image-primary2 snapshot 2024-09-12T13:13:51.625 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local ret=0 2024-09-12T13:13:51.625 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ timeout 5m bash -c 'zcat test-primary1/linux-5.4.tar.gz | pv -L 256K | tar xf - -C test-primary1' 2024-09-12T13:13:51.870 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Mirroring enabled 2024-09-12T13:13:51.874 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 664 2024-09-12T13:13:51.885 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:13:51.887 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 info mirror/image-primary2 2024-09-12T13:13:51.963 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:rbd image 'image-primary2': 2024-09-12T13:13:51.963 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: size 10 GiB in 2560 objects 2024-09-12T13:13:51.963 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: order 22 (4 MiB objects) 2024-09-12T13:13:51.963 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: snapshot_count: 1 2024-09-12T13:13:51.963 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: id: 1a82945779b2 2024-09-12T13:13:51.963 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: block_name_prefix: rbd_data.1a82945779b2 2024-09-12T13:13:51.963 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: format: 2 2024-09-12T13:13:51.964 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: features: layering, exclusive-lock, object-map, fast-diff 2024-09-12T13:13:51.964 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: op_features: 2024-09-12T13:13:51.964 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: flags: 2024-09-12T13:13:51.964 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: create_timestamp: Thu Sep 12 13:13:51 2024 2024-09-12T13:13:51.964 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: access_timestamp: Thu Sep 12 13:13:51 2024 2024-09-12T13:13:51.964 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: modify_timestamp: Thu Sep 12 13:13:51 2024 2024-09-12T13:13:51.964 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring state: enabled 2024-09-12T13:13:51.964 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring mode: snapshot 2024-09-12T13:13:51.964 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring global id: 3ceee0af-566c-4f40-8fbb-b3b44ed7d010 2024-09-12T13:13:51.964 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring primary: true 2024-09-12T13:13:51.968 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \n\b\d ]] 2024-09-12T13:13:51.968 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \k\r\b\d ]] 2024-09-12T13:13:51.969 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo rbd --cluster cluster1 device map -t krbd mirror/image-primary2 2024-09-12T13:13:52.132 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEV=/dev/rbd1 2024-09-12T13:13:52.132 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEVS+=($DEV) 2024-09-12T13:13:52.132 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo mkfs.ext4 /dev/rbd1 2024-09-12T13:13:52.153 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:mke2fs 1.46.5 (30-Dec-2021) 2024-09-12T13:13:52.166 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Discarding device blocks: 0/2621440 done 2024-09-12T13:13:52.166 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Creating filesystem with 2621440 4k blocks and 655360 inodes 2024-09-12T13:13:52.167 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Filesystem UUID: cb80f22d-19bc-418b-94e4-b113666da0c8 2024-09-12T13:13:52.167 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Superblock backups stored on blocks: 2024-09-12T13:13:52.167 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 2024-09-12T13:13:52.167 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 2024-09-12T13:13:52.167 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Allocating group tables: 0/80 done 2024-09-12T13:13:52.167 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Writing inode tables: 0/80 done 2024-09-12T13:13:52.172 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Creating journal (16384 blocks): done 2024-09-12T13:13:52.236 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Writing superblocks and filesystem accounting information: 0/80 done 2024-09-12T13:13:52.236 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 2024-09-12T13:13:52.238 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mkdir test-primary2 2024-09-12T13:13:52.241 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo mount /dev/rbd1 test-primary2 2024-09-12T13:13:52.321 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ whoami 2024-09-12T13:13:52.323 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo chown ubuntu test-primary2 2024-09-12T13:13:52.349 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ SNAP_PIDS+=($!) 2024-09-12T13:13:52.349 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ take_mirror_snapshots cluster1 mirror image-primary2 2024-09-12T13:13:52.349 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:13:52.349 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:13:52.349 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:13:52.350 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:13:52.350 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:13:52.350 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:13:52.350 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:13:52.350 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:13:52.350 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:13:52.350 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ WORKLOAD_PIDS+=($!) 2024-09-12T13:13:52.350 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:13:52.350 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ IMG=image-primary3 2024-09-12T13:13:52.350 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ slow_untar_workload test-primary2 2024-09-12T13:13:52.350 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ MNTPT=test-primary3 2024-09-12T13:13:52.350 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ create_image_and_enable_mirror cluster1 mirror image-primary3 snapshot 10G 2024-09-12T13:13:52.350 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:13:52.351 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:13:52.351 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:13:52.351 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:13:52.351 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:13:52.351 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:13:52.351 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mode=snapshot 2024-09-12T13:13:52.351 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mountpt=test-primary2 2024-09-12T13:13:52.351 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n snapshot ']' 2024-09-12T13:13:52.351 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:13:52.351 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ cp linux-5.4.tar.gz test-primary2 2024-09-12T13:13:52.351 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ create_image cluster1 mirror image-primary3 10G 2024-09-12T13:13:52.351 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:13:52.351 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:13:52.351 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:13:52.352 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:13:52.352 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:13:52.352 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:13:52.352 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local size=128 2024-09-12T13:13:52.352 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n 10G ']' 2024-09-12T13:13:52.352 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ size=10G 2024-09-12T13:13:52.352 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:13:52.352 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 create --size 10G --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image-primary3 2024-09-12T13:13:52.443 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ is_pool_mirror_mode_image mirror 2024-09-12T13:13:52.443 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:13:52.443 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' image = image ']' 2024-09-12T13:13:52.444 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:13:52.444 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ enable_mirror cluster1 mirror image-primary3 snapshot 2024-09-12T13:13:52.444 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:13:52.444 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:13:52.444 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:13:52.444 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mode=snapshot 2024-09-12T13:13:52.444 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 mirror image enable mirror/image-primary3 snapshot 2024-09-12T13:13:52.659 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local ret=0 2024-09-12T13:13:52.660 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ timeout 5m bash -c 'zcat test-primary2/linux-5.4.tar.gz | pv -L 256K | tar xf - -C test-primary2' 2024-09-12T13:13:52.872 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Mirroring enabled 2024-09-12T13:13:52.881 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 666 2024-09-12T13:13:52.893 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:13:52.894 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 info mirror/image-primary3 2024-09-12T13:13:52.970 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:rbd image 'image-primary3': 2024-09-12T13:13:52.971 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: size 10 GiB in 2560 objects 2024-09-12T13:13:52.971 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: order 22 (4 MiB objects) 2024-09-12T13:13:52.971 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: snapshot_count: 1 2024-09-12T13:13:52.971 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: id: 1a907cc1efd 2024-09-12T13:13:52.971 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: block_name_prefix: rbd_data.1a907cc1efd 2024-09-12T13:13:52.971 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: format: 2 2024-09-12T13:13:52.971 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: features: layering, exclusive-lock, object-map, fast-diff 2024-09-12T13:13:52.971 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: op_features: 2024-09-12T13:13:52.971 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: flags: 2024-09-12T13:13:52.971 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: create_timestamp: Thu Sep 12 13:13:52 2024 2024-09-12T13:13:52.971 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: access_timestamp: Thu Sep 12 13:13:52 2024 2024-09-12T13:13:52.971 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: modify_timestamp: Thu Sep 12 13:13:52 2024 2024-09-12T13:13:52.971 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring state: enabled 2024-09-12T13:13:52.972 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring mode: snapshot 2024-09-12T13:13:52.972 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring global id: 50e30e91-e1e0-44f3-8ad3-6fb3a5307bba 2024-09-12T13:13:52.972 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring primary: true 2024-09-12T13:13:52.977 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \n\b\d ]] 2024-09-12T13:13:52.977 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \k\r\b\d ]] 2024-09-12T13:13:52.977 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo rbd --cluster cluster1 device map -t krbd mirror/image-primary3 2024-09-12T13:13:53.125 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEV=/dev/rbd2 2024-09-12T13:13:53.126 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEVS+=($DEV) 2024-09-12T13:13:53.126 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo mkfs.ext4 /dev/rbd2 2024-09-12T13:13:53.146 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:mke2fs 1.46.5 (30-Dec-2021) 2024-09-12T13:13:53.160 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Discarding device blocks: 0/2621440 done 2024-09-12T13:13:53.160 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Creating filesystem with 2621440 4k blocks and 655360 inodes 2024-09-12T13:13:53.160 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Filesystem UUID: b558c3fd-fc24-4927-9ab9-726713c0acd9 2024-09-12T13:13:53.161 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Superblock backups stored on blocks: 2024-09-12T13:13:53.161 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 2024-09-12T13:13:53.161 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 2024-09-12T13:13:53.161 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Allocating group tables: 0/80 done 2024-09-12T13:13:53.161 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Writing inode tables: 0/80 done 2024-09-12T13:13:53.165 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Creating journal (16384 blocks): done 2024-09-12T13:13:53.226 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Writing superblocks and filesystem accounting information: 0/80 done 2024-09-12T13:13:53.226 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 2024-09-12T13:13:53.228 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mkdir test-primary3 2024-09-12T13:13:53.231 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo mount /dev/rbd2 test-primary3 2024-09-12T13:13:53.289 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ whoami 2024-09-12T13:13:53.291 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo chown ubuntu test-primary3 2024-09-12T13:13:53.317 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ SNAP_PIDS+=($!) 2024-09-12T13:13:53.318 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ take_mirror_snapshots cluster1 mirror image-primary3 2024-09-12T13:13:53.318 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:13:53.318 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:13:53.318 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:13:53.319 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:13:53.319 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:13:53.319 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:13:53.319 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:13:53.320 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:13:53.320 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ WORKLOAD_PIDS+=($!) 2024-09-12T13:13:53.320 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:13:53.320 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:13:53.320 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ IMG=image-primary4 2024-09-12T13:13:53.320 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ MNTPT=test-primary4 2024-09-12T13:13:53.320 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ create_image_and_enable_mirror cluster1 mirror image-primary4 snapshot 10G 2024-09-12T13:13:53.321 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:13:53.321 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:13:53.321 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:13:53.321 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:13:53.321 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:13:53.321 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:13:53.321 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mode=snapshot 2024-09-12T13:13:53.322 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n snapshot ']' 2024-09-12T13:13:53.322 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:13:53.322 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ create_image cluster1 mirror image-primary4 10G 2024-09-12T13:13:53.322 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:13:53.322 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:13:53.322 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:13:53.323 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:13:53.323 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:13:53.323 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:13:53.323 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local size=128 2024-09-12T13:13:53.323 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n 10G ']' 2024-09-12T13:13:53.323 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ size=10G 2024-09-12T13:13:53.323 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ slow_untar_workload test-primary3 2024-09-12T13:13:53.324 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:13:53.324 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 create --size 10G --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image-primary4 2024-09-12T13:13:53.324 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mountpt=test-primary3 2024-09-12T13:13:53.324 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ cp linux-5.4.tar.gz test-primary3 2024-09-12T13:13:53.385 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ is_pool_mirror_mode_image mirror 2024-09-12T13:13:53.385 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:13:53.386 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' image = image ']' 2024-09-12T13:13:53.386 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:13:53.386 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ enable_mirror cluster1 mirror image-primary4 snapshot 2024-09-12T13:13:53.386 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:13:53.386 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:13:53.386 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:13:53.386 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mode=snapshot 2024-09-12T13:13:53.386 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 mirror image enable mirror/image-primary4 snapshot 2024-09-12T13:13:53.604 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local ret=0 2024-09-12T13:13:53.604 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ timeout 5m bash -c 'zcat test-primary3/linux-5.4.tar.gz | pv -L 256K | tar xf - -C test-primary3' 2024-09-12T13:13:53.849 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Mirroring enabled 2024-09-12T13:13:53.858 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 668 2024-09-12T13:13:53.864 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 info mirror/image-primary4 2024-09-12T13:13:53.868 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:13:53.933 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:rbd image 'image-primary4': 2024-09-12T13:13:53.933 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: size 10 GiB in 2560 objects 2024-09-12T13:13:53.933 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: order 22 (4 MiB objects) 2024-09-12T13:13:53.933 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: snapshot_count: 1 2024-09-12T13:13:53.933 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: id: 1a9e3bbf4997 2024-09-12T13:13:53.934 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: block_name_prefix: rbd_data.1a9e3bbf4997 2024-09-12T13:13:53.934 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: format: 2 2024-09-12T13:13:53.934 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: features: layering, exclusive-lock, object-map, fast-diff 2024-09-12T13:13:53.934 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: op_features: 2024-09-12T13:13:53.934 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: flags: 2024-09-12T13:13:53.934 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: create_timestamp: Thu Sep 12 13:13:53 2024 2024-09-12T13:13:53.934 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: access_timestamp: Thu Sep 12 13:13:53 2024 2024-09-12T13:13:53.934 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: modify_timestamp: Thu Sep 12 13:13:53 2024 2024-09-12T13:13:53.934 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring state: enabled 2024-09-12T13:13:53.934 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring mode: snapshot 2024-09-12T13:13:53.934 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring global id: 3d37a1b1-ea15-4b8b-8894-ae4f8b6c65cd 2024-09-12T13:13:53.934 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring primary: true 2024-09-12T13:13:53.938 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \n\b\d ]] 2024-09-12T13:13:53.938 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \k\r\b\d ]] 2024-09-12T13:13:53.939 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo rbd --cluster cluster1 device map -t krbd mirror/image-primary4 2024-09-12T13:13:54.060 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEV=/dev/rbd3 2024-09-12T13:13:54.060 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEVS+=($DEV) 2024-09-12T13:13:54.060 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo mkfs.ext4 /dev/rbd3 2024-09-12T13:13:54.081 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:mke2fs 1.46.5 (30-Dec-2021) 2024-09-12T13:13:54.094 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Discarding device blocks: 0/2621440 done 2024-09-12T13:13:54.094 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Creating filesystem with 2621440 4k blocks and 655360 inodes 2024-09-12T13:13:54.094 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Filesystem UUID: ff8de285-8119-40ad-8b8a-65d070d4d56f 2024-09-12T13:13:54.095 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Superblock backups stored on blocks: 2024-09-12T13:13:54.095 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 2024-09-12T13:13:54.095 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 2024-09-12T13:13:54.095 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Allocating group tables: 0/80 done 2024-09-12T13:13:54.095 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Writing inode tables: 0/80 done 2024-09-12T13:13:54.099 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Creating journal (16384 blocks): done 2024-09-12T13:13:54.139 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Writing superblocks and filesystem accounting information: 0/80 done 2024-09-12T13:13:54.139 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 2024-09-12T13:13:54.142 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mkdir test-primary4 2024-09-12T13:13:54.144 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo mount /dev/rbd3 test-primary4 2024-09-12T13:13:54.205 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ whoami 2024-09-12T13:13:54.206 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo chown ubuntu test-primary4 2024-09-12T13:13:54.230 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ SNAP_PIDS+=($!) 2024-09-12T13:13:54.230 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ take_mirror_snapshots cluster1 mirror image-primary4 2024-09-12T13:13:54.230 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:13:54.230 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ WORKLOAD_PIDS+=($!) 2024-09-12T13:13:54.230 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:13:54.230 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:13:54.230 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:13:54.231 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:13:54.231 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ IMG=image-primary5 2024-09-12T13:13:54.231 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ MNTPT=test-primary5 2024-09-12T13:13:54.231 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:13:54.231 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ create_image_and_enable_mirror cluster1 mirror image-primary5 snapshot 10G 2024-09-12T13:13:54.231 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:13:54.231 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ slow_untar_workload test-primary4 2024-09-12T13:13:54.231 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:13:54.231 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:13:54.231 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:13:54.231 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:13:54.231 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:13:54.231 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:13:54.231 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:13:54.232 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:13:54.232 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:13:54.232 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mode=snapshot 2024-09-12T13:13:54.232 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n snapshot ']' 2024-09-12T13:13:54.232 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:13:54.232 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ create_image cluster1 mirror image-primary5 10G 2024-09-12T13:13:54.232 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:13:54.232 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:13:54.232 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mountpt=test-primary4 2024-09-12T13:13:54.232 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:13:54.232 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:13:54.232 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:13:54.232 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:13:54.232 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local size=128 2024-09-12T13:13:54.233 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n 10G ']' 2024-09-12T13:13:54.233 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ cp linux-5.4.tar.gz test-primary4 2024-09-12T13:13:54.233 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ size=10G 2024-09-12T13:13:54.233 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:13:54.233 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 create --size 10G --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image-primary5 2024-09-12T13:13:54.302 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ is_pool_mirror_mode_image mirror 2024-09-12T13:13:54.302 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:13:54.302 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' image = image ']' 2024-09-12T13:13:54.302 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:13:54.302 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ enable_mirror cluster1 mirror image-primary5 snapshot 2024-09-12T13:13:54.302 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:13:54.302 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:13:54.302 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:13:54.302 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mode=snapshot 2024-09-12T13:13:54.302 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 mirror image enable mirror/image-primary5 snapshot 2024-09-12T13:13:54.519 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local ret=0 2024-09-12T13:13:54.519 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ timeout 5m bash -c 'zcat test-primary4/linux-5.4.tar.gz | pv -L 256K | tar xf - -C test-primary4' 2024-09-12T13:13:54.860 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Mirroring enabled 2024-09-12T13:13:54.863 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 670 2024-09-12T13:13:54.873 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 info mirror/image-primary5 2024-09-12T13:13:54.874 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:13:54.888 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:13:54.888 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:13:54.889 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:13:54.889 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:13:54.889 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:13:54.889 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:13:54.940 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:rbd image 'image-primary5': 2024-09-12T13:13:54.941 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: size 10 GiB in 2560 objects 2024-09-12T13:13:54.941 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: order 22 (4 MiB objects) 2024-09-12T13:13:54.941 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: snapshot_count: 1 2024-09-12T13:13:54.941 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: id: 1aacdc3474a9 2024-09-12T13:13:54.941 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: block_name_prefix: rbd_data.1aacdc3474a9 2024-09-12T13:13:54.941 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: format: 2 2024-09-12T13:13:54.941 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: features: layering, exclusive-lock, object-map, fast-diff 2024-09-12T13:13:54.941 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: op_features: 2024-09-12T13:13:54.941 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: flags: 2024-09-12T13:13:54.941 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: create_timestamp: Thu Sep 12 13:13:54 2024 2024-09-12T13:13:54.941 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: access_timestamp: Thu Sep 12 13:13:54 2024 2024-09-12T13:13:54.941 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: modify_timestamp: Thu Sep 12 13:13:54 2024 2024-09-12T13:13:54.942 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring state: enabled 2024-09-12T13:13:54.942 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring mode: snapshot 2024-09-12T13:13:54.942 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring global id: cfd4ea21-2703-4ce9-a4e3-e41c240a1165 2024-09-12T13:13:54.942 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring primary: true 2024-09-12T13:13:54.946 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \n\b\d ]] 2024-09-12T13:13:54.946 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \k\r\b\d ]] 2024-09-12T13:13:54.947 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo rbd --cluster cluster1 device map -t krbd mirror/image-primary5 2024-09-12T13:13:55.069 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEV=/dev/rbd4 2024-09-12T13:13:55.069 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEVS+=($DEV) 2024-09-12T13:13:55.069 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo mkfs.ext4 /dev/rbd4 2024-09-12T13:13:55.090 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:mke2fs 1.46.5 (30-Dec-2021) 2024-09-12T13:13:55.101 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Discarding device blocks: 0/2621440 done 2024-09-12T13:13:55.102 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Creating filesystem with 2621440 4k blocks and 655360 inodes 2024-09-12T13:13:55.102 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Filesystem UUID: 26f6e5bd-7f3d-4dcf-a5d7-db9768e57464 2024-09-12T13:13:55.102 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Superblock backups stored on blocks: 2024-09-12T13:13:55.102 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 2024-09-12T13:13:55.102 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 2024-09-12T13:13:55.102 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Allocating group tables: 0/80 done 2024-09-12T13:13:55.102 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Writing inode tables: 0/80 done 2024-09-12T13:13:55.107 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Creating journal (16384 blocks): done 2024-09-12T13:13:55.147 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Writing superblocks and filesystem accounting information: 0/80 done 2024-09-12T13:13:55.147 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 2024-09-12T13:13:55.149 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mkdir test-primary5 2024-09-12T13:13:55.152 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo mount /dev/rbd4 test-primary5 2024-09-12T13:13:55.206 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ whoami 2024-09-12T13:13:55.208 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo chown ubuntu test-primary5 2024-09-12T13:13:55.233 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ SNAP_PIDS+=($!) 2024-09-12T13:13:55.233 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ take_mirror_snapshots cluster1 mirror image-primary5 2024-09-12T13:13:55.234 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:13:55.234 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:13:55.234 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:13:55.234 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:13:55.234 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:13:55.234 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:13:55.234 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:13:55.234 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:13:55.234 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:13:55.234 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ WORKLOAD_PIDS+=($!) 2024-09-12T13:13:55.234 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:13:55.234 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ IMG=image-primary6 2024-09-12T13:13:55.234 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ MNTPT=test-primary6 2024-09-12T13:13:55.234 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ slow_untar_workload test-primary5 2024-09-12T13:13:55.235 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ create_image_and_enable_mirror cluster1 mirror image-primary6 snapshot 10G 2024-09-12T13:13:55.235 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mountpt=test-primary5 2024-09-12T13:13:55.235 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:13:55.235 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:13:55.235 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:13:55.235 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:13:55.235 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:13:55.235 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:13:55.235 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ cp linux-5.4.tar.gz test-primary5 2024-09-12T13:13:55.235 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mode=snapshot 2024-09-12T13:13:55.235 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n snapshot ']' 2024-09-12T13:13:55.235 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:13:55.235 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ create_image cluster1 mirror image-primary6 10G 2024-09-12T13:13:55.235 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:13:55.236 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:13:55.236 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:13:55.236 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:13:55.236 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:13:55.236 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:13:55.236 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local size=128 2024-09-12T13:13:55.236 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n 10G ']' 2024-09-12T13:13:55.236 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ size=10G 2024-09-12T13:13:55.236 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:13:55.236 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 create --size 10G --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image-primary6 2024-09-12T13:13:55.302 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ is_pool_mirror_mode_image mirror 2024-09-12T13:13:55.302 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:13:55.302 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' image = image ']' 2024-09-12T13:13:55.302 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:13:55.302 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ enable_mirror cluster1 mirror image-primary6 snapshot 2024-09-12T13:13:55.302 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:13:55.302 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:13:55.302 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:13:55.302 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mode=snapshot 2024-09-12T13:13:55.302 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 mirror image enable mirror/image-primary6 snapshot 2024-09-12T13:13:55.520 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local ret=0 2024-09-12T13:13:55.520 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ timeout 5m bash -c 'zcat test-primary5/linux-5.4.tar.gz | pv -L 256K | tar xf - -C test-primary5' 2024-09-12T13:13:55.870 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Mirroring enabled 2024-09-12T13:13:55.874 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 672 2024-09-12T13:13:55.876 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 673 2024-09-12T13:13:55.888 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 info mirror/image-primary6 2024-09-12T13:13:55.888 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:13:55.889 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:13:55.895 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:13:55.895 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:13:55.895 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:13:55.895 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:13:55.895 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:13:55.896 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:13:55.950 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:rbd image 'image-primary6': 2024-09-12T13:13:55.950 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: size 10 GiB in 2560 objects 2024-09-12T13:13:55.950 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: order 22 (4 MiB objects) 2024-09-12T13:13:55.950 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: snapshot_count: 1 2024-09-12T13:13:55.950 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: id: 1abd5a0021ed 2024-09-12T13:13:55.951 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: block_name_prefix: rbd_data.1abd5a0021ed 2024-09-12T13:13:55.951 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: format: 2 2024-09-12T13:13:55.951 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: features: layering, exclusive-lock, object-map, fast-diff 2024-09-12T13:13:55.951 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: op_features: 2024-09-12T13:13:55.951 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: flags: 2024-09-12T13:13:55.951 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: create_timestamp: Thu Sep 12 13:13:55 2024 2024-09-12T13:13:55.951 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: access_timestamp: Thu Sep 12 13:13:55 2024 2024-09-12T13:13:55.951 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: modify_timestamp: Thu Sep 12 13:13:55 2024 2024-09-12T13:13:55.951 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring state: enabled 2024-09-12T13:13:55.951 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring mode: snapshot 2024-09-12T13:13:55.951 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring global id: 14efbe85-d4d9-42ff-aa35-0963a8746f30 2024-09-12T13:13:55.951 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring primary: true 2024-09-12T13:13:55.956 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \n\b\d ]] 2024-09-12T13:13:55.956 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \k\r\b\d ]] 2024-09-12T13:13:55.957 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo rbd --cluster cluster1 device map -t krbd mirror/image-primary6 2024-09-12T13:13:56.078 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEV=/dev/rbd5 2024-09-12T13:13:56.079 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEVS+=($DEV) 2024-09-12T13:13:56.079 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo mkfs.ext4 /dev/rbd5 2024-09-12T13:13:56.099 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:mke2fs 1.46.5 (30-Dec-2021) 2024-09-12T13:13:56.113 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Discarding device blocks: 0/2621440 done 2024-09-12T13:13:56.113 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Creating filesystem with 2621440 4k blocks and 655360 inodes 2024-09-12T13:13:56.113 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Filesystem UUID: cf03f74c-1455-4eaa-b82d-84878b91bb03 2024-09-12T13:13:56.113 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Superblock backups stored on blocks: 2024-09-12T13:13:56.113 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 2024-09-12T13:13:56.113 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 2024-09-12T13:13:56.114 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Allocating group tables: 0/80 done 2024-09-12T13:13:56.114 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Writing inode tables: 0/80 done 2024-09-12T13:13:56.117 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Creating journal (16384 blocks): done 2024-09-12T13:13:56.161 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Writing superblocks and filesystem accounting information: 0/80 done 2024-09-12T13:13:56.161 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 2024-09-12T13:13:56.163 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mkdir test-primary6 2024-09-12T13:13:56.166 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo mount /dev/rbd5 test-primary6 2024-09-12T13:13:56.219 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ whoami 2024-09-12T13:13:56.221 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo chown ubuntu test-primary6 2024-09-12T13:13:56.247 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ SNAP_PIDS+=($!) 2024-09-12T13:13:56.247 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ take_mirror_snapshots cluster1 mirror image-primary6 2024-09-12T13:13:56.247 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:13:56.247 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:13:56.247 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:13:56.247 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:13:56.247 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:13:56.247 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:13:56.247 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:13:56.247 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:13:56.248 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:13:56.248 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ WORKLOAD_PIDS+=($!) 2024-09-12T13:13:56.248 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:13:56.248 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ IMG=image-primary7 2024-09-12T13:13:56.248 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ MNTPT=test-primary7 2024-09-12T13:13:56.248 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ create_image_and_enable_mirror cluster1 mirror image-primary7 snapshot 10G 2024-09-12T13:13:56.248 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ slow_untar_workload test-primary6 2024-09-12T13:13:56.248 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:13:56.248 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:13:56.248 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:13:56.248 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:13:56.248 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:13:56.249 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:13:56.249 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mode=snapshot 2024-09-12T13:13:56.249 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n snapshot ']' 2024-09-12T13:13:56.249 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:13:56.249 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ create_image cluster1 mirror image-primary7 10G 2024-09-12T13:13:56.249 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mountpt=test-primary6 2024-09-12T13:13:56.249 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ cp linux-5.4.tar.gz test-primary6 2024-09-12T13:13:56.249 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:13:56.249 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:13:56.249 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:13:56.249 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:13:56.249 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:13:56.249 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:13:56.250 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local size=128 2024-09-12T13:13:56.250 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n 10G ']' 2024-09-12T13:13:56.250 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ size=10G 2024-09-12T13:13:56.250 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:13:56.250 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 create --size 10G --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image-primary7 2024-09-12T13:13:56.317 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ is_pool_mirror_mode_image mirror 2024-09-12T13:13:56.317 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:13:56.318 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' image = image ']' 2024-09-12T13:13:56.318 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:13:56.318 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ enable_mirror cluster1 mirror image-primary7 snapshot 2024-09-12T13:13:56.318 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:13:56.318 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:13:56.318 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:13:56.318 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mode=snapshot 2024-09-12T13:13:56.318 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 mirror image enable mirror/image-primary7 snapshot 2024-09-12T13:13:56.533 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local ret=0 2024-09-12T13:13:56.533 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ timeout 5m bash -c 'zcat test-primary6/linux-5.4.tar.gz | pv -L 256K | tar xf - -C test-primary6' 2024-09-12T13:13:56.871 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:13:56.871 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:13:56.871 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:13:56.871 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:13:56.871 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:13:56.871 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:13:56.874 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Mirroring enabled 2024-09-12T13:13:56.876 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 676 2024-09-12T13:13:56.881 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 675 2024-09-12T13:13:56.889 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:13:56.891 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 info mirror/image-primary7 2024-09-12T13:13:56.896 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:13:56.952 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:rbd image 'image-primary7': 2024-09-12T13:13:56.952 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: size 10 GiB in 2560 objects 2024-09-12T13:13:56.952 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: order 22 (4 MiB objects) 2024-09-12T13:13:56.952 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: snapshot_count: 1 2024-09-12T13:13:56.952 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: id: 1acfee7a7741 2024-09-12T13:13:56.952 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: block_name_prefix: rbd_data.1acfee7a7741 2024-09-12T13:13:56.952 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: format: 2 2024-09-12T13:13:56.952 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: features: layering, exclusive-lock, object-map, fast-diff 2024-09-12T13:13:56.952 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: op_features: 2024-09-12T13:13:56.952 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: flags: 2024-09-12T13:13:56.953 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: create_timestamp: Thu Sep 12 13:13:56 2024 2024-09-12T13:13:56.953 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: access_timestamp: Thu Sep 12 13:13:56 2024 2024-09-12T13:13:56.953 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: modify_timestamp: Thu Sep 12 13:13:56 2024 2024-09-12T13:13:56.953 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring state: enabled 2024-09-12T13:13:56.953 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring mode: snapshot 2024-09-12T13:13:56.953 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring global id: 3f634cfd-f07a-4271-8c29-0ad3777a474c 2024-09-12T13:13:56.953 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring primary: true 2024-09-12T13:13:56.957 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \n\b\d ]] 2024-09-12T13:13:56.957 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \k\r\b\d ]] 2024-09-12T13:13:56.957 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo rbd --cluster cluster1 device map -t krbd mirror/image-primary7 2024-09-12T13:13:57.078 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEV=/dev/rbd6 2024-09-12T13:13:57.078 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEVS+=($DEV) 2024-09-12T13:13:57.078 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo mkfs.ext4 /dev/rbd6 2024-09-12T13:13:57.098 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:mke2fs 1.46.5 (30-Dec-2021) 2024-09-12T13:13:57.109 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Discarding device blocks: 0/2621440 done 2024-09-12T13:13:57.110 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Creating filesystem with 2621440 4k blocks and 655360 inodes 2024-09-12T13:13:57.110 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Filesystem UUID: 966b70e7-1556-4f8c-9826-c68a2088d6dc 2024-09-12T13:13:57.110 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Superblock backups stored on blocks: 2024-09-12T13:13:57.110 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 2024-09-12T13:13:57.110 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 2024-09-12T13:13:57.110 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Allocating group tables: 0/80 done 2024-09-12T13:13:57.110 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Writing inode tables: 0/80 done 2024-09-12T13:13:57.115 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Creating journal (16384 blocks): done 2024-09-12T13:13:57.152 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Writing superblocks and filesystem accounting information: 0/80 done 2024-09-12T13:13:57.153 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 2024-09-12T13:13:57.155 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mkdir test-primary7 2024-09-12T13:13:57.158 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo mount /dev/rbd6 test-primary7 2024-09-12T13:13:57.211 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ whoami 2024-09-12T13:13:57.213 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo chown ubuntu test-primary7 2024-09-12T13:13:57.239 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ SNAP_PIDS+=($!) 2024-09-12T13:13:57.239 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ take_mirror_snapshots cluster1 mirror image-primary7 2024-09-12T13:13:57.239 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:13:57.239 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:13:57.239 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:13:57.239 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:13:57.239 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:13:57.239 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:13:57.239 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:13:57.239 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:13:57.239 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:13:57.240 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ WORKLOAD_PIDS+=($!) 2024-09-12T13:13:57.240 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:13:57.240 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ IMG=image-primary8 2024-09-12T13:13:57.240 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ MNTPT=test-primary8 2024-09-12T13:13:57.240 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ slow_untar_workload test-primary7 2024-09-12T13:13:57.240 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ create_image_and_enable_mirror cluster1 mirror image-primary8 snapshot 10G 2024-09-12T13:13:57.240 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:13:57.240 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:13:57.240 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:13:57.240 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:13:57.240 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:13:57.240 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:13:57.240 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mode=snapshot 2024-09-12T13:13:57.240 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n snapshot ']' 2024-09-12T13:13:57.241 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:13:57.241 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ create_image cluster1 mirror image-primary8 10G 2024-09-12T13:13:57.241 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:13:57.241 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:13:57.241 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:13:57.241 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:13:57.241 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:13:57.241 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mountpt=test-primary7 2024-09-12T13:13:57.241 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:13:57.241 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local size=128 2024-09-12T13:13:57.241 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n 10G ']' 2024-09-12T13:13:57.241 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ size=10G 2024-09-12T13:13:57.241 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:13:57.241 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ cp linux-5.4.tar.gz test-primary7 2024-09-12T13:13:57.242 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 create --size 10G --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image-primary8 2024-09-12T13:13:57.356 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ is_pool_mirror_mode_image mirror 2024-09-12T13:13:57.357 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:13:57.357 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' image = image ']' 2024-09-12T13:13:57.357 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:13:57.357 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ enable_mirror cluster1 mirror image-primary8 snapshot 2024-09-12T13:13:57.357 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:13:57.357 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:13:57.357 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:13:57.357 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mode=snapshot 2024-09-12T13:13:57.357 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 mirror image enable mirror/image-primary8 snapshot 2024-09-12T13:13:57.536 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local ret=0 2024-09-12T13:13:57.536 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ timeout 5m bash -c 'zcat test-primary7/linux-5.4.tar.gz | pv -L 256K | tar xf - -C test-primary7' 2024-09-12T13:13:57.866 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 679 2024-09-12T13:13:57.868 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Mirroring enabled 2024-09-12T13:13:57.873 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 678 2024-09-12T13:13:57.878 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:13:57.878 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:13:57.878 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:13:57.879 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:13:57.879 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:13:57.879 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:13:57.885 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:13:57.892 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:13:57.899 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 info mirror/image-primary8 2024-09-12T13:13:57.969 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:rbd image 'image-primary8': 2024-09-12T13:13:57.969 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: size 10 GiB in 2560 objects 2024-09-12T13:13:57.969 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: order 22 (4 MiB objects) 2024-09-12T13:13:57.969 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: snapshot_count: 1 2024-09-12T13:13:57.969 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: id: 1ae09ae82d81 2024-09-12T13:13:57.969 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: block_name_prefix: rbd_data.1ae09ae82d81 2024-09-12T13:13:57.970 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: format: 2 2024-09-12T13:13:57.970 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: features: layering, exclusive-lock, object-map, fast-diff 2024-09-12T13:13:57.970 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: op_features: 2024-09-12T13:13:57.970 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: flags: 2024-09-12T13:13:57.970 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: create_timestamp: Thu Sep 12 13:13:57 2024 2024-09-12T13:13:57.970 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: access_timestamp: Thu Sep 12 13:13:57 2024 2024-09-12T13:13:57.970 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: modify_timestamp: Thu Sep 12 13:13:57 2024 2024-09-12T13:13:57.970 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring state: enabled 2024-09-12T13:13:57.970 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring mode: snapshot 2024-09-12T13:13:57.970 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring global id: 27e6f3f9-b993-4ad3-a6a6-74e8fc4feff4 2024-09-12T13:13:57.970 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring primary: true 2024-09-12T13:13:57.974 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \n\b\d ]] 2024-09-12T13:13:57.974 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \k\r\b\d ]] 2024-09-12T13:13:57.975 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo rbd --cluster cluster1 device map -t krbd mirror/image-primary8 2024-09-12T13:13:58.094 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEV=/dev/rbd7 2024-09-12T13:13:58.094 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEVS+=($DEV) 2024-09-12T13:13:58.094 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo mkfs.ext4 /dev/rbd7 2024-09-12T13:13:58.115 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:mke2fs 1.46.5 (30-Dec-2021) 2024-09-12T13:13:58.129 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Discarding device blocks: 0/2621440 done 2024-09-12T13:13:58.129 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Creating filesystem with 2621440 4k blocks and 655360 inodes 2024-09-12T13:13:58.129 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Filesystem UUID: 07ea314f-bae4-4808-a87e-89b7309af29d 2024-09-12T13:13:58.130 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Superblock backups stored on blocks: 2024-09-12T13:13:58.130 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 2024-09-12T13:13:58.130 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 2024-09-12T13:13:58.130 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Allocating group tables: 0/80 done 2024-09-12T13:13:58.130 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Writing inode tables: 0/80 done 2024-09-12T13:13:58.133 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Creating journal (16384 blocks): done 2024-09-12T13:13:58.175 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Writing superblocks and filesystem accounting information: 0/80 done 2024-09-12T13:13:58.175 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 2024-09-12T13:13:58.178 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mkdir test-primary8 2024-09-12T13:13:58.180 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo mount /dev/rbd7 test-primary8 2024-09-12T13:13:58.234 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ whoami 2024-09-12T13:13:58.236 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo chown ubuntu test-primary8 2024-09-12T13:13:58.262 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ SNAP_PIDS+=($!) 2024-09-12T13:13:58.262 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ take_mirror_snapshots cluster1 mirror image-primary8 2024-09-12T13:13:58.262 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:13:58.262 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:13:58.262 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:13:58.263 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:13:58.263 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:13:58.263 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:13:58.263 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:13:58.263 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:13:58.263 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ WORKLOAD_PIDS+=($!) 2024-09-12T13:13:58.263 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:13:58.263 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:13:58.263 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ IMG=image-primary9 2024-09-12T13:13:58.263 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ MNTPT=test-primary9 2024-09-12T13:13:58.263 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ create_image_and_enable_mirror cluster1 mirror image-primary9 snapshot 10G 2024-09-12T13:13:58.264 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ slow_untar_workload test-primary8 2024-09-12T13:13:58.264 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:13:58.264 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:13:58.264 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:13:58.264 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:13:58.264 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:13:58.264 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:13:58.265 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mode=snapshot 2024-09-12T13:13:58.265 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n snapshot ']' 2024-09-12T13:13:58.265 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:13:58.265 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ create_image cluster1 mirror image-primary9 10G 2024-09-12T13:13:58.265 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:13:58.265 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:13:58.265 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:13:58.266 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:13:58.266 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:13:58.266 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:13:58.266 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local size=128 2024-09-12T13:13:58.266 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n 10G ']' 2024-09-12T13:13:58.266 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ size=10G 2024-09-12T13:13:58.266 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:13:58.267 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 create --size 10G --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image-primary9 2024-09-12T13:13:58.267 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mountpt=test-primary8 2024-09-12T13:13:58.267 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ cp linux-5.4.tar.gz test-primary8 2024-09-12T13:13:58.392 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ is_pool_mirror_mode_image mirror 2024-09-12T13:13:58.392 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:13:58.392 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' image = image ']' 2024-09-12T13:13:58.392 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:13:58.392 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ enable_mirror cluster1 mirror image-primary9 snapshot 2024-09-12T13:13:58.392 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:13:58.392 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:13:58.392 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:13:58.392 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mode=snapshot 2024-09-12T13:13:58.393 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 mirror image enable mirror/image-primary9 snapshot 2024-09-12T13:13:58.554 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local ret=0 2024-09-12T13:13:58.554 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ timeout 5m bash -c 'zcat test-primary8/linux-5.4.tar.gz | pv -L 256K | tar xf - -C test-primary8' 2024-09-12T13:13:58.886 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Mirroring enabled 2024-09-12T13:13:58.888 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 681 2024-09-12T13:13:58.889 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:13:58.889 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:13:58.889 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:13:58.889 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:13:58.889 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:13:58.890 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:13:58.892 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:13:58.893 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:13:58.893 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:13:58.893 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:13:58.893 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:13:58.893 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:13:58.896 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 682 2024-09-12T13:13:58.899 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:13:58.901 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 info mirror/image-primary9 2024-09-12T13:13:58.909 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:13:58.977 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:rbd image 'image-primary9': 2024-09-12T13:13:58.977 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: size 10 GiB in 2560 objects 2024-09-12T13:13:58.977 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: order 22 (4 MiB objects) 2024-09-12T13:13:58.977 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: snapshot_count: 1 2024-09-12T13:13:58.977 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: id: 1af0c6cf26ea 2024-09-12T13:13:58.978 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: block_name_prefix: rbd_data.1af0c6cf26ea 2024-09-12T13:13:58.978 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: format: 2 2024-09-12T13:13:58.978 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: features: layering, exclusive-lock, object-map, fast-diff 2024-09-12T13:13:58.978 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: op_features: 2024-09-12T13:13:58.978 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: flags: 2024-09-12T13:13:58.978 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: create_timestamp: Thu Sep 12 13:13:58 2024 2024-09-12T13:13:58.978 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: access_timestamp: Thu Sep 12 13:13:58 2024 2024-09-12T13:13:58.978 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: modify_timestamp: Thu Sep 12 13:13:58 2024 2024-09-12T13:13:58.978 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring state: enabled 2024-09-12T13:13:58.978 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring mode: snapshot 2024-09-12T13:13:58.978 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring global id: b8a82bf7-89cf-4fad-b167-0553ebb15a5e 2024-09-12T13:13:58.978 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring primary: true 2024-09-12T13:13:58.982 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \n\b\d ]] 2024-09-12T13:13:58.982 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \k\r\b\d ]] 2024-09-12T13:13:58.983 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo rbd --cluster cluster1 device map -t krbd mirror/image-primary9 2024-09-12T13:13:59.104 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEV=/dev/rbd8 2024-09-12T13:13:59.104 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEVS+=($DEV) 2024-09-12T13:13:59.104 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo mkfs.ext4 /dev/rbd8 2024-09-12T13:13:59.125 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:mke2fs 1.46.5 (30-Dec-2021) 2024-09-12T13:13:59.140 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Discarding device blocks: 0/2621440 done 2024-09-12T13:13:59.140 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Creating filesystem with 2621440 4k blocks and 655360 inodes 2024-09-12T13:13:59.140 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Filesystem UUID: 742db221-1a74-40ac-acfb-83bee0d617b0 2024-09-12T13:13:59.140 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Superblock backups stored on blocks: 2024-09-12T13:13:59.140 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 2024-09-12T13:13:59.140 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 2024-09-12T13:13:59.140 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Allocating group tables: 0/80 done 2024-09-12T13:13:59.140 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Writing inode tables: 0/80 done 2024-09-12T13:13:59.144 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Creating journal (16384 blocks): done 2024-09-12T13:13:59.184 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Writing superblocks and filesystem accounting information: 0/80 done 2024-09-12T13:13:59.184 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 2024-09-12T13:13:59.186 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mkdir test-primary9 2024-09-12T13:13:59.189 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo mount /dev/rbd8 test-primary9 2024-09-12T13:13:59.243 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ whoami 2024-09-12T13:13:59.245 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo chown ubuntu test-primary9 2024-09-12T13:13:59.270 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ SNAP_PIDS+=($!) 2024-09-12T13:13:59.270 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ take_mirror_snapshots cluster1 mirror image-primary9 2024-09-12T13:13:59.270 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:13:59.270 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:13:59.271 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:13:59.271 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:13:59.271 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:13:59.271 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:13:59.271 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:13:59.271 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:13:59.271 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:13:59.271 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ WORKLOAD_PIDS+=($!) 2024-09-12T13:13:59.271 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:13:59.271 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ slow_untar_workload test-primary9 2024-09-12T13:13:59.271 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ IMG=image-primary10 2024-09-12T13:13:59.271 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ MNTPT=test-primary10 2024-09-12T13:13:59.271 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ create_image_and_enable_mirror cluster1 mirror image-primary10 snapshot 10G 2024-09-12T13:13:59.272 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mountpt=test-primary9 2024-09-12T13:13:59.272 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:13:59.272 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:13:59.272 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ cp linux-5.4.tar.gz test-primary9 2024-09-12T13:13:59.272 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:13:59.272 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:13:59.272 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:13:59.272 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:13:59.272 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mode=snapshot 2024-09-12T13:13:59.272 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n snapshot ']' 2024-09-12T13:13:59.272 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:13:59.273 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ create_image cluster1 mirror image-primary10 10G 2024-09-12T13:13:59.273 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:13:59.273 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:13:59.273 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:13:59.273 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:13:59.273 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:13:59.273 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:13:59.273 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local size=128 2024-09-12T13:13:59.273 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n 10G ']' 2024-09-12T13:13:59.273 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ size=10G 2024-09-12T13:13:59.273 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:13:59.273 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 create --size 10G --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image-primary10 2024-09-12T13:13:59.381 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ is_pool_mirror_mode_image mirror 2024-09-12T13:13:59.381 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:13:59.381 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' image = image ']' 2024-09-12T13:13:59.381 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:13:59.381 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ enable_mirror cluster1 mirror image-primary10 snapshot 2024-09-12T13:13:59.382 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:13:59.382 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:13:59.382 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:13:59.382 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mode=snapshot 2024-09-12T13:13:59.382 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 mirror image enable mirror/image-primary10 snapshot 2024-09-12T13:13:59.571 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local ret=0 2024-09-12T13:13:59.571 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ timeout 5m bash -c 'zcat test-primary9/linux-5.4.tar.gz | pv -L 256K | tar xf - -C test-primary9' 2024-09-12T13:13:59.892 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:13:59.893 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:13:59.893 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:13:59.893 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:13:59.893 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:13:59.893 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:13:59.898 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Mirroring enabled 2024-09-12T13:13:59.899 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:13:59.899 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:13:59.899 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:13:59.899 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:13:59.899 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:13:59.899 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:13:59.905 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 684 2024-09-12T13:13:59.914 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 686 2024-09-12T13:13:59.930 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 info mirror/image-primary10 2024-09-12T13:13:59.933 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:13:59.933 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:13:59.938 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 685 2024-09-12T13:13:59.954 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:14:00.015 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:rbd image 'image-primary10': 2024-09-12T13:14:00.015 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: size 10 GiB in 2560 objects 2024-09-12T13:14:00.015 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: order 22 (4 MiB objects) 2024-09-12T13:14:00.015 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: snapshot_count: 1 2024-09-12T13:14:00.015 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: id: 1b05c0cecee7 2024-09-12T13:14:00.015 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: block_name_prefix: rbd_data.1b05c0cecee7 2024-09-12T13:14:00.016 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: format: 2 2024-09-12T13:14:00.016 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: features: layering, exclusive-lock, object-map, fast-diff 2024-09-12T13:14:00.016 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: op_features: 2024-09-12T13:14:00.016 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: flags: 2024-09-12T13:14:00.016 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: create_timestamp: Thu Sep 12 13:13:59 2024 2024-09-12T13:14:00.016 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: access_timestamp: Thu Sep 12 13:13:59 2024 2024-09-12T13:14:00.016 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: modify_timestamp: Thu Sep 12 13:13:59 2024 2024-09-12T13:14:00.016 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring state: enabled 2024-09-12T13:14:00.016 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring mode: snapshot 2024-09-12T13:14:00.016 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring global id: a41c1371-bc6b-4e5a-b8a6-2d2252dfda67 2024-09-12T13:14:00.017 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring primary: true 2024-09-12T13:14:00.020 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \n\b\d ]] 2024-09-12T13:14:00.020 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \k\r\b\d ]] 2024-09-12T13:14:00.021 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo rbd --cluster cluster1 device map -t krbd mirror/image-primary10 2024-09-12T13:14:00.164 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEV=/dev/rbd9 2024-09-12T13:14:00.165 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEVS+=($DEV) 2024-09-12T13:14:00.165 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo mkfs.ext4 /dev/rbd9 2024-09-12T13:14:00.188 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:mke2fs 1.46.5 (30-Dec-2021) 2024-09-12T13:14:00.197 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Discarding device blocks: 0/2621440 done 2024-09-12T13:14:00.197 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Creating filesystem with 2621440 4k blocks and 655360 inodes 2024-09-12T13:14:00.198 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Filesystem UUID: eae89a0b-65d2-45ca-aa0c-25632b258ab5 2024-09-12T13:14:00.198 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Superblock backups stored on blocks: 2024-09-12T13:14:00.198 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 2024-09-12T13:14:00.198 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 2024-09-12T13:14:00.198 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Allocating group tables: 0/80 done 2024-09-12T13:14:00.198 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Writing inode tables: 0/80 done 2024-09-12T13:14:00.203 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Creating journal (16384 blocks): done 2024-09-12T13:14:00.270 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Writing superblocks and filesystem accounting information: 0/80 done 2024-09-12T13:14:00.270 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 2024-09-12T13:14:00.272 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mkdir test-primary10 2024-09-12T13:14:00.275 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo mount /dev/rbd9 test-primary10 2024-09-12T13:14:00.388 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ whoami 2024-09-12T13:14:00.392 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo chown ubuntu test-primary10 2024-09-12T13:14:00.416 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ SNAP_PIDS+=($!) 2024-09-12T13:14:00.416 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ take_mirror_snapshots cluster1 mirror image-primary10 2024-09-12T13:14:00.416 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:14:00.416 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:14:00.416 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:14:00.416 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:14:00.417 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ WORKLOAD_PIDS+=($!) 2024-09-12T13:14:00.417 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:14:00.417 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:14:00.417 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:14:00.417 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:14:00.417 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:14:00.417 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${SNAP_PIDS[@]} 2024-09-12T13:14:00.417 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 59054 2024-09-12T13:14:00.417 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ slow_untar_workload test-primary10 2024-09-12T13:14:00.418 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mountpt=test-primary10 2024-09-12T13:14:00.418 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ cp linux-5.4.tar.gz test-primary10 2024-09-12T13:14:00.735 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local ret=0 2024-09-12T13:14:00.735 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ timeout 5m bash -c 'zcat test-primary10/linux-5.4.tar.gz | pv -L 256K | tar xf - -C test-primary10' 2024-09-12T13:14:00.890 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:14:00.890 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:14:00.890 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:14:00.890 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:14:00.890 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:14:00.890 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:14:00.895 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:14:00.896 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 689 2024-09-12T13:14:00.897 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:14:00.897 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:14:00.897 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:14:00.897 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:14:00.897 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:14:00.930 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 690 2024-09-12T13:14:00.933 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:14:00.936 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 688 2024-09-12T13:14:00.948 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:14:00.952 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:14:01.895 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 691 2024-09-12T13:14:01.903 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 692 2024-09-12T13:14:01.903 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:14:01.903 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:14:01.903 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:14:01.903 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:14:01.903 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:14:01.904 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:14:01.914 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:14:01.919 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:14:01.919 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:14:01.919 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:14:01.919 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:14:01.919 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:14:01.919 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:14:01.933 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:14:02.904 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 694 2024-09-12T13:14:02.923 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:14:02.933 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 693 2024-09-12T13:14:02.937 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:14:02.937 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:14:02.937 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:14:02.937 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:14:02.937 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:14:02.937 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:14:02.939 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:14:02.939 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:14:02.940 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:14:02.940 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:14:02.940 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:14:02.940 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:14:02.958 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:14:02.962 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:14:02.962 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:14:02.963 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:14:02.963 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:14:02.963 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:14:02.963 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:14:03.912 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 697 2024-09-12T13:14:03.916 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 696 2024-09-12T13:14:03.924 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 695 2024-09-12T13:14:03.939 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:14:03.939 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:14:03.939 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:14:03.939 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:14:03.939 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:14:03.939 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:14:03.943 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:14:03.945 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:14:03.953 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:14:03.953 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:14:03.953 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:14:03.953 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:14:03.954 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:14:03.954 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:14:03.960 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:14:03.960 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:14:03.960 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:14:03.960 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:14:03.960 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:14:03.961 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:14:03.963 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:14:04.917 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:14:04.917 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:14:04.917 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:14:04.917 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:14:04.917 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:14:04.917 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:14:04.937 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:14:04.937 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:14:04.938 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:14:04.938 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:14:04.938 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:14:04.938 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:14:04.945 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 698 2024-09-12T13:14:04.951 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 700 2024-09-12T13:14:04.988 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 699 2024-09-12T13:14:04.988 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:14:05.014 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:14:05.038 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:14:05.928 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:14:05.928 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:14:05.929 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:14:05.929 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:14:05.929 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:14:05.929 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:14:05.958 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 702 2024-09-12T13:14:05.965 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:14:05.965 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:14:05.966 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:14:05.966 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:14:05.966 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:14:05.966 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:14:05.973 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 701 2024-09-12T13:14:05.995 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:14:06.005 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:14:06.947 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:14:06.948 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:14:06.948 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:14:06.948 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:14:06.948 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:14:06.948 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:14:06.949 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:14:06.949 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:14:06.949 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:14:06.949 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:14:06.949 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:14:06.949 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:14:06.952 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 704 2024-09-12T13:14:06.968 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 703 2024-09-12T13:14:06.974 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:14:06.974 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:14:06.974 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:14:06.974 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:14:06.975 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:14:06.975 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:14:07.004 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:14:07.019 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:14:07.972 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 706 2024-09-12T13:14:07.997 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:14:07.997 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:14:07.997 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:14:07.997 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:14:07.997 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:14:07.997 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:14:08.012 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 705 2024-09-12T13:14:08.021 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:14:08.021 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:14:08.021 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:14:08.021 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:14:08.021 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:14:08.021 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:14:08.024 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:14:08.042 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:14:08.043 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:14:08.046 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:14:08.046 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:14:08.046 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:14:08.046 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:14:08.070 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:14:08.075 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 707 2024-09-12T13:14:08.148 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:14:08.999 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:14:09.000 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:14:09.000 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:14:09.000 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:14:09.000 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:14:09.000 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:14:09.012 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:14:09.012 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:14:09.013 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:14:09.013 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:14:09.014 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:14:09.014 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:14:09.025 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 709 2024-09-12T13:14:09.080 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 710 2024-09-12T13:14:09.097 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 708 2024-09-12T13:14:09.107 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:14:09.131 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:14:09.178 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:14:09.954 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 711 2024-09-12T13:14:09.974 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 712 2024-09-12T13:14:09.978 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:14:10.009 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:14:10.016 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:14:10.016 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:14:10.017 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:14:10.017 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:14:10.017 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:14:10.017 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:14:10.027 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:14:10.027 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:14:10.027 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:14:10.027 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:14:10.028 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:14:10.028 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:14:10.964 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 713 2024-09-12T13:14:10.969 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 714 2024-09-12T13:14:10.995 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:14:10.995 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:14:11.031 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:14:11.032 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:14:11.032 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:14:11.032 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:14:11.032 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:14:11.032 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:14:11.074 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:14:11.074 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:14:11.074 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:14:11.074 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:14:11.074 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:14:11.075 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:14:11.151 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:14:11.151 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:14:11.152 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:14:11.152 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:14:11.152 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:14:11.152 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:14:11.956 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 716 2024-09-12T13:14:11.973 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:14:11.974 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 715 2024-09-12T13:14:11.996 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:14:12.110 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:14:12.110 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:14:12.111 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:14:12.111 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:14:12.111 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:14:12.111 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:14:12.135 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:14:12.135 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:14:12.135 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:14:12.135 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:14:12.135 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:14:12.135 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:14:12.181 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:14:12.181 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:14:12.181 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:14:12.181 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:14:12.181 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:14:12.181 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:14:12.982 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:14:12.983 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:14:12.983 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:14:12.983 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:14:12.983 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:14:12.983 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:14:12.989 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 720 2024-09-12T13:14:13.005 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 718 2024-09-12T13:14:13.010 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:14:13.013 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:14:13.013 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:14:13.013 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:14:13.013 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:14:13.013 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:14:13.013 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:14:13.026 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:14:13.953 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 717 2024-09-12T13:14:13.993 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 719 2024-09-12T13:14:13.997 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:14:13.999 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:14:13.999 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:14:14.000 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:14:14.000 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:14:14.000 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:14:14.000 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:14:14.000 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:14:14.000 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:14:14.000 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:14:14.000 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:14:14.000 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:14:14.000 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:14:14.023 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:14:14.030 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 721 2024-09-12T13:14:14.050 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:14:14.975 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:14:14.975 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:14:14.976 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:14:14.976 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:14:14.976 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:14:14.976 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:14:14.999 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:14:14.999 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:14:14.999 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:14:14.999 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:14:14.999 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:14:14.999 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:14:15.009 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 724 2024-09-12T13:14:15.029 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:14:15.941 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 722 2024-09-12T13:14:15.973 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:14:16.014 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:14:16.014 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:14:16.014 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:14:16.014 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:14:16.014 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:14:16.014 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:14:16.029 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:14:16.029 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:14:16.029 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:14:16.029 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:14:16.029 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:14:16.029 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:14:16.051 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 725 2024-09-12T13:14:16.077 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:14:16.951 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 723 2024-09-12T13:14:16.993 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:14:17.000 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:14:17.000 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:14:17.000 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:14:17.000 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:14:17.000 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:14:17.000 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:14:17.028 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:14:17.028 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:14:17.028 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:14:17.028 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:14:17.028 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:14:17.028 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:14:17.050 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 727 2024-09-12T13:14:17.052 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:14:17.052 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:14:17.053 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:14:17.053 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:14:17.053 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:14:17.053 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:14:17.101 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:14:17.834 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 726 2024-09-12T13:14:17.881 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:14:17.883 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 728 2024-09-12T13:14:17.899 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:14:17.913 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 729 2024-09-12T13:14:17.939 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:14:18.032 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:14:18.032 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:14:18.032 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:14:18.032 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:14:18.032 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:14:18.032 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:14:18.872 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 730 2024-09-12T13:14:18.886 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 731 2024-09-12T13:14:18.903 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:14:18.906 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:14:18.977 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:14:18.977 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:14:18.977 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:14:18.977 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:14:18.977 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:14:18.977 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:14:19.080 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:14:19.080 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:14:19.080 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:14:19.081 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:14:19.081 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:14:19.081 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:14:19.880 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 733 2024-09-12T13:14:19.901 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:14:19.995 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:14:19.995 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:14:19.996 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:14:19.996 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:14:19.996 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:14:19.996 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:14:20.104 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:14:20.104 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:14:20.104 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:14:20.105 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:14:20.105 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:14:20.105 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:14:20.827 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 732 2024-09-12T13:14:20.869 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:14:20.886 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:14:20.886 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:14:20.886 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:14:20.886 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:14:20.886 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:14:20.886 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:14:20.902 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 734 2024-09-12T13:14:20.903 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:14:20.903 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:14:20.903 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:14:20.903 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:14:20.903 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:14:20.903 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:14:20.914 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 735 2024-09-12T13:14:20.916 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:14:20.935 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:14:20.942 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:14:20.942 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:14:20.942 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:14:20.942 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:14:20.942 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:14:20.942 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:14:21.899 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 736 2024-09-12T13:14:21.909 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:14:21.909 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:14:21.909 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:14:21.909 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:14:21.909 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:14:21.909 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:14:21.910 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:14:21.910 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:14:21.910 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:14:21.910 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:14:21.910 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:14:21.910 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:14:21.912 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:14:21.914 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 737 2024-09-12T13:14:21.946 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:14:22.883 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 738 2024-09-12T13:14:22.900 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 739 2024-09-12T13:14:22.904 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:14:22.904 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:14:22.905 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:14:22.905 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:14:22.905 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:14:22.905 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:14:22.912 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:14:22.919 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:14:23.871 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:14:23.871 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:14:23.871 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:14:23.871 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:14:23.871 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:14:23.872 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:14:23.892 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 741 2024-09-12T13:14:23.919 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:14:23.919 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:14:23.919 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:14:23.919 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:14:23.919 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:14:23.920 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:14:23.935 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 740 2024-09-12T13:14:23.941 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:14:23.942 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:14:23.942 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:14:23.942 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:14:23.942 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:14:23.942 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:14:23.949 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:14:23.956 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:14:24.915 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:14:24.915 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:14:24.915 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:14:24.915 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:14:24.915 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:14:24.915 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:14:24.943 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 745 2024-09-12T13:14:24.949 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:14:24.949 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:14:24.949 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:14:24.949 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:14:24.949 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:14:24.950 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:14:24.994 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:14:25.868 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 742 2024-09-12T13:14:25.916 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:14:25.916 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:14:25.916 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:14:25.916 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:14:25.916 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:14:25.916 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:14:25.922 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:14:25.923 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:14:25.923 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:14:25.923 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:14:25.923 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:14:25.923 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:14:25.941 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:14:26.884 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 744 2024-09-12T13:14:26.884 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 743 2024-09-12T13:14:26.919 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:14:26.931 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:14:26.950 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:14:26.950 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:14:26.950 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:14:26.950 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:14:26.950 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:14:26.951 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:14:26.959 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:14:26.960 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:14:26.960 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:14:26.960 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:14:26.960 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:14:26.960 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:14:27.841 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 746 2024-09-12T13:14:27.847 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 747 2024-09-12T13:14:27.884 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:14:27.888 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:14:27.997 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:14:27.997 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:14:27.997 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:14:27.997 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:14:27.997 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:14:27.997 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:14:28.891 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 748 2024-09-12T13:14:28.928 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 749 2024-09-12T13:14:28.946 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:14:28.946 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:14:28.946 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:14:28.946 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:14:28.946 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:14:28.946 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:14:28.974 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 751 2024-09-12T13:14:28.989 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:14:28.997 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 750 2024-09-12T13:14:29.024 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:14:29.029 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:14:29.038 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:14:29.922 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:14:29.922 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:14:29.922 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:14:29.922 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:14:29.922 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:14:29.922 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:14:29.934 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:14:29.934 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:14:29.934 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:14:29.934 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:14:29.934 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:14:29.934 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:14:29.937 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 753 2024-09-12T13:14:29.946 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:14:30.881 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 752 2024-09-12T13:14:30.887 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:14:30.887 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:14:30.887 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:14:30.887 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:14:30.887 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:14:30.887 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:14:30.891 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:14:30.891 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:14:30.891 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:14:30.891 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:14:30.891 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:14:30.891 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:14:30.922 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:14:30.947 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 754 2024-09-12T13:14:30.955 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 755 2024-09-12T13:14:30.969 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:14:30.973 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:14:31.942 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 757 2024-09-12T13:14:31.962 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 756 2024-09-12T13:14:31.981 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:14:31.981 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:14:31.994 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:14:31.994 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:14:31.994 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:14:31.995 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:14:31.995 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:14:31.995 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:14:32.030 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:14:32.030 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:14:32.030 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:14:32.030 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:14:32.030 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:14:32.030 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:14:32.034 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:14:32.034 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:14:32.034 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:14:32.034 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:14:32.034 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:14:32.034 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:14:32.042 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:14:32.044 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:14:32.044 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:14:32.044 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:14:32.044 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:14:32.044 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:14:32.950 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:14:32.950 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:14:32.951 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:14:32.951 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:14:32.951 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:14:32.951 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:14:32.982 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 761 2024-09-12T13:14:33.014 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 758 2024-09-12T13:14:33.060 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:14:33.075 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:14:33.927 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:14:33.927 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:14:33.927 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:14:33.927 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:14:33.927 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:14:33.928 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:14:33.974 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:14:33.979 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:14:33.979 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:14:33.979 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:14:33.979 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:14:33.979 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:14:33.979 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:14:33.979 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:14:33.979 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:14:33.979 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:14:33.979 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:14:33.980 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:14:34.010 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 760 2024-09-12T13:14:34.018 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 759 2024-09-12T13:14:34.044 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:14:34.061 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:14:34.962 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 763 2024-09-12T13:14:34.981 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:14:34.985 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:14:34.986 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:14:34.986 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:14:34.986 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:14:34.986 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:14:34.986 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:14:34.986 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:14:34.986 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:14:34.986 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:14:34.986 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:14:34.986 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:14:34.986 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:14:35.917 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 762 2024-09-12T13:14:35.955 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:14:36.007 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:14:36.006+0000 7f67dc600640 -1 rbd::mirror::image_replayer::snapshot::Replayer: 0x5605568d1000 handle_unlink_peer: failed to unlink local peer from remote image: (22) Invalid argument 2024-09-12T13:14:36.068 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:14:36.068 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:14:36.068 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:14:36.068 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:14:36.068 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:14:36.068 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:14:36.078 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:14:36.078 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:14:36.078 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:14:36.079 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:14:36.079 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:14:36.079 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:14:36.920 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 764 2024-09-12T13:14:36.929 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 765 2024-09-12T13:14:36.964 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:14:36.971 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:14:37.008 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:14:37.008+0000 7f67dc600640 -1 rbd::mirror::image_replayer::snapshot::Replayer: 0x56055b84e400 handle_unlink_peer: failed to unlink local peer from remote image: (22) Invalid argument 2024-09-12T13:14:37.052 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:14:37.052 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:14:37.053 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:14:37.053 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:14:37.053 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:14:37.053 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:14:37.070 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:14:37.070 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:14:37.070 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:14:37.070 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:14:37.070 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:14:37.070 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:14:37.832 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 767 2024-09-12T13:14:37.837 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 766 2024-09-12T13:14:37.886 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 769 2024-09-12T13:14:37.889 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:14:37.891 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:14:37.925 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:14:37.939 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 768 2024-09-12T13:14:37.956 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:14:37.984 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:14:37.984 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:14:37.984 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:14:37.984 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:14:37.984 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:14:37.985 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:14:38.958 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:14:38.958 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:14:38.958 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:14:38.958 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:14:38.958 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:14:38.959 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:14:39.922 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 771 2024-09-12T13:14:39.928 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 770 2024-09-12T13:14:39.951 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:14:39.966 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:14:39.967 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:14:39.967 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:14:39.967 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:14:39.968 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:14:39.968 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:14:39.968 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:14:39.974 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:14:39.974 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:14:39.975 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:14:39.975 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:14:39.975 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:14:39.975 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:14:40.005 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 773 2024-09-12T13:14:40.026 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:14:40.894 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:14:40.894 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:14:40.894 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:14:40.894 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:14:40.894 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:14:40.895 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:14:40.895 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:14:40.895 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:14:40.895 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:14:40.895 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:14:40.895 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:14:40.895 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:14:40.913 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 772 2024-09-12T13:14:40.927 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:14:40.927 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:14:40.927 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:14:40.927 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:14:40.927 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:14:40.927 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:14:40.953 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:14:40.958 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:14:40.958 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:14:40.958 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:14:40.959 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:14:40.959 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:14:40.959 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:14:41.007 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 774 2024-09-12T13:14:41.040 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 775 2024-09-12T13:14:41.046 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:14:41.056 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:14:42.017 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 778 2024-09-12T13:14:42.031 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 777 2024-09-12T13:14:42.042 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:14:42.058 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:14:42.954 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:14:42.954 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:14:42.954 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:14:42.954 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:14:42.954 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:14:42.954 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:14:42.969 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:14:42.969 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:14:42.969 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:14:42.969 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:14:42.969 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:14:42.969 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:14:43.029 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:14:43.029 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:14:43.029 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:14:43.029 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:14:43.029 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:14:43.029 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:14:43.931 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 776 2024-09-12T13:14:43.936 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 779 2024-09-12T13:14:43.956 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:14:43.956 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:14:43.956 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:14:43.956 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:14:43.956 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:14:43.956 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:14:43.969 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:14:43.970 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:14:44.003 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 780 2024-09-12T13:14:44.008 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 781 2024-09-12T13:14:44.014 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:14:44.016 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:14:44.050 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:14:44.050 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:14:44.050 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:14:44.050 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:14:44.050 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:14:44.050 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:14:44.059 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:14:44.059 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:14:44.059 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:14:44.059 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:14:44.059 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:14:44.059 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:14:44.981 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 783 2024-09-12T13:14:44.992 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 784 2024-09-12T13:14:45.001 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:14:45.006 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:14:45.045 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:14:45.045 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:14:45.045 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:14:45.045 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:14:45.045 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:14:45.045 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:14:45.061 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:14:45.061 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:14:45.061 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:14:45.061 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:14:45.061 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:14:45.061 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:14:45.943 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 782 2024-09-12T13:14:45.971 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:14:46.002 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 787 2024-09-12T13:14:46.010 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:14:46.941 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 785 2024-09-12T13:14:46.972 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:14:46.973 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:14:46.973 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:14:46.973 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:14:46.973 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:14:46.973 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:14:46.974 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:14:46.974 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:14:46.974 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:14:46.974 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:14:46.974 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:14:46.974 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:14:46.985 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:14:47.017 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:14:47.017 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:14:47.017 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:14:47.017 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:14:47.017 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:14:47.018 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:14:47.018 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:14:47.018 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:14:47.018 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:14:47.019 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:14:47.019 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:14:47.019 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:14:47.841 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 786 2024-09-12T13:14:47.870 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:14:47.896 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 789 2024-09-12T13:14:47.911 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:14:47.912 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 788 2024-09-12T13:14:47.924 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:14:48.004 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:14:48.004 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:14:48.004 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:14:48.005 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:14:48.005 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:14:48.005 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:14:48.008 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:14:48.009 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:14:48.009 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:14:48.009 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:14:48.009 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:14:48.009 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:14:48.884 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 791 2024-09-12T13:14:48.894 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 790 2024-09-12T13:14:48.900 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 792 2024-09-12T13:14:48.916 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:14:48.917 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:14:48.924 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:14:48.973 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:14:48.973 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:14:48.973 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:14:48.974 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:14:48.974 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:14:48.974 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:14:49.012 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:14:49.012 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:14:49.012 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:14:49.012 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:14:49.012 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:14:49.013 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:14:49.988 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:14:49.988 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:14:49.988 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:14:49.988 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:14:49.988 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:14:49.988 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:14:49.994 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 794 2024-09-12T13:14:50.008 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:14:50.875 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:14:50.875 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:14:50.875 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:14:50.875 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:14:50.875 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:14:50.875 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:14:50.914 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:14:50.915 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:14:50.915 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:14:50.915 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:14:50.915 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:14:50.915 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:14:50.927 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:14:50.927 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:14:50.927 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:14:50.927 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:14:50.927 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:14:50.927 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:14:50.960 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 793 2024-09-12T13:14:50.994 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:14:50.995 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 795 2024-09-12T13:14:51.005 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:14:51.027 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 796 2024-09-12T13:14:51.042 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:14:51.919 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:14:51.920 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:14:51.920 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:14:51.920 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:14:51.920 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:14:51.920 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:14:51.920 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:14:51.921 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:14:51.921 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:14:51.921 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:14:51.921 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:14:51.921 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:14:51.928 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:14:51.928 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:14:51.928 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:14:51.928 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:14:51.928 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:14:51.928 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:14:51.995 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 797 2024-09-12T13:14:52.005 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:14:53.011 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:14:53.011 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:14:53.011 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:14:53.011 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:14:53.011 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:14:53.012 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:14:53.986 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 799 2024-09-12T13:14:53.996 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 798 2024-09-12T13:14:53.998 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:14:53.998 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:14:53.998 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:14:53.998 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:14:53.998 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:14:53.998 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:14:54.010 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:14:54.010 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:14:54.010 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:14:54.010 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:14:54.011 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:14:54.011 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:14:54.035 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 800 2024-09-12T13:14:54.047 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:14:54.047 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:14:54.047 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:14:54.047 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:14:54.048 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:14:54.048 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:14:54.048 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:14:54.053 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:14:54.068 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 801 2024-09-12T13:14:54.074 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:14:54.089 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:14:55.008 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:14:55.009 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:14:55.009 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:14:55.009 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:14:55.009 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:14:55.009 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:14:55.043 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 805 2024-09-12T13:14:55.052 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:14:55.970 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 803 2024-09-12T13:14:56.009 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:14:56.989 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 802 2024-09-12T13:14:56.992 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 806 2024-09-12T13:14:57.024 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:14:57.033 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:14:57.054 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:14:57.054 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:14:57.054 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:14:57.055 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:14:57.055 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:14:57.055 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:14:57.056 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:14:57.056 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:14:57.056 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:14:57.056 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:14:57.056 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:14:57.056 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:14:57.077 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:14:57.077 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:14:57.077 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:14:57.077 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:14:57.078 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:14:57.078 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:14:57.093 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:14:57.093 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:14:57.093 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:14:57.093 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:14:57.093 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:14:57.093 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:14:57.852 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 807 2024-09-12T13:14:57.860 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 804 2024-09-12T13:14:57.885 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:14:57.889 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:14:57.923 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 809 2024-09-12T13:14:57.927 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 808 2024-09-12T13:14:57.936 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:14:57.938 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:14:58.056 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:14:58.056 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:14:58.056 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:14:58.056 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:14:58.056 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:14:58.056 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:14:58.877 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 811 2024-09-12T13:14:58.886 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 810 2024-09-12T13:14:58.897 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:14:58.901 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:14:59.012 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:14:59.012 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:14:59.012 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:14:59.012 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:14:59.012 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:14:59.012 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:14:59.875 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 812 2024-09-12T13:14:59.887 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:14:59.895 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 813 2024-09-12T13:14:59.904 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:15:00.027 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:15:00.027 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:15:00.027 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:15:00.027 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:15:00.027 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:15:00.027 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:15:00.036 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:15:00.036 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:15:00.036 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:15:00.036 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:15:00.036 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:15:00.036 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:15:00.870 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 815 2024-09-12T13:15:00.875 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 814 2024-09-12T13:15:00.883 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:15:00.884 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:15:00.887 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:15:00.887 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:15:00.887 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:15:00.887 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:15:00.887 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:15:00.888 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:15:00.892 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:15:00.892 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:15:00.892 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:15:00.893 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:15:00.893 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:15:00.893 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:15:00.939 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:15:00.939 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:15:00.939 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:15:00.939 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:15:00.939 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:15:00.939 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:15:00.940 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:15:00.940 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:15:00.940 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:15:00.940 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:15:00.941 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:15:00.941 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:15:01.900 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:15:01.900 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:15:01.901 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:15:01.901 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:15:01.901 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:15:01.901 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:15:01.903 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 816 2024-09-12T13:15:01.904 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:15:01.904 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:15:01.904 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:15:01.904 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:15:01.904 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:15:01.904 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:15:01.907 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 817 2024-09-12T13:15:01.921 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:15:01.921 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:15:02.890 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:15:02.890 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:15:02.890 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:15:02.890 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:15:02.890 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:15:02.890 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:15:02.903 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 819 2024-09-12T13:15:02.909 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:15:02.911 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:15:02.911 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:15:02.911 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:15:02.911 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:15:02.911 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:15:02.914 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 818 2024-09-12T13:15:02.929 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:15:02.931 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:15:03.886 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:15:03.886 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:15:03.886 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:15:03.886 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:15:03.886 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:15:03.886 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:15:03.887 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:15:03.887 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:15:03.888 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:15:03.888 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:15:03.888 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:15:03.888 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:15:03.891 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 821 2024-09-12T13:15:03.895 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 820 2024-09-12T13:15:03.921 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:15:03.923 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:15:04.925 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:15:04.925 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:15:04.925 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:15:04.925 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:15:04.926 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:15:04.926 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:15:04.926 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:15:04.926 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:15:04.926 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:15:04.926 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:15:04.926 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:15:04.926 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:15:05.861 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 823 2024-09-12T13:15:05.894 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 822 2024-09-12T13:15:05.910 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:15:05.932 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:15:05.933 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:15:05.933 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:15:05.933 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:15:05.933 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:15:05.933 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:15:05.933 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:15:05.935 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:15:05.935 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:15:05.935 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:15:05.935 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:15:05.935 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:15:05.935 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:15:05.952 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:15:05.952+0000 7f67dd000640 -1 rbd::mirror::image_replayer::snapshot::Replayer: 0x56055c2f9400 handle_unlink_peer: failed to unlink local peer from remote image: (22) Invalid argument 2024-09-12T13:15:06.882 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 825 2024-09-12T13:15:06.891 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 824 2024-09-12T13:15:06.925 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:15:06.925 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:15:06.925 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:15:06.925 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:15:06.925 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:15:06.925 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:15:06.926 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:15:06.927 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:15:06.927 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:15:06.927 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:15:06.927 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:15:06.927 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:15:06.929 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:15:06.942 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:15:07.854 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 826 2024-09-12T13:15:07.869 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 827 2024-09-12T13:15:07.897 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:15:07.905 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:15:08.882 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 828 2024-09-12T13:15:08.893 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 829 2024-09-12T13:15:08.913 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:15:08.913 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:15:08.913 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:15:08.913 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:15:08.913 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:15:08.913 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:15:08.922 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:15:08.932 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:15:08.935 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:15:08.935 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:15:08.935 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:15:08.935 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:15:08.936 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:15:08.936 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:15:09.887 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 830 2024-09-12T13:15:09.897 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 831 2024-09-12T13:15:09.921 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:15:09.921 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:15:09.933 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:15:09.933 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:15:09.933 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:15:09.933 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:15:09.934 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:15:09.934 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:15:09.947 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:15:09.947 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:15:09.947 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:15:09.947 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:15:09.947 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:15:09.947 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:15:09.952 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 833 2024-09-12T13:15:09.957 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 832 2024-09-12T13:15:09.963 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:15:09.967 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:15:10.900 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:15:10.900 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:15:10.900 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:15:10.900 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:15:10.901 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:15:10.901 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:15:10.901 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 835 2024-09-12T13:15:10.907 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:15:10.907 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:15:10.907 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:15:10.907 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:15:10.907 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:15:10.908 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:15:10.911 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 834 2024-09-12T13:15:10.920 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:15:10.922 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:15:11.910 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 837 2024-09-12T13:15:11.913 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 836 2024-09-12T13:15:11.919 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:15:11.921 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:15:11.924 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:15:11.924 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:15:11.925 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:15:11.925 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:15:11.925 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:15:11.925 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:15:11.935 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:15:11.935 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:15:11.935 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:15:11.935 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:15:11.935 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:15:11.935 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:15:12.912 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 839 2024-09-12T13:15:12.917 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 838 2024-09-12T13:15:12.921 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:15:12.924 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:15:12.924 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:15:12.924 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:15:12.924 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:15:12.924 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:15:12.924 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:15:12.924 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:15:12.924 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:15:12.924 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:15:12.925 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:15:12.925 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:15:12.925 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:15:12.925 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:15:12.967 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:15:12.967 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:15:12.967 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:15:12.967 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:15:12.968 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:15:12.968 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:15:12.970 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:15:12.970 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:15:12.970 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:15:12.970 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:15:12.971 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:15:12.971 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:15:13.923 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:15:13.924 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:15:13.924 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:15:13.924 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:15:13.924 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:15:13.924 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:15:13.924 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 840 2024-09-12T13:15:13.925 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:15:13.925 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:15:13.925 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:15:13.925 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:15:13.925 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:15:13.925 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:15:13.927 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 841 2024-09-12T13:15:13.939 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:15:13.944 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:15:14.922 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:15:14.923 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:15:14.923 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:15:14.923 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:15:14.924 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:15:14.924 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:15:14.924 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:15:14.925 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:15:14.925 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:15:14.925 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:15:14.925 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:15:14.925 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:15:14.960 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 843 2024-09-12T13:15:14.960 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 842 2024-09-12T13:15:14.970 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 844 2024-09-12T13:15:14.984 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:15:14.991 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:15:15.003 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:15:15.924 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:15:15.924 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:15:15.924 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:15:15.924 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:15:15.926 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:15:15.926 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:15:15.929 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:15:15.929 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:15:15.929 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:15:15.929 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:15:15.929 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:15:15.929 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:15:15.939 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 845 2024-09-12T13:15:15.959 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:15:16.943 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:15:16.943 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:15:16.943 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:15:16.943 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:15:16.943 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:15:16.943 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:15:16.946 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:15:16.946 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:15:16.946 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:15:16.946 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:15:16.946 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:15:16.946 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:15:17.852 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 846 2024-09-12T13:15:17.861 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 847 2024-09-12T13:15:17.902 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:15:17.914 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:15:17.952 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 848 2024-09-12T13:15:17.964 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 849 2024-09-12T13:15:17.974 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:15:17.980 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:15:17.986 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:15:17.986 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:15:17.986 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:15:17.986 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:15:17.987 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:15:17.987 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:15:17.995 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:15:17.995 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:15:17.995 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:15:17.996 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:15:17.996 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:15:17.996 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:15:18.007 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:15:18.007 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:15:18.007 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:15:18.007 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:15:18.007 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:15:18.007 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:15:18.963 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:15:18.963 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:15:18.964 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:15:18.964 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:15:18.964 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:15:18.964 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:15:18.966 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:15:18.966+0000 7f67dc600640 -1 rbd::mirror::image_replayer::snapshot::Replayer: 0x56055c0a6000 handle_unlink_peer: failed to unlink local peer from remote image: (22) Invalid argument 2024-09-12T13:15:18.966 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:15:18.966+0000 7f67dd000640 -1 rbd::mirror::image_replayer::snapshot::Replayer: 0x56055bab7800 handle_unlink_peer: failed to unlink local peer from remote image: (22) Invalid argument 2024-09-12T13:15:18.973 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 854 2024-09-12T13:15:18.988 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:15:19.915 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 850 2024-09-12T13:15:19.922 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 851 2024-09-12T13:15:19.957 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:15:19.963 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:15:20.905 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:15:20.906 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:15:20.906 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:15:20.906 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:15:20.906 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:15:20.906 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:15:20.918 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:15:20.918 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:15:20.918 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:15:20.918 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:15:20.918 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:15:20.918 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:15:20.927 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 853 2024-09-12T13:15:20.936 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 852 2024-09-12T13:15:20.960 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:15:20.967 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:15:20.978 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:15:20.978 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:15:20.978 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:15:20.978 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:15:20.979 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:15:20.979 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:15:20.983 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:15:20.983 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:15:20.983 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:15:20.983 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:15:20.983 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:15:20.983 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:15:21.925 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 855 2024-09-12T13:15:21.944 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:15:21.966 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 857 2024-09-12T13:15:21.977 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:15:21.979 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 856 2024-09-12T13:15:21.987 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:15:21.990 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:15:21.990 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:15:21.990 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:15:21.990 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:15:21.990 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:15:21.990 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:15:22.960 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:15:22.960 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:15:22.960 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:15:22.960 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:15:22.960 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:15:22.961 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:15:22.966 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:15:22.966 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:15:22.966 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:15:22.966 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:15:22.966 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:15:22.966 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:15:23.919 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 859 2024-09-12T13:15:23.933 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 858 2024-09-12T13:15:23.961 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:15:23.963 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:15:23.963 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:15:23.963 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:15:23.963 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:15:23.963 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:15:23.963 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:15:23.970 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:15:23.970 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:15:23.970 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:15:23.971 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:15:23.971 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:15:23.971 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:15:23.971 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:15:23.998 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 862 2024-09-12T13:15:24.000 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 861 2024-09-12T13:15:24.016 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:15:24.016 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:15:24.948 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:15:24.949 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:15:24.949 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:15:24.949 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:15:24.949 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:15:24.949 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:15:24.980 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:15:24.980 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:15:24.980 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:15:24.981 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:15:24.981 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:15:24.981 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:15:24.982 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 863 2024-09-12T13:15:24.989 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 864 2024-09-12T13:15:24.990 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:15:24.990 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:15:24.990 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:15:24.990 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:15:24.990 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:15:24.990 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:15:24.991 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:15:24.999 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:15:25.984 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 865 2024-09-12T13:15:25.991 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:15:26.958 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 860 2024-09-12T13:15:26.964 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:15:26.965 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:15:26.965 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:15:26.965 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:15:26.965 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:15:26.965 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:15:26.972 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:15:26.973 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:15:26.973 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:15:26.973 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:15:26.973 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:15:26.973 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:15:26.994 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:15:27.019 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:15:27.020 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:15:27.020 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:15:27.020 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:15:27.020 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:15:27.020 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:15:27.020 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:15:27.020 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:15:27.020 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:15:27.020 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:15:27.020 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:15:27.020 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:15:27.858 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 867 2024-09-12T13:15:27.864 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 866 2024-09-12T13:15:27.880 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:15:27.883 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:15:27.908 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 870 2024-09-12T13:15:27.910 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 871 2024-09-12T13:15:27.911 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 869 2024-09-12T13:15:27.913 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 868 2024-09-12T13:15:27.922 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:15:27.924 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:15:27.926 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:15:27.926 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:15:27.994 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:15:27.994 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:15:27.994 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:15:27.994 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:15:27.994 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:15:27.994 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:15:28.002 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:15:28.002 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:15:28.002 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:15:28.002 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:15:28.002 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:15:28.003 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:15:28.994 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:15:28.994 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:15:28.994 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:15:28.995 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:15:28.995 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:15:28.995 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:15:29.997 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:15:29.997 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:15:29.997 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:15:29.997 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:15:29.997 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:15:29.997 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:15:30.883 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:15:30.883 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:15:30.883 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:15:30.883 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:15:30.883 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:15:30.883 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:15:30.885 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:15:30.885 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:15:30.885 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:15:30.885 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:15:30.885 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:15:30.885 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:15:30.925 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:15:30.925 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:15:30.925 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:15:30.925 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:15:30.925 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:15:30.926 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:15:30.926 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:15:30.926 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:15:30.926 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:15:30.926 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:15:30.926 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:15:30.926 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:15:30.928 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:15:30.928 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:15:30.928 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:15:30.928 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:15:30.928 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:15:30.929 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:15:30.929 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:15:30.929 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:15:30.929 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:15:30.929 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:15:30.930 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:15:30.930 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:15:30.961 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 873 2024-09-12T13:15:30.971 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 872 2024-09-12T13:15:31.001 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:15:31.006 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:15:31.061 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 875 2024-09-12T13:15:31.077 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:15:31.965 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 874 2024-09-12T13:15:31.984 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:15:32.004 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 879 2024-09-12T13:15:32.020 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 876 2024-09-12T13:15:32.025 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 881 2024-09-12T13:15:32.026 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:15:32.026 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 877 2024-09-12T13:15:32.029 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:15:32.036 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:15:32.039 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:15:33.966 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 878 2024-09-12T13:15:33.973 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 880 2024-09-12T13:15:33.993 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:15:33.997 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:15:34.003 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:15:34.003 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:15:34.003 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:15:34.004 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:15:34.004 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:15:34.004 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:15:34.008 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:15:34.009 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:15:34.009 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:15:34.009 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:15:34.009 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:15:34.009 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:15:34.081 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:15:34.081 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:15:34.081 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:15:34.081 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:15:34.081 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:15:34.081 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:15:34.988 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:15:34.988 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:15:34.988 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:15:34.988 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:15:34.989 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:15:34.989 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:15:35.012 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 883 2024-09-12T13:15:35.013 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 882 2024-09-12T13:15:35.020 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:15:35.021 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:15:35.029 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:15:35.029 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:15:35.029 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:15:35.029 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:15:35.029 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:15:35.029 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:15:35.032 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:15:35.033 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:15:35.033 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:15:35.033 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:15:35.033 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:15:35.033 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:15:35.039 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:15:35.039 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:15:35.039 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:15:35.039 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:15:35.039 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:15:35.039 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:15:35.043 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:15:35.043 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:15:35.043 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:15:35.043 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:15:35.043 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:15:35.043 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:15:36.025 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 889 2024-09-12T13:15:36.029 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 886 2024-09-12T13:15:36.042 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 885 2024-09-12T13:15:36.044 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:15:36.045 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:15:36.050 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:15:36.996 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:15:36.996 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:15:36.996 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:15:36.996 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:15:36.996 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:15:36.996 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:15:36.997 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 884 2024-09-12T13:15:36.999 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:15:36.999 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:15:37.000 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:15:37.000 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:15:37.000 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:15:37.000 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:15:37.038 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:15:37.848 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 887 2024-09-12T13:15:37.855 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 888 2024-09-12T13:15:37.898 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:15:37.901 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:15:37.916 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 890 2024-09-12T13:15:37.921 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 891 2024-09-12T13:15:37.925 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:15:37.929 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:15:38.024 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:15:38.024 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:15:38.024 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:15:38.024 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:15:38.024 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:15:38.024 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:15:38.024 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:15:38.024 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:15:38.024 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:15:38.024 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:15:38.025 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:15:38.025 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:15:39.048 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:15:39.049 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:15:39.049 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:15:39.049 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:15:39.049 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:15:39.049 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:15:39.049 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:15:39.049 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:15:39.049 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:15:39.049 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:15:39.049 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:15:39.049 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:15:39.053 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:15:39.053 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:15:39.053 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:15:39.053 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:15:39.053 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:15:39.053 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:15:40.042 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:15:40.042 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:15:40.042 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:15:40.042 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:15:40.042 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:15:40.042 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:15:40.904 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:15:40.904 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:15:40.904 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:15:40.905 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:15:40.905 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:15:40.905 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:15:40.906 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:15:40.906 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:15:40.907 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:15:40.907 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:15:40.907 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:15:40.907 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:15:40.929 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:15:40.929 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:15:40.929 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:15:40.929 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:15:40.930 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:15:40.930 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:15:40.932 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:15:40.932 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:15:40.932 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:15:40.932 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:15:40.933 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:15:40.933 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:15:40.986 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 895 2024-09-12T13:15:41.002 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 893 2024-09-12T13:15:41.011 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 894 2024-09-12T13:15:41.025 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 892 2024-09-12T13:15:41.032 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:15:41.058 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:15:41.061 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:15:41.061 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:15:41.110 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 897 2024-09-12T13:15:41.125 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:15:41.996 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 896 2024-09-12T13:15:42.018 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:15:42.049 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 899 2024-09-12T13:15:42.060 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:15:42.073 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 898 2024-09-12T13:15:42.081 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:15:44.003 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 901 2024-09-12T13:15:44.009 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 900 2024-09-12T13:15:44.027 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:15:44.028 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:15:44.035 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:15:44.035 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:15:44.035 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:15:44.035 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:15:44.035 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:15:44.035 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:15:44.061 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:15:44.061 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:15:44.062 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:15:44.062 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:15:44.062 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:15:44.062 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:15:44.064 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:15:44.064 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:15:44.064 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:15:44.064 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:15:44.064 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:15:44.064 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:15:44.064 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:15:44.064 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:15:44.064 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:15:44.065 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:15:44.065 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:15:44.065 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:15:44.128 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:15:44.128 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:15:44.128 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:15:44.128 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:15:44.128 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:15:44.128 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:15:45.021 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:15:45.021 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:15:45.021 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:15:45.021 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:15:45.021 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:15:45.021 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:15:45.063 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:15:45.063 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:15:45.063 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:15:45.063 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:15:45.063 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:15:45.063 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:15:45.081 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 905 2024-09-12T13:15:45.083 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:15:45.083 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:15:45.083 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:15:45.083 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:15:45.084 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:15:45.084 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:15:45.086 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 904 2024-09-12T13:15:45.094 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:15:45.095 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:15:46.018 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 902 2024-09-12T13:15:46.024 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 903 2024-09-12T13:15:46.049 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:15:46.063 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:15:46.087 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 907 2024-09-12T13:15:46.095 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:15:47.016 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 906 2024-09-12T13:15:47.030 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:15:47.030 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:15:47.030 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:15:47.030 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:15:47.030 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:15:47.030 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:15:47.031 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:15:47.031 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:15:47.031 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:15:47.031 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:15:47.031 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:15:47.031 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:15:47.051 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:15:47.862 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 909 2024-09-12T13:15:47.867 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 908 2024-09-12T13:15:47.885 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:15:47.887 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:15:47.917 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 910 2024-09-12T13:15:47.924 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 911 2024-09-12T13:15:47.927 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:15:47.935 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:15:48.098 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:15:48.098 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:15:48.098 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:15:48.098 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:15:48.098 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:15:48.098 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:15:48.099 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:15:48.099 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:15:48.099 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:15:48.099 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:15:48.099 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:15:48.099 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:15:49.051 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:15:49.052 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:15:49.052 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:15:49.052 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:15:49.052 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:15:49.052 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:15:49.066 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:15:49.066 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:15:49.066 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:15:49.066 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:15:49.066 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:15:49.066 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:15:49.098 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:15:49.098 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:15:49.098 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:15:49.098 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:15:49.098 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:15:49.098 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:15:50.054 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:15:50.054 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:15:50.054 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:15:50.054 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:15:50.054 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:15:50.054 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:15:50.888 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:15:50.890 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:15:50.890 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:15:50.890 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:15:50.890 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:15:50.890 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:15:50.891 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:15:50.891 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:15:50.891 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:15:50.891 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:15:50.891 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:15:50.891 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:15:50.931 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:15:50.931 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:15:50.931 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:15:50.931 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:15:50.931 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:15:50.931 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:15:50.939 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:15:50.939 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:15:50.939 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:15:50.939 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:15:50.939 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:15:50.939 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:15:51.030 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 913 2024-09-12T13:15:51.043 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 912 2024-09-12T13:15:51.072 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:15:51.086 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:15:51.106 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 915 2024-09-12T13:15:51.114 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 914 2024-09-12T13:15:51.132 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:15:51.138 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:15:51.152 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 917 2024-09-12T13:15:51.165 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:15:52.028 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 916 2024-09-12T13:15:52.052 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:15:52.083 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 918 2024-09-12T13:15:52.085 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 919 2024-09-12T13:15:52.092 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:15:52.094 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:15:54.030 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 920 2024-09-12T13:15:54.035 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 921 2024-09-12T13:15:54.060 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:15:54.065 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:15:54.075 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:15:54.075 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:15:54.075 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:15:54.075 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:15:54.075 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:15:54.075 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:15:54.088 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:15:54.088 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:15:54.088 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:15:54.088 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:15:54.089 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:15:54.089 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:15:54.135 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:15:54.135 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:15:54.135 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:15:54.135 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:15:54.135 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:15:54.135 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:15:54.141 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:15:54.141 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:15:54.141 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:15:54.141 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:15:54.141 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:15:54.141 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:15:54.168 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:15:54.168 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:15:54.168 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:15:54.168 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:15:54.168 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:15:54.168 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:15:55.055 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:15:55.055 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:15:55.055 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:15:55.055 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:15:55.055 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:15:55.055 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:15:55.083 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 922 2024-09-12T13:15:55.093 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:15:55.094 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:15:55.095 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:15:55.095 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:15:55.095 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:15:55.095 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:15:55.095 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:15:55.096 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:15:55.096 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:15:55.096 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:15:55.096 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:15:55.096 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:15:55.096 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:15:55.100 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 923 2024-09-12T13:15:55.111 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:15:56.141 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 927 2024-09-12T13:15:56.149 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:15:57.055 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 926 2024-09-12T13:15:57.062 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:15:57.062 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:15:57.063 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:15:57.063 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:15:57.063 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:15:57.063 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:15:57.068 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:15:57.068 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:15:57.068 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:15:57.068 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:15:57.068 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:15:57.068 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:15:57.100 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:15:57.867 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 929 2024-09-12T13:15:57.873 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 928 2024-09-12T13:15:57.914 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:15:57.916 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:15:57.955 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:15:57.955+0000 7f67dc600640 -1 rbd::mirror::image_replayer::snapshot::Replayer: 0x56055c196400 handle_unlink_peer: failed to unlink local peer from remote image: (22) Invalid argument 2024-09-12T13:15:57.971 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 930 2024-09-12T13:15:57.977 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 931 2024-09-12T13:15:57.990 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:15:57.992 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:15:58.096 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:15:58.097 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:15:58.097 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:15:58.097 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:15:58.097 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:15:58.097 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:15:58.113 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:15:58.114 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:15:58.114 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:15:58.114 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:15:58.114 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:15:58.114 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:15:58.857 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 924 2024-09-12T13:15:58.860 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 925 2024-09-12T13:15:58.886 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:15:58.894 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:15:59.152 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:15:59.152 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:15:59.152 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:15:59.152 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:15:59.152 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:15:59.152 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:15:59.896 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 933 2024-09-12T13:15:59.918 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:16:00.103 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:16:00.103 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:16:00.103 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:16:00.104 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:16:00.104 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:16:00.104 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:16:00.856 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 932 2024-09-12T13:16:00.889 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:16:00.902 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:16:00.901+0000 7f67dd000640 -1 rbd::mirror::image_replayer::snapshot::Replayer: 0x560556bea800 handle_unlink_peer: failed to unlink local peer from remote image: (22) Invalid argument 2024-09-12T13:16:00.915 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 935 2024-09-12T13:16:00.916 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:16:00.917 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:16:00.917 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:16:00.917 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:16:00.917 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:16:00.917 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:16:00.919 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:16:00.919 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:16:00.919 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:16:00.919 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:16:00.919 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:16:00.919 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:16:00.928 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:16:00.992 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:16:00.992 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:16:00.993 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:16:00.993 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:16:00.993 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:16:00.993 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:16:00.994 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:16:00.994 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:16:00.994 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:16:00.995 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:16:00.995 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:16:00.995 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:16:01.848 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 934 2024-09-12T13:16:01.885 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:16:01.889 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:16:01.889 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:16:01.889 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:16:01.889 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:16:01.889 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:16:01.889 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:16:01.897 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:16:01.897 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:16:01.898 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:16:01.898 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:16:01.898 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:16:01.898 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:16:01.914 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 936 2024-09-12T13:16:01.938 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:16:01.949 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 937 2024-09-12T13:16:01.958 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:16:02.921 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:16:02.921 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:16:02.921 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:16:02.922 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:16:02.922 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:16:02.922 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:16:02.929 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:16:02.928+0000 7f67dc600640 -1 rbd::mirror::image_replayer::snapshot::Replayer: 0x56055bab6c00 handle_unlink_peer: failed to unlink local peer from remote image: (22) Invalid argument 2024-09-12T13:16:02.942 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 941 2024-09-12T13:16:02.957 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:16:03.874 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 938 2024-09-12T13:16:03.880 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 939 2024-09-12T13:16:03.892 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:16:03.892 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:16:03.892 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:16:03.892 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:16:03.892 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:16:03.892 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:16:03.918 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:16:03.920 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:16:03.931 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:16:03.931 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:16:03.931 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:16:03.931 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:16:03.932 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:16:03.932 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:16:04.878 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 940 2024-09-12T13:16:04.888 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:16:04.888 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:16:04.888 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:16:04.889 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:16:04.889 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:16:04.889 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:16:04.915 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:16:04.940 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 943 2024-09-12T13:16:04.942 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:16:04.942 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:16:04.942 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:16:04.942 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:16:04.943 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:16:04.943 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:16:04.954 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:16:04.961 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:16:04.961 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:16:04.961 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:16:04.961 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:16:04.962 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:16:04.962 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:16:05.894 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 942 2024-09-12T13:16:05.930 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:16:05.934 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:16:05.934+0000 7f67dd000640 -1 rbd::mirror::image_replayer::snapshot::Replayer: 0x56055c10ec00 handle_unlink_peer: failed to unlink local peer from remote image: (22) Invalid argument 2024-09-12T13:16:05.945 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 945 2024-09-12T13:16:05.959 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:16:05.960 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:16:05.960 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:16:05.960 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:16:05.960 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:16:05.960 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:16:05.960 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:16:06.875 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 944 2024-09-12T13:16:06.918 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:16:06.921 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:16:06.921 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:16:06.921 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:16:06.921 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:16:06.921 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:16:06.921 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:16:06.923 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:16:06.923 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:16:06.923 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:16:06.923 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:16:06.923 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:16:06.923 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:16:06.985 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 948 2024-09-12T13:16:06.998 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:16:07.848 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 947 2024-09-12T13:16:07.855 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 946 2024-09-12T13:16:07.885 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:16:07.900 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:16:07.918 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:16:07.918 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:16:07.918 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:16:07.918 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:16:07.918 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:16:07.919 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:16:07.919 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 950 2024-09-12T13:16:07.926 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 949 2024-09-12T13:16:07.936 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:16:07.937 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:16:07.956 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:16:07.956 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:16:07.956 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:16:07.956 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:16:07.956 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:16:07.957 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:16:08.907 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 951 2024-09-12T13:16:08.920 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:16:08.933 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:16:08.933 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:16:08.933 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:16:08.934 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:16:08.934 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:16:08.934 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:16:08.962 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:16:08.962 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:16:08.962 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:16:08.962 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:16:08.962 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:16:08.962 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:16:09.921 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:16:09.922 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:16:09.922 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:16:09.922 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:16:09.922 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:16:09.922 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:16:09.924 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 954 2024-09-12T13:16:09.928 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 953 2024-09-12T13:16:09.937 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:16:09.941 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:16:10.001 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:16:10.001 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:16:10.001 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:16:10.002 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:16:10.002 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:16:10.002 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:16:10.889 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:16:10.889 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:16:10.889 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:16:10.889 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:16:10.889 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:16:10.889 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:16:10.894 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 952 2024-09-12T13:16:10.903 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:16:10.904 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:16:10.904 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:16:10.904 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:16:10.904 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:16:10.904 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:16:10.925 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:16:10.933 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 955 2024-09-12T13:16:10.935 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 956 2024-09-12T13:16:10.940 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:16:10.940 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:16:10.940 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:16:10.940 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:16:10.940 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:16:10.940 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:16:10.940 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:16:10.940 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:16:10.941 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:16:10.941 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:16:10.941 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:16:10.941 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:16:10.948 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:16:10.956 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:16:11.922 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 959 2024-09-12T13:16:11.923 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:16:11.923 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:16:11.923 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:16:11.923 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:16:11.923 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:16:11.923 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:16:11.935 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 957 2024-09-12T13:16:11.945 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:16:11.968 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 958 2024-09-12T13:16:11.970 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:16:11.978 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:16:12.940 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:16:12.940 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:16:12.940 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:16:12.940 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:16:12.940 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:16:12.940 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:16:12.941 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 960 2024-09-12T13:16:12.946 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:16:12.946 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:16:12.946 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:16:12.946 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:16:12.946 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:16:12.946 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:16:12.964 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:16:13.930 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:16:13.931 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:16:13.931 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:16:13.931 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:16:13.931 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:16:13.931 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:16:13.931 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 961 2024-09-12T13:16:13.946 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:16:13.951 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:16:13.951 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:16:13.951 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:16:13.951 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:16:13.951 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:16:13.952 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:16:13.954 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 963 2024-09-12T13:16:13.970 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:16:14.946 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 964 2024-09-12T13:16:14.947 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 965 2024-09-12T13:16:14.948 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:16:14.948 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:16:14.949 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:16:14.949 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:16:14.949 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:16:14.949 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:16:14.962 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:16:14.964 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:16:14.973 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:16:14.974 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:16:14.974 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:16:14.974 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:16:14.974 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:16:14.974 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:16:14.980 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:16:14.980 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:16:14.980 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:16:14.980 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:16:14.980 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:16:14.981 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:16:15.916 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 962 2024-09-12T13:16:15.938 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:16:15.950 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 967 2024-09-12T13:16:15.964 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:16:15.969 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${SNAP_PIDS[@]} 2024-09-12T13:16:15.969 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 59186 2024-09-12T13:16:15.969 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${SNAP_PIDS[@]} 2024-09-12T13:16:15.969 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 59318 2024-09-12T13:16:16.949 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:16:16.949 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:16:16.949 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:16:16.949 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:16:16.949 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:16:16.949 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:16:17.856 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 968 2024-09-12T13:16:17.864 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 966 2024-09-12T13:16:17.892 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:16:17.893 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:16:17.965 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:16:17.965 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:16:17.965 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:16:17.965 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:16:17.966 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:16:17.966 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:16:17.967 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:16:17.967 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:16:17.967 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:16:17.967 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:16:17.967 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:16:17.967 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:16:18.941 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${SNAP_PIDS[@]} 2024-09-12T13:16:18.941 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 59450 2024-09-12T13:16:18.941 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${SNAP_PIDS[@]} 2024-09-12T13:16:18.941 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 59602 2024-09-12T13:16:18.973 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 970 2024-09-12T13:16:18.990 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:16:19.932 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 969 2024-09-12T13:16:19.961 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:16:20.897 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:16:20.897 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:16:20.897 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:16:20.897 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:16:20.897 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:16:20.897 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:16:20.898 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${SNAP_PIDS[@]} 2024-09-12T13:16:20.898 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 59755 2024-09-12T13:16:20.898 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${SNAP_PIDS[@]} 2024-09-12T13:16:20.898 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 59908 2024-09-12T13:16:20.925 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 971 2024-09-12T13:16:20.942 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:16:21.951 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 972 2024-09-12T13:16:21.964 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:16:22.965 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${SNAP_PIDS[@]} 2024-09-12T13:16:22.965 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 60061 2024-09-12T13:16:23.945 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${SNAP_PIDS[@]} 2024-09-12T13:16:23.946 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 60234 2024-09-12T13:16:23.946 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${SNAP_PIDS[@]} 2024-09-12T13:16:23.946 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 60408 2024-09-12T13:16:24.967 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ (( RET != 0 )) 2024-09-12T13:16:24.968 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${WORKLOAD_PIDS[@]} 2024-09-12T13:16:24.968 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 59055 2024-09-12T13:18:51.647 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ ret=124 2024-09-12T13:18:51.648 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ (( ret != 124 )) 2024-09-12T13:18:51.648 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${WORKLOAD_PIDS[@]} 2024-09-12T13:18:51.648 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 59187 2024-09-12T13:18:52.663 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ ret=124 2024-09-12T13:18:52.663 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ (( ret != 124 )) 2024-09-12T13:18:52.663 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${WORKLOAD_PIDS[@]} 2024-09-12T13:18:52.663 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 59319 2024-09-12T13:18:53.606 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ ret=124 2024-09-12T13:18:53.607 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ (( ret != 124 )) 2024-09-12T13:18:53.607 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${WORKLOAD_PIDS[@]} 2024-09-12T13:18:53.607 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 59451 2024-09-12T13:18:54.522 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ ret=124 2024-09-12T13:18:54.522 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ (( ret != 124 )) 2024-09-12T13:18:54.523 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${WORKLOAD_PIDS[@]} 2024-09-12T13:18:54.523 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 59603 2024-09-12T13:18:55.523 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ ret=124 2024-09-12T13:18:55.523 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ (( ret != 124 )) 2024-09-12T13:18:55.523 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${WORKLOAD_PIDS[@]} 2024-09-12T13:18:55.523 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 59756 2024-09-12T13:18:56.537 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ ret=124 2024-09-12T13:18:56.537 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ (( ret != 124 )) 2024-09-12T13:18:56.537 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${WORKLOAD_PIDS[@]} 2024-09-12T13:18:56.537 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 59909 2024-09-12T13:18:57.539 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ ret=124 2024-09-12T13:18:57.539 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ (( ret != 124 )) 2024-09-12T13:18:57.540 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${WORKLOAD_PIDS[@]} 2024-09-12T13:18:57.540 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 60062 2024-09-12T13:18:58.557 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ ret=124 2024-09-12T13:18:58.558 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ (( ret != 124 )) 2024-09-12T13:18:58.558 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${WORKLOAD_PIDS[@]} 2024-09-12T13:18:58.558 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 60235 2024-09-12T13:18:59.574 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ ret=124 2024-09-12T13:18:59.574 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ (( ret != 124 )) 2024-09-12T13:18:59.574 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${WORKLOAD_PIDS[@]} 2024-09-12T13:18:59.574 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 60409 2024-09-12T13:19:00.738 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ ret=124 2024-09-12T13:19:00.738 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ (( ret != 124 )) 2024-09-12T13:19:00.739 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ (( RET != 0 )) 2024-09-12T13:19:00.739 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:19:00.739 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ COMPARE_PIDS+=($!) 2024-09-12T13:19:00.739 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:19:00.740 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ compare_demoted_promoted_image 1 2024-09-12T13:19:00.741 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local dev=/dev/rbd0 2024-09-12T13:19:00.742 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local img=image-primary1 2024-09-12T13:19:00.742 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mntpt=test-primary1 2024-09-12T13:19:00.742 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local demote_md5 promote_md5 2024-09-12T13:19:00.742 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo umount test-primary1 2024-09-12T13:19:00.742 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ COMPARE_PIDS+=($!) 2024-09-12T13:19:00.742 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:19:00.742 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ compare_demoted_promoted_image 2 2024-09-12T13:19:00.742 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local dev=/dev/rbd1 2024-09-12T13:19:00.742 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local img=image-primary2 2024-09-12T13:19:00.742 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mntpt=test-primary2 2024-09-12T13:19:00.742 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local demote_md5 promote_md5 2024-09-12T13:19:00.742 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo umount test-primary2 2024-09-12T13:19:00.743 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ COMPARE_PIDS+=($!) 2024-09-12T13:19:00.743 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:19:00.743 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ compare_demoted_promoted_image 3 2024-09-12T13:19:00.743 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local dev=/dev/rbd2 2024-09-12T13:19:00.743 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local img=image-primary3 2024-09-12T13:19:00.743 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mntpt=test-primary3 2024-09-12T13:19:00.743 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local demote_md5 promote_md5 2024-09-12T13:19:00.743 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo umount test-primary3 2024-09-12T13:19:00.743 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ COMPARE_PIDS+=($!) 2024-09-12T13:19:00.743 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:19:00.743 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ compare_demoted_promoted_image 4 2024-09-12T13:19:00.745 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local dev=/dev/rbd3 2024-09-12T13:19:00.745 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local img=image-primary4 2024-09-12T13:19:00.745 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ COMPARE_PIDS+=($!) 2024-09-12T13:19:00.745 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mntpt=test-primary4 2024-09-12T13:19:00.745 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local demote_md5 promote_md5 2024-09-12T13:19:00.745 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo umount test-primary4 2024-09-12T13:19:00.745 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:19:00.745 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ compare_demoted_promoted_image 5 2024-09-12T13:19:00.745 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local dev=/dev/rbd4 2024-09-12T13:19:00.745 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local img=image-primary5 2024-09-12T13:19:00.746 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mntpt=test-primary5 2024-09-12T13:19:00.746 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local demote_md5 promote_md5 2024-09-12T13:19:00.746 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo umount test-primary5 2024-09-12T13:19:00.746 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ COMPARE_PIDS+=($!) 2024-09-12T13:19:00.746 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:19:00.746 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ compare_demoted_promoted_image 6 2024-09-12T13:19:00.746 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local dev=/dev/rbd5 2024-09-12T13:19:00.746 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local img=image-primary6 2024-09-12T13:19:00.746 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mntpt=test-primary6 2024-09-12T13:19:00.746 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local demote_md5 promote_md5 2024-09-12T13:19:00.746 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo umount test-primary6 2024-09-12T13:19:00.746 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ COMPARE_PIDS+=($!) 2024-09-12T13:19:00.746 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:19:00.747 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ compare_demoted_promoted_image 7 2024-09-12T13:19:00.747 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local dev=/dev/rbd6 2024-09-12T13:19:00.747 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local img=image-primary7 2024-09-12T13:19:00.747 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mntpt=test-primary7 2024-09-12T13:19:00.747 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local demote_md5 promote_md5 2024-09-12T13:19:00.747 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo umount test-primary7 2024-09-12T13:19:00.747 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ COMPARE_PIDS+=($!) 2024-09-12T13:19:00.747 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:19:00.748 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ compare_demoted_promoted_image 8 2024-09-12T13:19:00.748 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local dev=/dev/rbd7 2024-09-12T13:19:00.748 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ COMPARE_PIDS+=($!) 2024-09-12T13:19:00.748 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local img=image-primary8 2024-09-12T13:19:00.748 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mntpt=test-primary8 2024-09-12T13:19:00.748 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local demote_md5 promote_md5 2024-09-12T13:19:00.748 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo umount test-primary8 2024-09-12T13:19:00.749 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ compare_demoted_promoted_image 9 2024-09-12T13:19:00.749 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:19:00.749 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ COMPARE_PIDS+=($!) 2024-09-12T13:19:00.750 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local dev=/dev/rbd8 2024-09-12T13:19:00.751 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local img=image-primary9 2024-09-12T13:19:00.751 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mntpt=test-primary9 2024-09-12T13:19:00.751 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local demote_md5 promote_md5 2024-09-12T13:19:00.751 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo umount test-primary9 2024-09-12T13:19:00.751 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ compare_demoted_promoted_image 10 2024-09-12T13:19:00.751 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${COMPARE_PIDS[@]} 2024-09-12T13:19:00.752 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 66362 2024-09-12T13:19:00.752 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local dev=/dev/rbd9 2024-09-12T13:19:00.752 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local img=image-primary10 2024-09-12T13:19:00.752 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mntpt=test-primary10 2024-09-12T13:19:00.753 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local demote_md5 promote_md5 2024-09-12T13:19:00.753 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo umount test-primary10 2024-09-12T13:19:01.268 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo md5sum /dev/rbd0 2024-09-12T13:19:01.278 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ awk '{print $1}' 2024-09-12T13:19:01.316 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ awk '{print $1}' 2024-09-12T13:19:01.317 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo md5sum /dev/rbd2 2024-09-12T13:19:01.540 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo md5sum /dev/rbd3 2024-09-12T13:19:01.545 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ awk '{print $1}' 2024-09-12T13:19:01.555 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ awk '{print $1}' 2024-09-12T13:19:01.557 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo md5sum /dev/rbd1 2024-09-12T13:19:01.578 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ awk '{print $1}' 2024-09-12T13:19:01.578 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo md5sum /dev/rbd6 2024-09-12T13:19:01.665 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo md5sum /dev/rbd9 2024-09-12T13:19:01.667 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ awk '{print $1}' 2024-09-12T13:19:01.669 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo md5sum /dev/rbd7 2024-09-12T13:19:01.670 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ awk '{print $1}' 2024-09-12T13:19:01.675 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ awk '{print $1}' 2024-09-12T13:19:01.676 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo md5sum /dev/rbd4 2024-09-12T13:19:01.698 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo md5sum /dev/rbd5 2024-09-12T13:19:01.698 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ awk '{print $1}' 2024-09-12T13:19:01.699 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo md5sum /dev/rbd8 2024-09-12T13:19:01.702 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ awk '{print $1}' 2024-09-12T13:19:53.292 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ demote_md5=39408e54a2f5295dd5daa911a416fda4 2024-09-12T13:19:53.293 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo rbd --cluster cluster1 device unmap -t krbd mirror/image-primary7 2024-09-12T13:19:54.103 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ demote_image cluster1 mirror image-primary7 2024-09-12T13:19:54.104 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:19:54.104 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:19:54.104 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:19:54.104 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 mirror image demote mirror/image-primary7 2024-09-12T13:19:54.808 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ demote_md5=5d2c4017cbcdcb1cd6f6730465d75450 2024-09-12T13:19:54.808 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo rbd --cluster cluster1 device unmap -t krbd mirror/image-primary3 2024-09-12T13:19:54.953 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Image demoted to non-primary 2024-09-12T13:19:54.978 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary7 up+unknown 2024-09-12T13:19:54.978 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:19:54.978 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:19:54.978 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:19:54.979 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:19:54.979 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:19:54.979 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:19:54.979 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:19:54.979 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:19:55.025 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ demote_image cluster1 mirror image-primary3 2024-09-12T13:19:55.026 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:19:55.026 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:19:55.026 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:19:55.026 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 mirror image demote mirror/image-primary3 2024-09-12T13:19:55.095 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ demote_md5=faa0aaae9753072b2cac78d3e2467795 2024-09-12T13:19:55.095 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo rbd --cluster cluster1 device unmap -t krbd mirror/image-primary10 2024-09-12T13:19:55.195 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ demote_md5=4120d89c38b264f38b25861711499df2 2024-09-12T13:19:55.195 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo rbd --cluster cluster1 device unmap -t krbd mirror/image-primary2 2024-09-12T13:19:55.316 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ demote_image cluster1 mirror image-primary10 2024-09-12T13:19:55.316 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:19:55.317 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:19:55.317 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:19:55.317 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 mirror image demote mirror/image-primary10 2024-09-12T13:19:55.396 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ demote_image cluster1 mirror image-primary2 2024-09-12T13:19:55.396 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:19:55.396 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:19:55.396 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:19:55.396 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 mirror image demote mirror/image-primary2 2024-09-12T13:19:55.956 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Image demoted to non-primary 2024-09-12T13:19:56.010 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary7 up+unknown '' '' 2024-09-12T13:19:56.010 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:19:56.010 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:19:56.012 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:19:56.012 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:19:56.012 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:19:56.012 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:19:56.012 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:19:56.015 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:19:56.015 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary7 2024-09-12T13:19:56.041 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary3 up+unknown 2024-09-12T13:19:56.042 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:19:56.042 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:19:56.042 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:19:56.042 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:19:56.042 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:19:56.043 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:19:56.043 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:19:56.043 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:19:56.249 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary7: 2024-09-12T13:19:56.249 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 3f634cfd-f07a-4271-8c29-0ad3777a474c 2024-09-12T13:19:56.249 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:19:56.249 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:19:56.249 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:19:56.250 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:19:47' 2024-09-12T13:19:56.250 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:19:56.254 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ demote_md5=e4c2be7299c521f6670ee00aa7fe149d 2024-09-12T13:19:56.254 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:19:56.254 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo rbd --cluster cluster1 device unmap -t krbd mirror/image-primary9 2024-09-12T13:19:56.255 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:19:56.255 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 2 2024-09-12T13:19:56.411 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ demote_image cluster1 mirror image-primary9 2024-09-12T13:19:56.412 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:19:56.412 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:19:56.412 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:19:56.412 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 mirror image demote mirror/image-primary9 2024-09-12T13:19:56.418 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ demote_md5=207063409a811f9645abeef41738206b 2024-09-12T13:19:56.418 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo rbd --cluster cluster1 device unmap -t krbd mirror/image-primary5 2024-09-12T13:19:56.568 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ demote_image cluster1 mirror image-primary5 2024-09-12T13:19:56.568 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:19:56.568 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:19:56.568 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:19:56.568 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 mirror image demote mirror/image-primary5 2024-09-12T13:19:56.719 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ demote_md5=39dcba30863a72fe7cf39c1a605bbd65 2024-09-12T13:19:56.719 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo rbd --cluster cluster1 device unmap -t krbd mirror/image-primary6 2024-09-12T13:19:56.830 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ demote_image cluster1 mirror image-primary6 2024-09-12T13:19:56.830 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:19:56.831 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:19:56.831 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:19:56.831 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 mirror image demote mirror/image-primary6 2024-09-12T13:19:56.917 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ demote_md5=ce4382640fa81da8b7b95c4017c84e83 2024-09-12T13:19:56.917 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo rbd --cluster cluster1 device unmap -t krbd mirror/image-primary4 2024-09-12T13:19:56.996 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Image demoted to non-primary 2024-09-12T13:19:57.030 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Image demoted to non-primary 2024-09-12T13:19:57.050 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary3 up+unknown '' '' 2024-09-12T13:19:57.051 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:19:57.051 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:19:57.051 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:19:57.052 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:19:57.052 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:19:57.052 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:19:57.052 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:19:57.052 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:19:57.052 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary3 2024-09-12T13:19:57.053 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary10 up+unknown 2024-09-12T13:19:57.053 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:19:57.056 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:19:57.056 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:19:57.056 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:19:57.056 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:19:57.056 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:19:57.057 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:19:57.058 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:19:57.060 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary5 up+unknown 2024-09-12T13:19:57.060 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:19:57.061 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:19:57.061 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:19:57.061 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:19:57.062 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:19:57.062 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:19:57.062 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:19:57.062 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:19:57.221 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ demote_image cluster1 mirror image-primary4 2024-09-12T13:19:57.224 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:19:57.225 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:19:57.225 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:19:57.225 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 mirror image demote mirror/image-primary4 2024-09-12T13:19:57.257 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary3: 2024-09-12T13:19:57.257 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 50e30e91-e1e0-44f3-8ad3-6fb3a5307bba 2024-09-12T13:19:57.257 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:19:57.257 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:19:57.257 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:19:57.257 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:19:47' 2024-09-12T13:19:57.257 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:19:57.269 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:19:57.269 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:19:57.269 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 2 2024-09-12T13:19:57.598 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ demote_md5=15523d877809e4743f3932b22d38c387 2024-09-12T13:19:57.598 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo rbd --cluster cluster1 device unmap -t krbd mirror/image-primary8 2024-09-12T13:19:57.879 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ demote_image cluster1 mirror image-primary8 2024-09-12T13:19:57.879 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:19:57.879 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:19:57.879 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:19:57.879 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 mirror image demote mirror/image-primary8 2024-09-12T13:19:58.061 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary10 up+unknown '' '' 2024-09-12T13:19:58.061 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:19:58.061 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:19:58.062 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:19:58.062 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:19:58.062 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:19:58.062 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:19:58.062 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:19:58.064 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:19:58.065 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary10 2024-09-12T13:19:58.065 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary5 up+unknown '' '' 2024-09-12T13:19:58.066 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:19:58.066 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:19:58.066 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:19:58.066 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:19:58.066 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:19:58.067 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:19:58.067 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:19:58.068 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:19:58.068 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary5 2024-09-12T13:19:58.211 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary5: 2024-09-12T13:19:58.211 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: cfd4ea21-2703-4ce9-a4e3-e41c240a1165 2024-09-12T13:19:58.211 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:19:58.211 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:19:58.211 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:19:58.211 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:19:47' 2024-09-12T13:19:58.211 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:19:58.215 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:19:58.216 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:19:58.216 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 2 2024-09-12T13:19:58.243 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary10: 2024-09-12T13:19:58.243 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: a41c1371-bc6b-4e5a-b8a6-2d2252dfda67 2024-09-12T13:19:58.243 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:19:58.243 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:19:58.243 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:19:58.243 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:19:47' 2024-09-12T13:19:58.243 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:19:58.246 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:19:58.246 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:19:58.247 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 2 2024-09-12T13:19:58.259 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary7 up+unknown '' '' 2024-09-12T13:19:58.259 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:19:58.260 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:19:58.260 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:19:58.260 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:19:58.260 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:19:58.260 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:19:58.260 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:19:58.260 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:19:58.261 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary7 2024-09-12T13:19:58.334 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary7: 2024-09-12T13:19:58.334 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 3f634cfd-f07a-4271-8c29-0ad3777a474c 2024-09-12T13:19:58.334 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:19:58.335 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:19:58.335 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:19:58.335 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:19:47' 2024-09-12T13:19:58.335 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:19:58.337 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:19:58.337 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:19:58.337 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 4 2024-09-12T13:19:59.036 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Image demoted to non-primary 2024-09-12T13:19:59.093 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary8 up+unknown 2024-09-12T13:19:59.093 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:19:59.093 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:19:59.093 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:19:59.093 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:19:59.093 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:19:59.093 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:19:59.093 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:19:59.093 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:19:59.166 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ demote_md5=b639770233b76ecd8e91005571f6041c 2024-09-12T13:19:59.167 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo rbd --cluster cluster1 device unmap -t krbd mirror/image-primary1 2024-09-12T13:19:59.274 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary3 up+unknown '' '' 2024-09-12T13:19:59.274 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:19:59.274 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:19:59.274 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:19:59.274 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:19:59.274 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:19:59.275 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:19:59.275 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:19:59.275 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:19:59.275 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary3 2024-09-12T13:19:59.285 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ demote_image cluster1 mirror image-primary1 2024-09-12T13:19:59.285 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:19:59.285 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:19:59.285 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:19:59.285 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 mirror image demote mirror/image-primary1 2024-09-12T13:19:59.350 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary3: 2024-09-12T13:19:59.350 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 50e30e91-e1e0-44f3-8ad3-6fb3a5307bba 2024-09-12T13:19:59.350 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:19:59.350 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:19:59.350 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:19:59.350 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:19:47' 2024-09-12T13:19:59.350 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:19:59.352 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:19:59.352 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:19:59.352 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 4 2024-09-12T13:19:59.991 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Image demoted to non-primary 2024-09-12T13:19:59.994 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Image demoted to non-primary 2024-09-12T13:20:00.015 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary9 up+unknown 2024-09-12T13:20:00.016 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:20:00.016 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:20:00.016 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary2 up+unknown 2024-09-12T13:20:00.016 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:20:00.016 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:20:00.017 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:20:00.017 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:20:00.017 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:20:00.017 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:20:00.017 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:20:00.017 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:20:00.017 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:20:00.017 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:20:00.017 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:20:00.017 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:20:00.017 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:20:00.017 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:20:00.098 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary8 up+unknown '' '' 2024-09-12T13:20:00.099 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:20:00.099 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:20:00.099 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:20:00.099 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:20:00.099 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:20:00.099 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:20:00.099 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:20:00.099 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:20:00.099 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary8 2024-09-12T13:20:00.172 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary8: 2024-09-12T13:20:00.172 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 27e6f3f9-b993-4ad3-a6a6-74e8fc4feff4 2024-09-12T13:20:00.172 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:20:00.172 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:20:00.172 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:20:00.173 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:19:47' 2024-09-12T13:20:00.173 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:20:00.174 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:20:00.174 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:20:00.174 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 2 2024-09-12T13:20:00.221 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary5 up+unknown '' '' 2024-09-12T13:20:00.222 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:20:00.222 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:20:00.222 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:20:00.222 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:20:00.222 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:20:00.222 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:20:00.222 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:20:00.222 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:20:00.222 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary5 2024-09-12T13:20:00.249 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary10 up+unknown '' '' 2024-09-12T13:20:00.249 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:20:00.249 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:20:00.249 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:20:00.249 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:20:00.250 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:20:00.250 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:20:00.250 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:20:00.250 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:20:00.250 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary10 2024-09-12T13:20:00.293 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary5: 2024-09-12T13:20:00.293 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: cfd4ea21-2703-4ce9-a4e3-e41c240a1165 2024-09-12T13:20:00.293 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:20:00.293 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:20:00.293 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:20:00.294 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:19:47' 2024-09-12T13:20:00.294 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:20:00.296 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:20:00.296 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:20:00.296 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 4 2024-09-12T13:20:00.321 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary10: 2024-09-12T13:20:00.321 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: a41c1371-bc6b-4e5a-b8a6-2d2252dfda67 2024-09-12T13:20:00.321 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:20:00.322 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:20:00.322 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:20:00.322 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:19:47' 2024-09-12T13:20:00.322 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:20:00.324 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:20:00.324 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:20:00.324 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 4 2024-09-12T13:20:00.979 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Image demoted to non-primary 2024-09-12T13:20:00.991 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Image demoted to non-primary 2024-09-12T13:20:00.994 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Image demoted to non-primary 2024-09-12T13:20:00.998 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary4 up+unknown 2024-09-12T13:20:00.998 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:20:00.999 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:20:00.999 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:20:00.999 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:20:00.999 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:20:00.999 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:20:00.999 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:20:00.999 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:20:01.010 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary6 up+unknown 2024-09-12T13:20:01.010 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:20:01.011 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:20:01.011 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:20:01.011 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:20:01.011 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:20:01.011 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:20:01.011 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:20:01.011 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:20:01.019 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary1 up+unknown 2024-09-12T13:20:01.019 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:20:01.020 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:20:01.020 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:20:01.020 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:20:01.020 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:20:01.020 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:20:01.020 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:20:01.020 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:20:01.021 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary2 up+unknown '' '' 2024-09-12T13:20:01.021 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:20:01.021 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:20:01.021 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:20:01.021 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:20:01.021 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:20:01.021 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:20:01.021 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:20:01.022 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:20:01.022 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary2 2024-09-12T13:20:01.025 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary9 up+unknown '' '' 2024-09-12T13:20:01.025 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:20:01.025 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:20:01.025 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:20:01.025 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:20:01.025 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:20:01.025 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:20:01.025 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:20:01.026 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:20:01.027 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary9 2024-09-12T13:20:01.107 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary2: 2024-09-12T13:20:01.107 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 3ceee0af-566c-4f40-8fbb-b3b44ed7d010 2024-09-12T13:20:01.107 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:20:01.107 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:20:01.107 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:20:01.107 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:19:47' 2024-09-12T13:20:01.107 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:20:01.110 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:20:01.110 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:20:01.110 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 2 2024-09-12T13:20:01.118 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary9: 2024-09-12T13:20:01.118 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: b8a82bf7-89cf-4fad-b167-0553ebb15a5e 2024-09-12T13:20:01.118 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:20:01.118 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:20:01.118 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:20:01.118 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:19:47' 2024-09-12T13:20:01.119 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:20:01.121 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:20:01.121 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:20:01.121 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 2 2024-09-12T13:20:02.002 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary4 up+unknown '' '' 2024-09-12T13:20:02.002 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:20:02.002 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:20:02.002 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:20:02.002 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:20:02.002 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:20:02.002 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:20:02.002 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:20:02.002 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:20:02.003 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary4 2024-09-12T13:20:02.016 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary6 up+unknown '' '' 2024-09-12T13:20:02.016 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:20:02.016 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:20:02.016 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:20:02.016 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:20:02.016 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:20:02.016 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:20:02.017 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:20:02.017 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:20:02.017 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary6 2024-09-12T13:20:02.023 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary1 up+unknown '' '' 2024-09-12T13:20:02.023 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:20:02.023 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:20:02.023 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:20:02.023 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:20:02.023 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:20:02.024 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:20:02.024 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:20:02.024 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:20:02.024 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary1 2024-09-12T13:20:02.082 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary4: 2024-09-12T13:20:02.083 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 3d37a1b1-ea15-4b8b-8894-ae4f8b6c65cd 2024-09-12T13:20:02.083 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:20:02.083 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:20:02.083 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:20:02.083 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:19:47' 2024-09-12T13:20:02.083 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:20:02.085 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:20:02.085 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:20:02.085 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 2 2024-09-12T13:20:02.095 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary6: 2024-09-12T13:20:02.095 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 14efbe85-d4d9-42ff-aa35-0963a8746f30 2024-09-12T13:20:02.095 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:20:02.095 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:20:02.095 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:20:02.095 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:19:47' 2024-09-12T13:20:02.095 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:20:02.098 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:20:02.098 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:20:02.098 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 2 2024-09-12T13:20:02.102 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary1: 2024-09-12T13:20:02.103 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 96ef9ec0-0a45-4e60-bcf0-79d28853db5e 2024-09-12T13:20:02.103 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:20:02.103 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:20:02.103 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:20:02.103 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:19:47' 2024-09-12T13:20:02.103 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:20:02.105 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:20:02.105 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:20:02.105 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 2 2024-09-12T13:20:02.177 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary8 up+unknown '' '' 2024-09-12T13:20:02.177 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:20:02.177 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:20:02.177 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:20:02.178 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:20:02.178 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:20:02.178 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:20:02.178 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:20:02.178 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:20:02.178 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary8 2024-09-12T13:20:02.254 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary8: 2024-09-12T13:20:02.254 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 27e6f3f9-b993-4ad3-a6a6-74e8fc4feff4 2024-09-12T13:20:02.254 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:20:02.254 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:20:02.254 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:20:02.254 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:19:47' 2024-09-12T13:20:02.255 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:20:02.257 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:20:02.257 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:20:02.257 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 4 2024-09-12T13:20:02.339 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary7 up+unknown '' '' 2024-09-12T13:20:02.340 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:20:02.340 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:20:02.340 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:20:02.340 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:20:02.340 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:20:02.340 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:20:02.340 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:20:02.341 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:20:02.341 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary7 2024-09-12T13:20:02.418 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary7: 2024-09-12T13:20:02.418 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 3f634cfd-f07a-4271-8c29-0ad3777a474c 2024-09-12T13:20:02.418 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:20:02.419 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:20:02.419 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:20:02.419 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:19:47' 2024-09-12T13:20:02.419 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:20:02.421 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:20:02.421 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:20:02.421 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 8 2024-09-12T13:20:03.113 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary2 up+unknown '' '' 2024-09-12T13:20:03.113 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:20:03.113 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:20:03.114 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:20:03.114 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:20:03.114 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:20:03.114 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:20:03.114 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:20:03.114 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:20:03.114 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary2 2024-09-12T13:20:03.123 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary9 up+unknown '' '' 2024-09-12T13:20:03.123 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:20:03.123 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:20:03.123 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:20:03.123 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:20:03.123 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:20:03.123 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:20:03.123 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:20:03.124 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:20:03.124 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary9 2024-09-12T13:20:03.194 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary2: 2024-09-12T13:20:03.194 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 3ceee0af-566c-4f40-8fbb-b3b44ed7d010 2024-09-12T13:20:03.195 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:20:03.195 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:20:03.195 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:20:03.195 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:19:47' 2024-09-12T13:20:03.195 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:20:03.197 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:20:03.197 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:20:03.197 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 4 2024-09-12T13:20:03.199 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary9: 2024-09-12T13:20:03.199 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: b8a82bf7-89cf-4fad-b167-0553ebb15a5e 2024-09-12T13:20:03.199 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:20:03.199 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:20:03.199 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:20:03.199 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:19:47' 2024-09-12T13:20:03.199 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:20:03.202 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:20:03.202 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:20:03.202 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 4 2024-09-12T13:20:03.355 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary3 up+unknown '' '' 2024-09-12T13:20:03.356 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:20:03.356 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:20:03.356 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:20:03.356 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:20:03.356 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:20:03.356 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:20:03.356 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:20:03.357 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:20:03.357 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary3 2024-09-12T13:20:03.433 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary3: 2024-09-12T13:20:03.433 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 50e30e91-e1e0-44f3-8ad3-6fb3a5307bba 2024-09-12T13:20:03.433 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:20:03.433 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:20:03.433 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:20:03.433 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:19:47' 2024-09-12T13:20:03.433 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:20:03.435 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:20:03.436 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:20:03.436 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 8 2024-09-12T13:20:04.088 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary4 up+unknown '' '' 2024-09-12T13:20:04.088 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:20:04.088 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:20:04.088 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:20:04.088 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:20:04.088 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:20:04.088 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:20:04.088 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:20:04.089 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:20:04.089 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary4 2024-09-12T13:20:04.100 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary6 up+unknown '' '' 2024-09-12T13:20:04.100 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:20:04.100 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:20:04.101 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:20:04.101 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:20:04.101 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:20:04.101 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:20:04.101 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:20:04.101 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:20:04.101 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary6 2024-09-12T13:20:04.107 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary1 up+unknown '' '' 2024-09-12T13:20:04.107 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:20:04.107 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:20:04.107 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:20:04.107 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:20:04.107 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:20:04.108 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:20:04.108 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:20:04.108 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:20:04.108 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary1 2024-09-12T13:20:04.169 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary4: 2024-09-12T13:20:04.169 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 3d37a1b1-ea15-4b8b-8894-ae4f8b6c65cd 2024-09-12T13:20:04.169 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:20:04.169 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:20:04.169 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:20:04.169 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:19:47' 2024-09-12T13:20:04.169 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:20:04.172 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:20:04.172 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:20:04.172 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 4 2024-09-12T13:20:04.175 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary6: 2024-09-12T13:20:04.175 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 14efbe85-d4d9-42ff-aa35-0963a8746f30 2024-09-12T13:20:04.175 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:20:04.175 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:20:04.175 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:20:04.176 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:19:47' 2024-09-12T13:20:04.176 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:20:04.178 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:20:04.178 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:20:04.178 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 4 2024-09-12T13:20:04.185 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary1: 2024-09-12T13:20:04.185 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 96ef9ec0-0a45-4e60-bcf0-79d28853db5e 2024-09-12T13:20:04.185 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:20:04.185 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:20:04.185 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:20:04.186 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:19:47' 2024-09-12T13:20:04.186 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:20:04.188 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:20:04.188 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:20:04.188 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 4 2024-09-12T13:20:04.298 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary5 up+unknown '' '' 2024-09-12T13:20:04.298 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:20:04.299 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:20:04.299 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:20:04.299 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:20:04.299 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:20:04.299 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:20:04.299 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:20:04.299 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:20:04.299 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary5 2024-09-12T13:20:04.327 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary10 up+unknown '' '' 2024-09-12T13:20:04.327 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:20:04.327 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:20:04.327 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:20:04.327 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:20:04.327 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:20:04.327 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:20:04.327 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:20:04.328 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:20:04.328 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary10 2024-09-12T13:20:04.374 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary5: 2024-09-12T13:20:04.374 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: cfd4ea21-2703-4ce9-a4e3-e41c240a1165 2024-09-12T13:20:04.374 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:20:04.375 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:20:04.375 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:20:04.375 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:19:47' 2024-09-12T13:20:04.375 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:20:04.377 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:20:04.377 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:20:04.377 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 8 2024-09-12T13:20:04.399 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary10: 2024-09-12T13:20:04.399 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: a41c1371-bc6b-4e5a-b8a6-2d2252dfda67 2024-09-12T13:20:04.399 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:20:04.399 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:20:04.399 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:20:04.399 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:19:47' 2024-09-12T13:20:04.399 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:20:04.402 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:20:04.402 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:20:04.402 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 8 2024-09-12T13:20:06.259 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary8 up+unknown '' '' 2024-09-12T13:20:06.260 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:20:06.260 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:20:06.260 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:20:06.260 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:20:06.260 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:20:06.260 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:20:06.260 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:20:06.260 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:20:06.261 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary8 2024-09-12T13:20:06.347 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary8: 2024-09-12T13:20:06.347 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 27e6f3f9-b993-4ad3-a6a6-74e8fc4feff4 2024-09-12T13:20:06.347 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:20:06.347 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:20:06.347 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:20:06.347 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:19:47' 2024-09-12T13:20:06.347 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:20:06.350 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:20:06.350 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:20:06.350 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 8 2024-09-12T13:20:07.200 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary2 up+unknown '' '' 2024-09-12T13:20:07.200 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:20:07.200 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:20:07.200 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:20:07.201 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:20:07.201 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:20:07.201 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:20:07.201 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:20:07.201 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:20:07.201 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary2 2024-09-12T13:20:07.204 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary9 up+unknown '' '' 2024-09-12T13:20:07.204 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:20:07.204 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:20:07.204 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:20:07.204 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:20:07.204 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:20:07.205 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:20:07.205 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:20:07.205 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:20:07.205 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary9 2024-09-12T13:20:07.284 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary2: 2024-09-12T13:20:07.284 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 3ceee0af-566c-4f40-8fbb-b3b44ed7d010 2024-09-12T13:20:07.284 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:20:07.285 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:20:07.285 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:20:07.285 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:19:47' 2024-09-12T13:20:07.285 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:20:07.285 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary9: 2024-09-12T13:20:07.285 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: b8a82bf7-89cf-4fad-b167-0553ebb15a5e 2024-09-12T13:20:07.285 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:20:07.285 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:20:07.285 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:20:07.285 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:19:47' 2024-09-12T13:20:07.285 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:20:07.287 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:20:07.287 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:20:07.287 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:20:07.287 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:20:07.288 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 8 2024-09-12T13:20:07.288 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 8 2024-09-12T13:20:08.175 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary4 up+unknown '' '' 2024-09-12T13:20:08.175 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:20:08.175 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:20:08.175 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:20:08.175 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:20:08.176 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:20:08.176 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:20:08.176 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:20:08.176 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:20:08.176 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary4 2024-09-12T13:20:08.180 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary6 up+unknown '' '' 2024-09-12T13:20:08.181 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:20:08.181 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:20:08.181 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:20:08.181 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:20:08.181 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:20:08.181 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:20:08.181 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:20:08.181 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:20:08.182 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary6 2024-09-12T13:20:08.190 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary1 up+unknown '' '' 2024-09-12T13:20:08.190 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:20:08.190 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:20:08.190 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:20:08.190 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:20:08.191 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:20:08.191 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:20:08.191 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:20:08.191 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:20:08.191 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary1 2024-09-12T13:20:08.264 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary4: 2024-09-12T13:20:08.264 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 3d37a1b1-ea15-4b8b-8894-ae4f8b6c65cd 2024-09-12T13:20:08.265 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:20:08.265 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:20:08.265 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:20:08.265 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:19:47' 2024-09-12T13:20:08.265 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:20:08.265 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary6: 2024-09-12T13:20:08.265 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 14efbe85-d4d9-42ff-aa35-0963a8746f30 2024-09-12T13:20:08.266 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:20:08.266 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:20:08.266 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:20:08.266 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:19:47' 2024-09-12T13:20:08.266 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:20:08.268 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:20:08.268 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:20:08.268 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 8 2024-09-12T13:20:08.268 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:20:08.268 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:20:08.268 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 8 2024-09-12T13:20:08.274 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary1: 2024-09-12T13:20:08.274 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 96ef9ec0-0a45-4e60-bcf0-79d28853db5e 2024-09-12T13:20:08.274 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:20:08.275 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:20:08.275 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:20:08.275 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:19:47' 2024-09-12T13:20:08.275 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:20:08.277 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:20:08.277 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:20:08.277 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 8 2024-09-12T13:20:10.424 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary7 up+unknown '' '' 2024-09-12T13:20:10.424 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:20:10.424 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:20:10.424 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:20:10.424 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:20:10.424 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:20:10.424 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:20:10.424 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:20:10.425 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:20:10.425 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary7 2024-09-12T13:20:10.507 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary7: 2024-09-12T13:20:10.507 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 3f634cfd-f07a-4271-8c29-0ad3777a474c 2024-09-12T13:20:10.507 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:20:10.507 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:20:10.507 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:20:10.508 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:19:47' 2024-09-12T13:20:10.508 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:20:10.510 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:20:10.510 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:20:10.510 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 8 2024-09-12T13:20:11.438 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary3 up+unknown '' '' 2024-09-12T13:20:11.438 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:20:11.438 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:20:11.438 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:20:11.438 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:20:11.439 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:20:11.439 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:20:11.439 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:20:11.439 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:20:11.439 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary3 2024-09-12T13:20:11.520 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary3: 2024-09-12T13:20:11.520 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 50e30e91-e1e0-44f3-8ad3-6fb3a5307bba 2024-09-12T13:20:11.520 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:20:11.521 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:20:11.521 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:20:11.521 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:19:47' 2024-09-12T13:20:11.521 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:20:11.523 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:20:11.523 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:20:11.523 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 8 2024-09-12T13:20:12.379 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary5 up+unknown '' '' 2024-09-12T13:20:12.379 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:20:12.379 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:20:12.380 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:20:12.380 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:20:12.380 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:20:12.380 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:20:12.380 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:20:12.380 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:20:12.380 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary5 2024-09-12T13:20:12.404 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary10 up+unknown '' '' 2024-09-12T13:20:12.404 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:20:12.404 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:20:12.405 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:20:12.405 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:20:12.405 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:20:12.405 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:20:12.405 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:20:12.405 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:20:12.405 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary10 2024-09-12T13:20:12.461 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary5: 2024-09-12T13:20:12.461 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: cfd4ea21-2703-4ce9-a4e3-e41c240a1165 2024-09-12T13:20:12.461 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:20:12.461 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:20:12.461 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:20:12.462 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:19:47' 2024-09-12T13:20:12.462 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:20:12.464 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:20:12.464 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:20:12.464 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 8 2024-09-12T13:20:12.482 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary10: 2024-09-12T13:20:12.482 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: a41c1371-bc6b-4e5a-b8a6-2d2252dfda67 2024-09-12T13:20:12.482 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:20:12.482 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:20:12.482 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:20:12.482 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:19:47' 2024-09-12T13:20:12.482 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:20:12.485 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:20:12.485 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:20:12.485 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 8 2024-09-12T13:20:14.353 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary8 up+unknown '' '' 2024-09-12T13:20:14.353 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:20:14.353 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:20:14.353 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:20:14.353 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:20:14.353 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:20:14.353 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:20:14.354 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:20:14.354 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:20:14.354 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary8 2024-09-12T13:20:14.441 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary8: 2024-09-12T13:20:14.441 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 27e6f3f9-b993-4ad3-a6a6-74e8fc4feff4 2024-09-12T13:20:14.441 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:20:14.442 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:20:14.442 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:20:14.442 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:19:47' 2024-09-12T13:20:14.442 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:20:14.444 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:20:14.444 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:20:14.444 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 8 2024-09-12T13:20:15.291 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary9 up+unknown '' '' 2024-09-12T13:20:15.291 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary2 up+unknown '' '' 2024-09-12T13:20:15.291 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:20:15.291 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:20:15.291 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:20:15.291 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:20:15.291 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:20:15.291 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:20:15.291 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:20:15.291 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:20:15.292 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:20:15.292 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:20:15.292 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:20:15.292 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:20:15.292 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:20:15.292 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:20:15.292 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:20:15.292 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary9 2024-09-12T13:20:15.293 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:20:15.293 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary2 2024-09-12T13:20:15.387 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary9: 2024-09-12T13:20:15.387 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: b8a82bf7-89cf-4fad-b167-0553ebb15a5e 2024-09-12T13:20:15.387 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:20:15.387 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:20:15.387 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:20:15.387 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:19:47' 2024-09-12T13:20:15.387 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:20:15.387 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary2: 2024-09-12T13:20:15.387 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 3ceee0af-566c-4f40-8fbb-b3b44ed7d010 2024-09-12T13:20:15.387 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:20:15.387 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:20:15.387 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:20:15.387 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:19:47' 2024-09-12T13:20:15.388 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:20:15.389 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:20:15.389 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:20:15.389 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 8 2024-09-12T13:20:15.389 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:20:15.389 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:20:15.389 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 8 2024-09-12T13:20:16.271 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary4 up+unknown '' '' 2024-09-12T13:20:16.271 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary6 up+unknown '' '' 2024-09-12T13:20:16.271 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:20:16.271 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:20:16.271 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:20:16.271 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:20:16.271 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:20:16.271 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:20:16.271 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:20:16.272 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:20:16.272 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:20:16.272 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:20:16.272 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:20:16.272 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:20:16.272 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:20:16.272 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:20:16.272 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:20:16.273 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary4 2024-09-12T13:20:16.273 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:20:16.273 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary6 2024-09-12T13:20:16.280 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary1 up+unknown '' '' 2024-09-12T13:20:16.280 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:20:16.280 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:20:16.280 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:20:16.280 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:20:16.280 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:20:16.280 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:20:16.280 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:20:16.281 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:20:16.281 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary1 2024-09-12T13:20:16.361 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary6: 2024-09-12T13:20:16.362 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 14efbe85-d4d9-42ff-aa35-0963a8746f30 2024-09-12T13:20:16.362 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:20:16.362 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:20:16.362 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:20:16.362 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:19:47' 2024-09-12T13:20:16.362 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:20:16.364 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:20:16.365 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:20:16.365 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 8 2024-09-12T13:20:16.365 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary1: 2024-09-12T13:20:16.365 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 96ef9ec0-0a45-4e60-bcf0-79d28853db5e 2024-09-12T13:20:16.365 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:20:16.365 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:20:16.365 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:20:16.365 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:19:47' 2024-09-12T13:20:16.366 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:20:16.366 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary4: 2024-09-12T13:20:16.366 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 3d37a1b1-ea15-4b8b-8894-ae4f8b6c65cd 2024-09-12T13:20:16.366 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:20:16.366 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:20:16.366 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:20:16.367 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:19:47' 2024-09-12T13:20:16.367 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:20:16.368 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:20:16.368 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:20:16.368 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 8 2024-09-12T13:20:16.369 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:20:16.369 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:20:16.369 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 8 2024-09-12T13:20:18.513 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary7 up+unknown '' '' 2024-09-12T13:20:18.513 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:20:18.513 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:20:18.513 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:20:18.514 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:20:18.514 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:20:18.514 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:20:18.514 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:20:18.514 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:20:18.514 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary7 2024-09-12T13:20:18.600 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary7: 2024-09-12T13:20:18.600 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 3f634cfd-f07a-4271-8c29-0ad3777a474c 2024-09-12T13:20:18.600 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+unknown 2024-09-12T13:20:18.600 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: remote image is not primary 2024-09-12T13:20:18.600 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:20:18.600 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:20:17' 2024-09-12T13:20:18.600 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:20:18.603 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+unknown 2024-09-12T13:20:18.604 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:20:18.606 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: remote image is not primary 2024-09-12T13:20:18.606 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:20:18.607 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:20:18.607 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:20:18.609 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:20:18.609 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:20:18.612 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:20:18.612 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:20:18.613 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:20:18.617 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:20:17' 2024-09-12T13:20:18.617 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary7 up+unknown '2024-09-12 13:20:17' 2024-09-12T13:20:18.617 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:20:18.617 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:20:18.617 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:20:18.617 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:20:18.617 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:20:17' 2024-09-12T13:20:18.617 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:20:18.618 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:20:18.618 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-09-12T13:20:18.726 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041934232mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary196ef9ec0-0a45-4e60-bcf0-79d28853db5eup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:20:17image-primary10a41c1371-bc6b-4e5a-b8a6-2d2252dfda67up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:20:17image-primary23ceee0af-566c-4f40-8fbb-b3b44ed7d010up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:20:17image-primary350e30e91-e1e0-44f3-8ad3-6fb3a5307bbaup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:20:17image-primary43d37a1b1-ea15-4b8b-8894-ae4f8b6c65cdup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:20:17image-primary5cfd4ea21-2703-4ce9-a4e3-e41c240a1165up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:20:17image-primary614efbe85-d4d9-42ff-aa35-0963a8746f30up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:20:17image-primary73f634cfd-f07a-4271-8c29-0ad3777a474cup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:20:17image-primary827e6f3f9-b993-4ad3-a6a6-74e8fc4feff4up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:20:17image-primary9b8a82bf7-89cf-4fad-b167-0553ebb15a5eup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:20:17' 2024-09-12T13:20:18.727 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:20:18.727 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary7'\'']/last_update' 2024-09-12T13:20:18.778 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:20:17' 2024-09-12T13:20:18.780 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary7'\'']/state' 2024-09-12T13:20:18.783 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+unknown 2024-09-12T13:20:18.784 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:20:18.785 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+unknown 2024-09-12T13:20:18.787 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:20:18.788 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:20:18.788 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:20:18.788 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster2 mirror image-primary7 up+unknown 2024-09-12T13:20:18.788 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:20:18.788 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:20:18.788 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:20:18.788 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:20:18.788 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:20:18.788 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:20:18.788 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:20:18.788 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:20:19.526 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary3 up+unknown '' '' 2024-09-12T13:20:19.526 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:20:19.526 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:20:19.526 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:20:19.526 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:20:19.526 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:20:19.526 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:20:19.526 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:20:19.527 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:20:19.527 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary3 2024-09-12T13:20:19.606 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary3: 2024-09-12T13:20:19.606 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 50e30e91-e1e0-44f3-8ad3-6fb3a5307bba 2024-09-12T13:20:19.606 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+unknown 2024-09-12T13:20:19.606 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: remote image is not primary 2024-09-12T13:20:19.606 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:20:19.606 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:20:17' 2024-09-12T13:20:19.607 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:20:19.609 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+unknown 2024-09-12T13:20:19.609 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:20:19.611 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: remote image is not primary 2024-09-12T13:20:19.611 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:20:19.612 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:20:19.613 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:20:19.614 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:20:19.615 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:20:19.617 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:20:19.617 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:20:19.618 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:20:19.622 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:20:17' 2024-09-12T13:20:19.622 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary3 up+unknown '2024-09-12 13:20:17' 2024-09-12T13:20:19.622 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:20:19.622 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:20:19.622 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:20:19.622 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:20:19.623 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:20:17' 2024-09-12T13:20:19.623 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:20:19.624 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:20:19.624 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-09-12T13:20:19.729 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041934232mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary196ef9ec0-0a45-4e60-bcf0-79d28853db5eup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:20:17image-primary10a41c1371-bc6b-4e5a-b8a6-2d2252dfda67up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:20:17image-primary23ceee0af-566c-4f40-8fbb-b3b44ed7d010up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:20:17image-primary350e30e91-e1e0-44f3-8ad3-6fb3a5307bbaup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:20:17image-primary43d37a1b1-ea15-4b8b-8894-ae4f8b6c65cdup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:20:17image-primary5cfd4ea21-2703-4ce9-a4e3-e41c240a1165up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:20:17image-primary614efbe85-d4d9-42ff-aa35-0963a8746f30up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:20:17image-primary73f634cfd-f07a-4271-8c29-0ad3777a474cup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:20:17image-primary827e6f3f9-b993-4ad3-a6a6-74e8fc4feff4up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:20:17image-primary9b8a82bf7-89cf-4fad-b167-0553ebb15a5eup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:20:17' 2024-09-12T13:20:19.729 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:20:19.730 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary3'\'']/last_update' 2024-09-12T13:20:19.733 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:20:17' 2024-09-12T13:20:19.734 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary3'\'']/state' 2024-09-12T13:20:19.738 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+unknown 2024-09-12T13:20:19.739 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:20:19.739 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+unknown 2024-09-12T13:20:19.741 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:20:19.741 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:20:19.742 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:20:19.742 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster2 mirror image-primary3 up+unknown 2024-09-12T13:20:19.742 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:20:19.742 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:20:19.742 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:20:19.742 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:20:19.742 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:20:19.742 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:20:19.742 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:20:19.742 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:20:19.789 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster2 mirror image-primary7 up+unknown '' '' 2024-09-12T13:20:19.789 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:20:19.789 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:20:19.789 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:20:19.789 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:20:19.789 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:20:19.790 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:20:19.790 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:20:19.790 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:20:19.790 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster2 mirror image status mirror/image-primary7 2024-09-12T13:20:19.853 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary7: 2024-09-12T13:20:19.853 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 3f634cfd-f07a-4271-8c29-0ad3777a474c 2024-09-12T13:20:19.853 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+unknown 2024-09-12T13:20:19.853 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: remote image is not primary 2024-09-12T13:20:19.853 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:20:19.853 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:20:17' 2024-09-12T13:20:19.853 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:20:19.855 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+unknown 2024-09-12T13:20:19.855 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:20:19.858 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: remote image is not primary 2024-09-12T13:20:19.858 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:20:19.859 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:20:19.859 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:20:19.861 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:20:19.861 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:20:19.864 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:20:19.864 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:20:19.865 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:20:19.869 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:20:17' 2024-09-12T13:20:19.869 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster2 mirror image-primary7 up+unknown '2024-09-12 13:20:17' 2024-09-12T13:20:19.869 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:20:19.869 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:20:19.869 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:20:19.869 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:20:19.869 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:20:17' 2024-09-12T13:20:19.869 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:20:19.870 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:20:19.870 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2024-09-12T13:20:19.968 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041924231mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary196ef9ec0-0a45-4e60-bcf0-79d28853db5eup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:20:17image-primary10a41c1371-bc6b-4e5a-b8a6-2d2252dfda67up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:20:17image-primary23ceee0af-566c-4f40-8fbb-b3b44ed7d010up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:20:17image-primary350e30e91-e1e0-44f3-8ad3-6fb3a5307bbaup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:20:17image-primary43d37a1b1-ea15-4b8b-8894-ae4f8b6c65cdup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:20:17image-primary5cfd4ea21-2703-4ce9-a4e3-e41c240a1165up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:20:17image-primary614efbe85-d4d9-42ff-aa35-0963a8746f30up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:20:17image-primary73f634cfd-f07a-4271-8c29-0ad3777a474cup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:20:17image-primary827e6f3f9-b993-4ad3-a6a6-74e8fc4feff4up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:20:17image-primary9b8a82bf7-89cf-4fad-b167-0553ebb15a5eup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:20:17' 2024-09-12T13:20:19.968 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:20:19.968 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary7'\'']/last_update' 2024-09-12T13:20:19.972 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:20:17' 2024-09-12T13:20:19.973 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary7'\'']/state' 2024-09-12T13:20:19.977 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+unknown 2024-09-12T13:20:19.977 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:20:19.978 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+unknown 2024-09-12T13:20:19.980 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:20:19.980 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:20:19.981 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:20:19.981 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ promote_image cluster2 mirror image-primary7 2024-09-12T13:20:19.981 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:20:19.981 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:20:19.981 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:20:19.981 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local force= 2024-09-12T13:20:19.981 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 mirror image promote mirror/image-primary7 2024-09-12T13:20:20.467 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary5 up+unknown '' '' 2024-09-12T13:20:20.467 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:20:20.467 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:20:20.467 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:20:20.467 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:20:20.467 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:20:20.468 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:20:20.468 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:20:20.468 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:20:20.468 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary5 2024-09-12T13:20:20.487 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary10 up+unknown '' '' 2024-09-12T13:20:20.487 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:20:20.487 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:20:20.487 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:20:20.488 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:20:20.488 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:20:20.488 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:20:20.488 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:20:20.488 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:20:20.488 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary10 2024-09-12T13:20:20.550 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary5: 2024-09-12T13:20:20.550 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: cfd4ea21-2703-4ce9-a4e3-e41c240a1165 2024-09-12T13:20:20.551 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+unknown 2024-09-12T13:20:20.551 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: remote image is not primary 2024-09-12T13:20:20.551 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:20:20.551 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:20:17' 2024-09-12T13:20:20.551 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:20:20.553 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+unknown 2024-09-12T13:20:20.553 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:20:20.556 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: remote image is not primary 2024-09-12T13:20:20.556 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:20:20.556 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:20:20.557 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:20:20.559 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:20:20.559 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:20:20.562 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:20:20.562 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:20:20.563 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:20:20.564 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary10: 2024-09-12T13:20:20.564 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: a41c1371-bc6b-4e5a-b8a6-2d2252dfda67 2024-09-12T13:20:20.564 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+unknown 2024-09-12T13:20:20.564 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: remote image is not primary 2024-09-12T13:20:20.564 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:20:20.564 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:20:17' 2024-09-12T13:20:20.564 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:20:20.566 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+unknown 2024-09-12T13:20:20.567 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:20:17' 2024-09-12T13:20:20.567 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary5 up+unknown '2024-09-12 13:20:17' 2024-09-12T13:20:20.567 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:20:20.567 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:20:20.567 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:20:20.567 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:20:20.568 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:20:17' 2024-09-12T13:20:20.568 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:20:20.568 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:20:20.568 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:20:20.568 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-09-12T13:20:20.569 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: remote image is not primary 2024-09-12T13:20:20.569 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:20:20.570 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:20:20.570 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:20:20.572 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:20:20.572 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:20:20.575 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:20:20.575 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:20:20.576 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:20:20.580 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:20:17' 2024-09-12T13:20:20.580 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary10 up+unknown '2024-09-12 13:20:17' 2024-09-12T13:20:20.580 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:20:20.580 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:20:20.580 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:20:20.580 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:20:20.580 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:20:17' 2024-09-12T13:20:20.580 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:20:20.581 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:20:20.581 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-09-12T13:20:20.684 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041934232mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary196ef9ec0-0a45-4e60-bcf0-79d28853db5eup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:20:17image-primary10a41c1371-bc6b-4e5a-b8a6-2d2252dfda67up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:20:17image-primary23ceee0af-566c-4f40-8fbb-b3b44ed7d010up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:20:17image-primary350e30e91-e1e0-44f3-8ad3-6fb3a5307bbaup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:20:17image-primary43d37a1b1-ea15-4b8b-8894-ae4f8b6c65cdup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:20:17image-primary5cfd4ea21-2703-4ce9-a4e3-e41c240a1165up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:20:17image-primary614efbe85-d4d9-42ff-aa35-0963a8746f30up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:20:17image-primary73f634cfd-f07a-4271-8c29-0ad3777a474cup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:20:17image-primary827e6f3f9-b993-4ad3-a6a6-74e8fc4feff4up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:20:17image-primary9b8a82bf7-89cf-4fad-b167-0553ebb15a5eup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:20:17' 2024-09-12T13:20:20.685 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:20:20.685 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary5'\'']/last_update' 2024-09-12T13:20:20.689 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:20:17' 2024-09-12T13:20:20.690 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary5'\'']/state' 2024-09-12T13:20:20.695 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+unknown 2024-09-12T13:20:20.696 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:20:20.696 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+unknown 2024-09-12T13:20:20.698 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:20:20.699 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041934232mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary196ef9ec0-0a45-4e60-bcf0-79d28853db5eup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:20:17image-primary10a41c1371-bc6b-4e5a-b8a6-2d2252dfda67up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:20:17image-primary23ceee0af-566c-4f40-8fbb-b3b44ed7d010up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:20:17image-primary350e30e91-e1e0-44f3-8ad3-6fb3a5307bbaup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:20:17image-primary43d37a1b1-ea15-4b8b-8894-ae4f8b6c65cdup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:20:17image-primary5cfd4ea21-2703-4ce9-a4e3-e41c240a1165up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:20:17image-primary614efbe85-d4d9-42ff-aa35-0963a8746f30up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:20:17image-primary73f634cfd-f07a-4271-8c29-0ad3777a474cup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:20:17image-primary827e6f3f9-b993-4ad3-a6a6-74e8fc4feff4up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:20:17image-primary9b8a82bf7-89cf-4fad-b167-0553ebb15a5eup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:20:17' 2024-09-12T13:20:20.699 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:20:20.700 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:20:20.700 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:20:20.700 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster2 mirror image-primary5 up+unknown 2024-09-12T13:20:20.700 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:20:20.700 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:20:20.700 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:20:20.700 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:20:20.701 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:20:20.701 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:20:20.701 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:20:20.701 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:20:20.701 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary10'\'']/last_update' 2024-09-12T13:20:20.702 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:20:17' 2024-09-12T13:20:20.703 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary10'\'']/state' 2024-09-12T13:20:20.707 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+unknown 2024-09-12T13:20:20.707 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:20:20.708 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+unknown 2024-09-12T13:20:20.709 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:20:20.710 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:20:20.710 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:20:20.711 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster2 mirror image-primary10 up+unknown 2024-09-12T13:20:20.711 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:20:20.711 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:20:20.711 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:20:20.711 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:20:20.711 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:20:20.711 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:20:20.711 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:20:20.711 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:20:20.744 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster2 mirror image-primary3 up+unknown '' '' 2024-09-12T13:20:20.744 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:20:20.744 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:20:20.744 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:20:20.744 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:20:20.744 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:20:20.744 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:20:20.744 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:20:20.745 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:20:20.745 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster2 mirror image status mirror/image-primary3 2024-09-12T13:20:20.810 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary3: 2024-09-12T13:20:20.810 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 50e30e91-e1e0-44f3-8ad3-6fb3a5307bba 2024-09-12T13:20:20.810 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+unknown 2024-09-12T13:20:20.810 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: remote image is not primary 2024-09-12T13:20:20.810 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:20:20.810 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:20:17' 2024-09-12T13:20:20.810 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:20:20.813 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+unknown 2024-09-12T13:20:20.814 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:20:20.816 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: remote image is not primary 2024-09-12T13:20:20.816 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:20:20.816 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:20:20.817 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:20:20.819 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:20:20.819 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:20:20.821 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:20:20.822 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:20:20.822 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:20:20.826 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:20:17' 2024-09-12T13:20:20.826 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster2 mirror image-primary3 up+unknown '2024-09-12 13:20:17' 2024-09-12T13:20:20.826 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:20:20.826 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:20:20.827 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:20:20.827 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:20:20.827 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:20:17' 2024-09-12T13:20:20.827 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:20:20.827 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:20:20.827 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2024-09-12T13:20:20.926 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041924231mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary196ef9ec0-0a45-4e60-bcf0-79d28853db5eup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:20:17image-primary10a41c1371-bc6b-4e5a-b8a6-2d2252dfda67up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:20:17image-primary23ceee0af-566c-4f40-8fbb-b3b44ed7d010up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:20:17image-primary350e30e91-e1e0-44f3-8ad3-6fb3a5307bbaup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:20:17image-primary43d37a1b1-ea15-4b8b-8894-ae4f8b6c65cdup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:20:17image-primary5cfd4ea21-2703-4ce9-a4e3-e41c240a1165up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:20:17image-primary614efbe85-d4d9-42ff-aa35-0963a8746f30up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:20:17image-primary73f634cfd-f07a-4271-8c29-0ad3777a474cup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:20:17image-primary827e6f3f9-b993-4ad3-a6a6-74e8fc4feff4up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:20:17image-primary9b8a82bf7-89cf-4fad-b167-0553ebb15a5eup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:20:17' 2024-09-12T13:20:20.926 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:20:20.927 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary3'\'']/last_update' 2024-09-12T13:20:20.931 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:20:17' 2024-09-12T13:20:20.931 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary3'\'']/state' 2024-09-12T13:20:20.935 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+unknown 2024-09-12T13:20:20.936 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:20:20.936 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+unknown 2024-09-12T13:20:20.938 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:20:20.939 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:20:20.939 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:20:20.939 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ promote_image cluster2 mirror image-primary3 2024-09-12T13:20:20.939 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:20:20.939 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:20:20.939 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:20:20.940 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local force= 2024-09-12T13:20:20.940 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 mirror image promote mirror/image-primary3 2024-09-12T13:20:21.004 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Image promoted to primary 2024-09-12T13:20:21.015 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \n\b\d ]] 2024-09-12T13:20:21.015 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \k\r\b\d ]] 2024-09-12T13:20:21.016 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo rbd --cluster cluster2 device map -t krbd mirror/image-primary7 2024-09-12T13:20:21.142 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ dev=/dev/rbd0 2024-09-12T13:20:21.143 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo md5sum /dev/rbd0 2024-09-12T13:20:21.143 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ awk '{print $1}' 2024-09-12T13:20:21.700 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster2 mirror image-primary5 up+unknown '' '' 2024-09-12T13:20:21.701 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:20:21.701 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:20:21.701 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:20:21.701 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:20:21.701 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:20:21.701 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:20:21.701 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:20:21.701 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:20:21.701 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster2 mirror image status mirror/image-primary5 2024-09-12T13:20:21.712 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster2 mirror image-primary10 up+unknown '' '' 2024-09-12T13:20:21.713 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:20:21.713 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:20:21.713 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:20:21.713 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:20:21.713 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:20:21.713 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:20:21.713 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:20:21.716 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:20:21.717 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster2 mirror image status mirror/image-primary10 2024-09-12T13:20:21.773 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary5: 2024-09-12T13:20:21.773 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: cfd4ea21-2703-4ce9-a4e3-e41c240a1165 2024-09-12T13:20:21.773 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+unknown 2024-09-12T13:20:21.773 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: remote image is not primary 2024-09-12T13:20:21.773 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:20:21.773 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:20:17' 2024-09-12T13:20:21.773 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:20:21.775 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+unknown 2024-09-12T13:20:21.776 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:20:21.778 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: remote image is not primary 2024-09-12T13:20:21.779 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:20:21.780 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:20:21.780 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:20:21.782 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:20:21.782 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:20:21.785 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:20:21.785 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:20:21.786 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:20:21.787 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary10: 2024-09-12T13:20:21.787 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: a41c1371-bc6b-4e5a-b8a6-2d2252dfda67 2024-09-12T13:20:21.787 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+unknown 2024-09-12T13:20:21.787 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: remote image is not primary 2024-09-12T13:20:21.787 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:20:21.787 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:20:17' 2024-09-12T13:20:21.787 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:20:21.790 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+unknown 2024-09-12T13:20:21.791 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:20:17' 2024-09-12T13:20:21.791 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster2 mirror image-primary5 up+unknown '2024-09-12 13:20:17' 2024-09-12T13:20:21.791 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:20:21.791 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:20:21.791 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:20:21.791 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:20:21.791 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:20:17' 2024-09-12T13:20:21.791 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:20:21.791 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:20:21.791 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:20:21.791 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2024-09-12T13:20:21.793 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: remote image is not primary 2024-09-12T13:20:21.794 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:20:21.795 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:20:21.795 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:20:21.797 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:20:21.797 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:20:21.801 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:20:21.802 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:20:21.802 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:20:21.806 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:20:17' 2024-09-12T13:20:21.806 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster2 mirror image-primary10 up+unknown '2024-09-12 13:20:17' 2024-09-12T13:20:21.806 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:20:21.806 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:20:21.806 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:20:21.806 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:20:21.806 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:20:17' 2024-09-12T13:20:21.806 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:20:21.807 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:20:21.807 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2024-09-12T13:20:21.925 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041924231mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary196ef9ec0-0a45-4e60-bcf0-79d28853db5eup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:20:17image-primary10a41c1371-bc6b-4e5a-b8a6-2d2252dfda67up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:20:17image-primary23ceee0af-566c-4f40-8fbb-b3b44ed7d010up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:20:17image-primary350e30e91-e1e0-44f3-8ad3-6fb3a5307bbaup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:20:17image-primary43d37a1b1-ea15-4b8b-8894-ae4f8b6c65cdup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:20:17image-primary5cfd4ea21-2703-4ce9-a4e3-e41c240a1165up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:20:17image-primary614efbe85-d4d9-42ff-aa35-0963a8746f30up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:20:17image-primary73f634cfd-f07a-4271-8c29-0ad3777a474cup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:20:17image-primary827e6f3f9-b993-4ad3-a6a6-74e8fc4feff4up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:20:17image-primary9b8a82bf7-89cf-4fad-b167-0553ebb15a5eup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:20:17' 2024-09-12T13:20:21.925 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:20:21.926 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary5'\'']/last_update' 2024-09-12T13:20:21.932 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:20:17' 2024-09-12T13:20:21.933 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary5'\'']/state' 2024-09-12T13:20:21.938 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+unknown 2024-09-12T13:20:21.939 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:20:21.940 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+unknown 2024-09-12T13:20:21.942 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:20:21.943 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041924231mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary196ef9ec0-0a45-4e60-bcf0-79d28853db5eup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:20:17image-primary10a41c1371-bc6b-4e5a-b8a6-2d2252dfda67up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:20:17image-primary23ceee0af-566c-4f40-8fbb-b3b44ed7d010up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:20:17image-primary350e30e91-e1e0-44f3-8ad3-6fb3a5307bbaup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:20:17image-primary43d37a1b1-ea15-4b8b-8894-ae4f8b6c65cdup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:20:17image-primary5cfd4ea21-2703-4ce9-a4e3-e41c240a1165up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:20:17image-primary614efbe85-d4d9-42ff-aa35-0963a8746f30up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:20:17image-primary73f634cfd-f07a-4271-8c29-0ad3777a474cup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:20:17image-primary827e6f3f9-b993-4ad3-a6a6-74e8fc4feff4up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:20:17image-primary9b8a82bf7-89cf-4fad-b167-0553ebb15a5eup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:20:17' 2024-09-12T13:20:21.943 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:20:21.943 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:20:21.943 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:20:21.944 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ promote_image cluster2 mirror image-primary5 2024-09-12T13:20:21.944 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:20:21.944 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:20:21.944 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:20:21.944 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local force= 2024-09-12T13:20:21.944 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 mirror image promote mirror/image-primary5 2024-09-12T13:20:21.944 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary10'\'']/last_update' 2024-09-12T13:20:21.947 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:20:17' 2024-09-12T13:20:21.948 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary10'\'']/state' 2024-09-12T13:20:21.951 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+unknown 2024-09-12T13:20:21.952 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:20:21.952 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+unknown 2024-09-12T13:20:21.954 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:20:21.955 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:20:21.955 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:20:21.956 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ promote_image cluster2 mirror image-primary10 2024-09-12T13:20:21.956 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:20:21.956 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:20:21.956 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:20:21.956 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local force= 2024-09-12T13:20:21.956 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 mirror image promote mirror/image-primary10 2024-09-12T13:20:22.017 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Image promoted to primary 2024-09-12T13:20:22.031 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \n\b\d ]] 2024-09-12T13:20:22.031 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \k\r\b\d ]] 2024-09-12T13:20:22.032 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo rbd --cluster cluster2 device map -t krbd mirror/image-primary3 2024-09-12T13:20:22.146 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ dev=/dev/rbd1 2024-09-12T13:20:22.148 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo md5sum /dev/rbd1 2024-09-12T13:20:22.148 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ awk '{print $1}' 2024-09-12T13:20:22.447 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary8 up+unknown '' '' 2024-09-12T13:20:22.448 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:20:22.448 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:20:22.448 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:20:22.448 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:20:22.448 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:20:22.448 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:20:22.448 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:20:22.448 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:20:22.448 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary8 2024-09-12T13:20:22.527 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary8: 2024-09-12T13:20:22.528 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 27e6f3f9-b993-4ad3-a6a6-74e8fc4feff4 2024-09-12T13:20:22.528 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+unknown 2024-09-12T13:20:22.528 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: remote image is not primary 2024-09-12T13:20:22.528 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:20:22.529 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:20:17' 2024-09-12T13:20:22.532 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:20:22.533 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+unknown 2024-09-12T13:20:22.533 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:20:22.534 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: remote image is not primary 2024-09-12T13:20:22.534 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:20:22.535 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:20:22.535 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:20:22.536 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:20:22.536 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:20:22.538 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:20:22.539 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:20:22.540 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:20:22.544 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:20:17' 2024-09-12T13:20:22.545 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary8 up+unknown '2024-09-12 13:20:17' 2024-09-12T13:20:22.545 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:20:22.545 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:20:22.545 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:20:22.545 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:20:22.545 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:20:17' 2024-09-12T13:20:22.546 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:20:22.546 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:20:22.546 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-09-12T13:20:22.650 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041934232mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary196ef9ec0-0a45-4e60-bcf0-79d28853db5eup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:20:17image-primary10a41c1371-bc6b-4e5a-b8a6-2d2252dfda67up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:20:17image-primary23ceee0af-566c-4f40-8fbb-b3b44ed7d010up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:20:17image-primary350e30e91-e1e0-44f3-8ad3-6fb3a5307bbaup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:20:17image-primary43d37a1b1-ea15-4b8b-8894-ae4f8b6c65cdup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:20:17image-primary5cfd4ea21-2703-4ce9-a4e3-e41c240a1165up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:20:17image-primary614efbe85-d4d9-42ff-aa35-0963a8746f30up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:20:17image-primary73f634cfd-f07a-4271-8c29-0ad3777a474cup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:20:17image-primary827e6f3f9-b993-4ad3-a6a6-74e8fc4feff4up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:20:17image-primary9b8a82bf7-89cf-4fad-b167-0553ebb15a5eup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:20:17' 2024-09-12T13:20:22.650 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:20:22.651 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary8'\'']/last_update' 2024-09-12T13:20:22.654 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:20:17' 2024-09-12T13:20:22.655 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary8'\'']/state' 2024-09-12T13:20:22.659 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+unknown 2024-09-12T13:20:22.660 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:20:22.660 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+unknown 2024-09-12T13:20:22.662 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:20:22.663 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:20:22.664 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:20:22.664 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster2 mirror image-primary8 up+unknown 2024-09-12T13:20:22.664 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:20:22.664 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:20:22.664 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:20:22.664 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:20:22.664 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:20:22.664 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:20:22.664 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:20:22.664 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:20:23.015 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Image promoted to primary 2024-09-12T13:20:23.018 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Image promoted to primary 2024-09-12T13:20:23.030 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \n\b\d ]] 2024-09-12T13:20:23.030 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \k\r\b\d ]] 2024-09-12T13:20:23.031 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo rbd --cluster cluster2 device map -t krbd mirror/image-primary10 2024-09-12T13:20:23.032 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \n\b\d ]] 2024-09-12T13:20:23.032 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \k\r\b\d ]] 2024-09-12T13:20:23.032 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo rbd --cluster cluster2 device map -t krbd mirror/image-primary5 2024-09-12T13:20:23.174 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ dev=/dev/rbd2 2024-09-12T13:20:23.175 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo md5sum /dev/rbd2 2024-09-12T13:20:23.176 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ awk '{print $1}' 2024-09-12T13:20:23.185 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ dev=/dev/rbd3 2024-09-12T13:20:23.187 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo md5sum /dev/rbd3 2024-09-12T13:20:23.187 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ awk '{print $1}' 2024-09-12T13:20:23.391 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary9 up+unknown '' '' 2024-09-12T13:20:23.391 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary2 up+unknown '' '' 2024-09-12T13:20:23.392 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:20:23.392 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:20:23.392 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:20:23.392 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:20:23.392 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:20:23.392 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:20:23.392 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:20:23.392 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:20:23.392 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:20:23.392 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:20:23.392 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:20:23.392 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:20:23.392 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:20:23.393 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:20:23.393 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:20:23.393 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary9 2024-09-12T13:20:23.393 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:20:23.393 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary2 2024-09-12T13:20:23.496 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary2: 2024-09-12T13:20:23.497 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 3ceee0af-566c-4f40-8fbb-b3b44ed7d010 2024-09-12T13:20:23.497 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+unknown 2024-09-12T13:20:23.497 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: remote image is not primary 2024-09-12T13:20:23.497 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:20:23.497 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:20:17' 2024-09-12T13:20:23.497 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:20:23.500 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+unknown 2024-09-12T13:20:23.500 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:20:23.503 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: remote image is not primary 2024-09-12T13:20:23.504 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:20:23.505 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:20:23.505 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:20:23.507 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:20:23.508 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:20:23.510 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary9: 2024-09-12T13:20:23.510 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: b8a82bf7-89cf-4fad-b167-0553ebb15a5e 2024-09-12T13:20:23.510 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+unknown 2024-09-12T13:20:23.510 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: remote image is not primary 2024-09-12T13:20:23.510 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:20:23.510 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:20:17' 2024-09-12T13:20:23.510 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:20:23.511 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:20:23.511 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:20:23.513 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:20:23.513 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+unknown 2024-09-12T13:20:23.514 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:20:23.519 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: remote image is not primary 2024-09-12T13:20:23.519 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:20:23.519 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:20:17' 2024-09-12T13:20:23.519 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary2 up+unknown '2024-09-12 13:20:17' 2024-09-12T13:20:23.520 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:20:23.520 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:20:23.520 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:20:23.520 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:20:23.520 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:20:17' 2024-09-12T13:20:23.520 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:20:23.520 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:20:23.520 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:20:23.520 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:20:23.521 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-09-12T13:20:23.522 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:20:23.523 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:20:23.527 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:20:23.528 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:20:23.529 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:20:23.533 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:20:17' 2024-09-12T13:20:23.533 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary9 up+unknown '2024-09-12 13:20:17' 2024-09-12T13:20:23.534 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:20:23.534 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:20:23.534 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:20:23.534 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:20:23.534 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:20:17' 2024-09-12T13:20:23.534 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:20:23.534 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:20:23.534 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-09-12T13:20:23.666 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster2 mirror image-primary8 up+unknown '' '' 2024-09-12T13:20:23.666 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:20:23.666 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:20:23.666 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:20:23.666 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:20:23.666 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:20:23.667 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:20:23.667 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:20:23.667 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:20:23.668 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster2 mirror image status mirror/image-primary8 2024-09-12T13:20:23.694 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041934232mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary196ef9ec0-0a45-4e60-bcf0-79d28853db5eup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:20:17image-primary10a41c1371-bc6b-4e5a-b8a6-2d2252dfda67up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:20:17image-primary23ceee0af-566c-4f40-8fbb-b3b44ed7d010up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:20:17image-primary350e30e91-e1e0-44f3-8ad3-6fb3a5307bbaup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:20:17image-primary43d37a1b1-ea15-4b8b-8894-ae4f8b6c65cdup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:20:17image-primary5cfd4ea21-2703-4ce9-a4e3-e41c240a1165up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:20:17image-primary614efbe85-d4d9-42ff-aa35-0963a8746f30up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:20:17image-primary73f634cfd-f07a-4271-8c29-0ad3777a474cup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:20:17image-primary827e6f3f9-b993-4ad3-a6a6-74e8fc4feff4up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:20:17image-primary9b8a82bf7-89cf-4fad-b167-0553ebb15a5eup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:20:17' 2024-09-12T13:20:23.694 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:20:23.694 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary9'\'']/last_update' 2024-09-12T13:20:23.702 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:20:17' 2024-09-12T13:20:23.702 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary9'\'']/state' 2024-09-12T13:20:23.711 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+unknown 2024-09-12T13:20:23.712 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:20:23.712 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+unknown 2024-09-12T13:20:23.716 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:20:23.717 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:20:23.717 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:20:23.718 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster2 mirror image-primary9 up+unknown 2024-09-12T13:20:23.718 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:20:23.718 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:20:23.718 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:20:23.718 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:20:23.718 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:20:23.718 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:20:23.718 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:20:23.718 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:20:23.767 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary8: 2024-09-12T13:20:23.767 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 27e6f3f9-b993-4ad3-a6a6-74e8fc4feff4 2024-09-12T13:20:23.767 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+unknown 2024-09-12T13:20:23.767 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: remote image is not primary 2024-09-12T13:20:23.767 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:20:23.767 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:20:17' 2024-09-12T13:20:23.767 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:20:23.770 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+unknown 2024-09-12T13:20:23.770 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:20:23.772 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: remote image is not primary 2024-09-12T13:20:23.773 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:20:23.774 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:20:23.775 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:20:23.776 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:20:23.777 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:20:23.779 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:20:23.781 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:20:23.781 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:20:23.785 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:20:17' 2024-09-12T13:20:23.785 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster2 mirror image-primary8 up+unknown '2024-09-12 13:20:17' 2024-09-12T13:20:23.786 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:20:23.786 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:20:23.786 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:20:23.786 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:20:23.786 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:20:17' 2024-09-12T13:20:23.787 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:20:23.787 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:20:23.787 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2024-09-12T13:20:23.909 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041934232mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary196ef9ec0-0a45-4e60-bcf0-79d28853db5eup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:20:17image-primary10a41c1371-bc6b-4e5a-b8a6-2d2252dfda67up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:20:17image-primary23ceee0af-566c-4f40-8fbb-b3b44ed7d010up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:20:17image-primary350e30e91-e1e0-44f3-8ad3-6fb3a5307bbaup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:20:17image-primary43d37a1b1-ea15-4b8b-8894-ae4f8b6c65cdup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:20:17image-primary5cfd4ea21-2703-4ce9-a4e3-e41c240a1165up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:20:17image-primary614efbe85-d4d9-42ff-aa35-0963a8746f30up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:20:17image-primary73f634cfd-f07a-4271-8c29-0ad3777a474cup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:20:17image-primary827e6f3f9-b993-4ad3-a6a6-74e8fc4feff4up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:20:17image-primary9b8a82bf7-89cf-4fad-b167-0553ebb15a5eup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:20:17' 2024-09-12T13:20:23.910 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:20:23.911 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary2'\'']/last_update' 2024-09-12T13:20:23.921 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:20:17' 2024-09-12T13:20:23.922 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary2'\'']/state' 2024-09-12T13:20:23.928 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+unknown 2024-09-12T13:20:23.929 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:20:23.929 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+unknown 2024-09-12T13:20:23.933 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041924231mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary196ef9ec0-0a45-4e60-bcf0-79d28853db5eup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:20:17image-primary10a41c1371-bc6b-4e5a-b8a6-2d2252dfda67up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:20:17image-primary23ceee0af-566c-4f40-8fbb-b3b44ed7d010up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:20:17image-primary350e30e91-e1e0-44f3-8ad3-6fb3a5307bbaup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:20:17image-primary43d37a1b1-ea15-4b8b-8894-ae4f8b6c65cdup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:20:17image-primary5cfd4ea21-2703-4ce9-a4e3-e41c240a1165up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:20:17image-primary614efbe85-d4d9-42ff-aa35-0963a8746f30up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:20:17image-primary73f634cfd-f07a-4271-8c29-0ad3777a474cup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:20:17image-primary827e6f3f9-b993-4ad3-a6a6-74e8fc4feff4up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:20:17image-primary9b8a82bf7-89cf-4fad-b167-0553ebb15a5eup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:20:17' 2024-09-12T13:20:23.933 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:20:23.933 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:20:23.933 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary8'\'']/last_update' 2024-09-12T13:20:23.934 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:20:23.934 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:20:23.934 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster2 mirror image-primary2 up+unknown 2024-09-12T13:20:23.934 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:20:23.934 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:20:23.934 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:20:23.934 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:20:23.935 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:20:23.935 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:20:23.935 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:20:23.935 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:20:23.940 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:20:17' 2024-09-12T13:20:23.941 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary8'\'']/state' 2024-09-12T13:20:23.944 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+unknown 2024-09-12T13:20:23.945 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:20:23.946 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+unknown 2024-09-12T13:20:23.948 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:20:23.949 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:20:23.949 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:20:23.950 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ promote_image cluster2 mirror image-primary8 2024-09-12T13:20:23.950 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:20:23.950 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:20:23.950 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:20:23.950 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local force= 2024-09-12T13:20:23.950 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 mirror image promote mirror/image-primary8 2024-09-12T13:20:24.367 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary6 up+unknown '' '' 2024-09-12T13:20:24.367 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:20:24.367 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:20:24.367 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:20:24.368 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:20:24.368 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:20:24.368 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:20:24.368 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:20:24.368 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:20:24.368 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary6 2024-09-12T13:20:24.371 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary1 up+unknown '' '' 2024-09-12T13:20:24.371 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:20:24.371 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:20:24.371 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:20:24.371 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:20:24.371 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:20:24.372 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:20:24.372 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:20:24.373 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary4 up+unknown '' '' 2024-09-12T13:20:24.373 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:20:24.373 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:20:24.373 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:20:24.373 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:20:24.373 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:20:24.373 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:20:24.373 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:20:24.373 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:20:24.373 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary1 2024-09-12T13:20:24.374 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:20:24.374 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary4 2024-09-12T13:20:24.484 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary4: 2024-09-12T13:20:24.484 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 3d37a1b1-ea15-4b8b-8894-ae4f8b6c65cd 2024-09-12T13:20:24.484 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+unknown 2024-09-12T13:20:24.484 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: remote image is not primary 2024-09-12T13:20:24.484 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:20:24.485 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:20:17' 2024-09-12T13:20:24.485 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:20:24.488 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+unknown 2024-09-12T13:20:24.489 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:20:24.490 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary6: 2024-09-12T13:20:24.490 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 14efbe85-d4d9-42ff-aa35-0963a8746f30 2024-09-12T13:20:24.490 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+unknown 2024-09-12T13:20:24.490 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: remote image is not primary 2024-09-12T13:20:24.490 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:20:24.490 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:20:17' 2024-09-12T13:20:24.490 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:20:24.493 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: remote image is not primary 2024-09-12T13:20:24.494 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+unknown 2024-09-12T13:20:24.494 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:20:24.494 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:20:24.495 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:20:24.496 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:20:24.499 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary1: 2024-09-12T13:20:24.499 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 96ef9ec0-0a45-4e60-bcf0-79d28853db5e 2024-09-12T13:20:24.499 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+unknown 2024-09-12T13:20:24.499 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: remote image is not primary 2024-09-12T13:20:24.499 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:20:24.499 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:20:17' 2024-09-12T13:20:24.500 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:20:24.500 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: remote image is not primary 2024-09-12T13:20:24.500 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:20:24.500 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:20:24.500 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:20:24.501 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:20:24.502 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:20:24.503 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+unknown 2024-09-12T13:20:24.503 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:20:24.504 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:20:24.505 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:20:24.505 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:20:24.506 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:20:24.506 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:20:24.507 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: remote image is not primary 2024-09-12T13:20:24.510 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:20:24.511 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:20:24.511 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:20:24.512 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:20:24.516 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:20:24.516 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:20:24.516 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:20:17' 2024-09-12T13:20:24.516 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary4 up+unknown '2024-09-12 13:20:17' 2024-09-12T13:20:24.516 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:20:24.516 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:20:24.516 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:20:24.516 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:20:24.516 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:20:17' 2024-09-12T13:20:24.516 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:20:24.517 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:20:24.517 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:20:24.517 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:20:24.517 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-09-12T13:20:24.519 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:20:17' 2024-09-12T13:20:24.519 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary6 up+unknown '2024-09-12 13:20:17' 2024-09-12T13:20:24.519 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:20:24.519 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:20:24.519 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:20:24.519 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:20:24.519 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:20:17' 2024-09-12T13:20:24.519 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:20:24.520 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:20:24.520 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-09-12T13:20:24.521 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:20:24.522 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:20:24.523 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:20:24.529 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:20:17' 2024-09-12T13:20:24.529 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary1 up+unknown '2024-09-12 13:20:17' 2024-09-12T13:20:24.529 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:20:24.529 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:20:24.529 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:20:24.530 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:20:24.530 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:20:17' 2024-09-12T13:20:24.530 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:20:24.530 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:20:24.530 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-09-12T13:20:24.709 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041934232mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary196ef9ec0-0a45-4e60-bcf0-79d28853db5eup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:20:17image-primary10a41c1371-bc6b-4e5a-b8a6-2d2252dfda67up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:20:17image-primary23ceee0af-566c-4f40-8fbb-b3b44ed7d010up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:20:17image-primary350e30e91-e1e0-44f3-8ad3-6fb3a5307bbaup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:20:17image-primary43d37a1b1-ea15-4b8b-8894-ae4f8b6c65cdup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:20:17image-primary5cfd4ea21-2703-4ce9-a4e3-e41c240a1165up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:20:17image-primary614efbe85-d4d9-42ff-aa35-0963a8746f30up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:20:17image-primary73f634cfd-f07a-4271-8c29-0ad3777a474cup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:20:17image-primary827e6f3f9-b993-4ad3-a6a6-74e8fc4feff4up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:20:17image-primary9b8a82bf7-89cf-4fad-b167-0553ebb15a5eup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:20:17' 2024-09-12T13:20:24.709 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:20:24.710 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary6'\'']/last_update' 2024-09-12T13:20:24.717 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:20:17' 2024-09-12T13:20:24.718 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary6'\'']/state' 2024-09-12T13:20:24.723 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041934232mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary196ef9ec0-0a45-4e60-bcf0-79d28853db5eup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:20:17image-primary10a41c1371-bc6b-4e5a-b8a6-2d2252dfda67up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:20:17image-primary23ceee0af-566c-4f40-8fbb-b3b44ed7d010up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:20:17image-primary350e30e91-e1e0-44f3-8ad3-6fb3a5307bbaup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:20:17image-primary43d37a1b1-ea15-4b8b-8894-ae4f8b6c65cdup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:20:17image-primary5cfd4ea21-2703-4ce9-a4e3-e41c240a1165up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:20:17image-primary614efbe85-d4d9-42ff-aa35-0963a8746f30up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:20:17image-primary73f634cfd-f07a-4271-8c29-0ad3777a474cup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:20:17image-primary827e6f3f9-b993-4ad3-a6a6-74e8fc4feff4up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:20:17image-primary9b8a82bf7-89cf-4fad-b167-0553ebb15a5eup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:20:17' 2024-09-12T13:20:24.723 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:20:24.723 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster2 mirror image-primary9 up+unknown '' '' 2024-09-12T13:20:24.723 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:20:24.723 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:20:24.723 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:20:24.723 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:20:24.723 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:20:24.723 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:20:24.723 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:20:24.724 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary1'\'']/last_update' 2024-09-12T13:20:24.724 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:20:24.724 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster2 mirror image status mirror/image-primary9 2024-09-12T13:20:24.726 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+unknown 2024-09-12T13:20:24.726 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041934232mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary196ef9ec0-0a45-4e60-bcf0-79d28853db5eup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:20:17image-primary10a41c1371-bc6b-4e5a-b8a6-2d2252dfda67up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:20:17image-primary23ceee0af-566c-4f40-8fbb-b3b44ed7d010up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:20:17image-primary350e30e91-e1e0-44f3-8ad3-6fb3a5307bbaup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:20:17image-primary43d37a1b1-ea15-4b8b-8894-ae4f8b6c65cdup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:20:17image-primary5cfd4ea21-2703-4ce9-a4e3-e41c240a1165up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:20:17image-primary614efbe85-d4d9-42ff-aa35-0963a8746f30up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:20:17image-primary73f634cfd-f07a-4271-8c29-0ad3777a474cup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:20:17image-primary827e6f3f9-b993-4ad3-a6a6-74e8fc4feff4up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:20:17image-primary9b8a82bf7-89cf-4fad-b167-0553ebb15a5eup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:20:17' 2024-09-12T13:20:24.727 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:20:24.727 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:20:24.727 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+unknown 2024-09-12T13:20:24.728 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary4'\'']/last_update' 2024-09-12T13:20:24.731 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:20:17' 2024-09-12T13:20:24.732 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:20:24.732 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary1'\'']/state' 2024-09-12T13:20:24.733 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:20:24.733 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:20:24.733 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster2 mirror image-primary6 up+unknown 2024-09-12T13:20:24.734 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:20:24.734 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:20:24.734 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:20:24.734 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:20:24.734 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:20:24.734 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:20:24.734 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:20:24.734 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:20:24.735 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:20:17' 2024-09-12T13:20:24.737 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary4'\'']/state' 2024-09-12T13:20:24.739 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+unknown 2024-09-12T13:20:24.740 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:20:24.741 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+unknown 2024-09-12T13:20:24.744 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:20:24.744 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+unknown 2024-09-12T13:20:24.745 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:20:24.745 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:20:24.745 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster2 mirror image-primary1 up+unknown 2024-09-12T13:20:24.745 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:20:24.745 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:20:24.746 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:20:24.746 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:20:24.746 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:20:24.746 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:20:24.746 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:20:24.746 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:20:24.746 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:20:24.746 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+unknown 2024-09-12T13:20:24.749 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:20:24.751 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:20:24.751 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:20:24.751 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster2 mirror image-primary4 up+unknown 2024-09-12T13:20:24.751 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:20:24.751 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:20:24.751 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:20:24.751 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:20:24.751 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:20:24.752 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:20:24.752 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:20:24.752 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:20:24.826 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary9: 2024-09-12T13:20:24.826 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: b8a82bf7-89cf-4fad-b167-0553ebb15a5e 2024-09-12T13:20:24.826 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+unknown 2024-09-12T13:20:24.826 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: remote image is not primary 2024-09-12T13:20:24.826 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:20:24.826 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:20:17' 2024-09-12T13:20:24.826 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:20:24.828 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+unknown 2024-09-12T13:20:24.829 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:20:24.832 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: remote image is not primary 2024-09-12T13:20:24.832 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:20:24.833 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:20:24.834 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:20:24.836 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:20:24.837 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:20:24.840 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:20:24.840 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:20:24.841 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:20:24.845 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:20:17' 2024-09-12T13:20:24.846 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster2 mirror image-primary9 up+unknown '2024-09-12 13:20:17' 2024-09-12T13:20:24.846 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:20:24.846 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:20:24.846 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:20:24.846 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:20:24.846 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:20:17' 2024-09-12T13:20:24.846 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:20:24.847 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:20:24.847 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2024-09-12T13:20:24.937 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster2 mirror image-primary2 up+unknown '' '' 2024-09-12T13:20:24.937 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:20:24.937 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:20:24.937 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:20:24.937 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:20:24.937 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:20:24.938 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:20:24.938 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:20:24.939 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:20:24.939 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster2 mirror image status mirror/image-primary2 2024-09-12T13:20:24.992 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041924231mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary196ef9ec0-0a45-4e60-bcf0-79d28853db5eup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:20:17image-primary10a41c1371-bc6b-4e5a-b8a6-2d2252dfda67up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:20:17image-primary23ceee0af-566c-4f40-8fbb-b3b44ed7d010up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:20:17image-primary350e30e91-e1e0-44f3-8ad3-6fb3a5307bbaup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:20:17image-primary43d37a1b1-ea15-4b8b-8894-ae4f8b6c65cdup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:20:17image-primary5cfd4ea21-2703-4ce9-a4e3-e41c240a1165up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:20:17image-primary614efbe85-d4d9-42ff-aa35-0963a8746f30up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:20:17image-primary73f634cfd-f07a-4271-8c29-0ad3777a474cup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:20:17image-primary827e6f3f9-b993-4ad3-a6a6-74e8fc4feff4up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:20:17image-primary9b8a82bf7-89cf-4fad-b167-0553ebb15a5eup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:20:17' 2024-09-12T13:20:24.993 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:20:24.993 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary9'\'']/last_update' 2024-09-12T13:20:24.999 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:20:17' 2024-09-12T13:20:25.003 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary9'\'']/state' 2024-09-12T13:20:25.009 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+unknown 2024-09-12T13:20:25.010 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:20:25.010 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+unknown 2024-09-12T13:20:25.013 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:20:25.014 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:20:25.014 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:20:25.014 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ promote_image cluster2 mirror image-primary9 2024-09-12T13:20:25.014 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:20:25.014 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:20:25.014 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:20:25.014 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local force= 2024-09-12T13:20:25.014 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 mirror image promote mirror/image-primary9 2024-09-12T13:20:25.057 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Image promoted to primary 2024-09-12T13:20:25.080 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \n\b\d ]] 2024-09-12T13:20:25.080 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \k\r\b\d ]] 2024-09-12T13:20:25.081 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo rbd --cluster cluster2 device map -t krbd mirror/image-primary8 2024-09-12T13:20:25.091 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary2: 2024-09-12T13:20:25.091 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 3ceee0af-566c-4f40-8fbb-b3b44ed7d010 2024-09-12T13:20:25.091 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+unknown 2024-09-12T13:20:25.091 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: remote image is not primary 2024-09-12T13:20:25.091 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:20:25.092 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:20:17' 2024-09-12T13:20:25.092 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:20:25.096 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+unknown 2024-09-12T13:20:25.097 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:20:25.102 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: remote image is not primary 2024-09-12T13:20:25.102 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:20:25.104 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:20:25.104 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:20:25.107 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:20:25.109 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:20:25.113 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:20:25.114 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:20:25.115 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:20:25.121 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:20:17' 2024-09-12T13:20:25.123 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster2 mirror image-primary2 up+unknown '2024-09-12 13:20:17' 2024-09-12T13:20:25.123 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:20:25.123 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:20:25.123 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:20:25.123 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:20:25.123 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:20:17' 2024-09-12T13:20:25.123 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:20:25.123 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:20:25.124 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2024-09-12T13:20:25.304 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ dev=/dev/rbd4 2024-09-12T13:20:25.307 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo md5sum /dev/rbd4 2024-09-12T13:20:25.307 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ awk '{print $1}' 2024-09-12T13:20:25.313 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041924231mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary196ef9ec0-0a45-4e60-bcf0-79d28853db5eup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:20:17image-primary10a41c1371-bc6b-4e5a-b8a6-2d2252dfda67up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:20:17image-primary23ceee0af-566c-4f40-8fbb-b3b44ed7d010up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:20:17image-primary350e30e91-e1e0-44f3-8ad3-6fb3a5307bbaup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:20:17image-primary43d37a1b1-ea15-4b8b-8894-ae4f8b6c65cdup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:20:17image-primary5cfd4ea21-2703-4ce9-a4e3-e41c240a1165up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:20:17image-primary614efbe85-d4d9-42ff-aa35-0963a8746f30up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:20:17image-primary73f634cfd-f07a-4271-8c29-0ad3777a474cup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:20:17image-primary827e6f3f9-b993-4ad3-a6a6-74e8fc4feff4up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:20:17image-primary9b8a82bf7-89cf-4fad-b167-0553ebb15a5eup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:20:17' 2024-09-12T13:20:25.313 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:20:25.313 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary2'\'']/last_update' 2024-09-12T13:20:25.317 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:20:17' 2024-09-12T13:20:25.319 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary2'\'']/state' 2024-09-12T13:20:25.322 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+unknown 2024-09-12T13:20:25.323 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:20:25.324 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+unknown 2024-09-12T13:20:25.326 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:20:25.327 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:20:25.327 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:20:25.327 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ promote_image cluster2 mirror image-primary2 2024-09-12T13:20:25.327 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:20:25.327 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:20:25.327 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:20:25.327 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local force= 2024-09-12T13:20:25.327 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 mirror image promote mirror/image-primary2 2024-09-12T13:20:25.737 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster2 mirror image-primary6 up+unknown '' '' 2024-09-12T13:20:25.738 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:20:25.738 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:20:25.738 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:20:25.738 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:20:25.738 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:20:25.738 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:20:25.738 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:20:25.739 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:20:25.739 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster2 mirror image status mirror/image-primary6 2024-09-12T13:20:25.749 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster2 mirror image-primary1 up+unknown '' '' 2024-09-12T13:20:25.749 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:20:25.749 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:20:25.749 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:20:25.749 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:20:25.749 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:20:25.749 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:20:25.750 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:20:25.750 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:20:25.750 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster2 mirror image status mirror/image-primary1 2024-09-12T13:20:25.755 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster2 mirror image-primary4 up+unknown '' '' 2024-09-12T13:20:25.755 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:20:25.755 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:20:25.755 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:20:25.755 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:20:25.756 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:20:25.756 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:20:25.756 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:20:25.756 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:20:25.756 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster2 mirror image status mirror/image-primary4 2024-09-12T13:20:25.865 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary6: 2024-09-12T13:20:25.865 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 14efbe85-d4d9-42ff-aa35-0963a8746f30 2024-09-12T13:20:25.866 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+unknown 2024-09-12T13:20:25.866 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: remote image is not primary 2024-09-12T13:20:25.866 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:20:25.866 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:20:17' 2024-09-12T13:20:25.866 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:20:25.870 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+unknown 2024-09-12T13:20:25.871 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:20:25.876 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: remote image is not primary 2024-09-12T13:20:25.877 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:20:25.878 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:20:25.878 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:20:25.882 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:20:25.883 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:20:25.885 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary4: 2024-09-12T13:20:25.885 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 3d37a1b1-ea15-4b8b-8894-ae4f8b6c65cd 2024-09-12T13:20:25.885 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+unknown 2024-09-12T13:20:25.886 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: remote image is not primary 2024-09-12T13:20:25.886 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:20:25.886 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:20:17' 2024-09-12T13:20:25.886 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:20:25.886 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary1: 2024-09-12T13:20:25.887 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 96ef9ec0-0a45-4e60-bcf0-79d28853db5e 2024-09-12T13:20:25.887 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+unknown 2024-09-12T13:20:25.887 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: remote image is not primary 2024-09-12T13:20:25.887 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:20:25.887 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:20:17' 2024-09-12T13:20:25.887 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:20:25.888 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:20:25.888 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:20:25.890 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:20:25.890 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+unknown 2024-09-12T13:20:25.890 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+unknown 2024-09-12T13:20:25.891 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:20:25.891 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:20:25.896 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: remote image is not primary 2024-09-12T13:20:25.896 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: remote image is not primary 2024-09-12T13:20:25.897 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:20:17' 2024-09-12T13:20:25.897 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:20:25.897 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:20:25.898 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster2 mirror image-primary6 up+unknown '2024-09-12 13:20:17' 2024-09-12T13:20:25.898 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:20:25.898 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:20:25.898 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:20:25.898 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:20:25.898 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:20:17' 2024-09-12T13:20:25.898 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:20:25.898 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:20:25.899 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:20:25.899 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:20:25.899 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2024-09-12T13:20:25.899 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:20:25.900 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:20:25.903 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:20:25.903 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:20:25.904 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:20:25.905 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:20:25.908 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:20:25.909 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:20:25.909 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:20:25.910 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:20:25.910 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:20:25.911 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:20:25.917 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:20:17' 2024-09-12T13:20:25.917 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster2 mirror image-primary1 up+unknown '2024-09-12 13:20:17' 2024-09-12T13:20:25.919 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:20:25.919 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:20:17' 2024-09-12T13:20:25.919 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster2 mirror image-primary4 up+unknown '2024-09-12 13:20:17' 2024-09-12T13:20:25.919 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:20:25.919 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:20:25.919 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:20:25.919 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:20:17' 2024-09-12T13:20:25.919 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:20:25.919 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:20:25.919 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:20:25.919 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:20:25.919 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:20:25.919 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:20:17' 2024-09-12T13:20:25.920 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:20:25.920 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:20:25.920 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2024-09-12T13:20:25.920 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:20:25.920 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2024-09-12T13:20:26.106 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Image promoted to primary 2024-09-12T13:20:26.154 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Image promoted to primary 2024-09-12T13:20:26.208 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \n\b\d ]] 2024-09-12T13:20:26.209 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \k\r\b\d ]] 2024-09-12T13:20:26.211 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo rbd --cluster cluster2 device map -t krbd mirror/image-primary2 2024-09-12T13:20:26.232 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041924231mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary196ef9ec0-0a45-4e60-bcf0-79d28853db5eup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:20:17image-primary10a41c1371-bc6b-4e5a-b8a6-2d2252dfda67up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:20:17image-primary23ceee0af-566c-4f40-8fbb-b3b44ed7d010up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:20:17image-primary350e30e91-e1e0-44f3-8ad3-6fb3a5307bbaup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:20:17image-primary43d37a1b1-ea15-4b8b-8894-ae4f8b6c65cdup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:20:17image-primary5cfd4ea21-2703-4ce9-a4e3-e41c240a1165up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:20:17image-primary614efbe85-d4d9-42ff-aa35-0963a8746f30up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:20:17image-primary73f634cfd-f07a-4271-8c29-0ad3777a474cup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:20:17image-primary827e6f3f9-b993-4ad3-a6a6-74e8fc4feff4up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:20:17image-primary9b8a82bf7-89cf-4fad-b167-0553ebb15a5eup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:20:17' 2024-09-12T13:20:26.232 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:20:26.235 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \n\b\d ]] 2024-09-12T13:20:26.235 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \k\r\b\d ]] 2024-09-12T13:20:26.236 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary6'\'']/last_update' 2024-09-12T13:20:26.238 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo rbd --cluster cluster2 device map -t krbd mirror/image-primary9 2024-09-12T13:20:26.248 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:20:17' 2024-09-12T13:20:26.250 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary6'\'']/state' 2024-09-12T13:20:26.257 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041924231mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary196ef9ec0-0a45-4e60-bcf0-79d28853db5eup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:20:17image-primary10a41c1371-bc6b-4e5a-b8a6-2d2252dfda67up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:20:17image-primary23ceee0af-566c-4f40-8fbb-b3b44ed7d010up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:20:17image-primary350e30e91-e1e0-44f3-8ad3-6fb3a5307bbaup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:20:17image-primary43d37a1b1-ea15-4b8b-8894-ae4f8b6c65cdup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:20:17image-primary5cfd4ea21-2703-4ce9-a4e3-e41c240a1165up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:20:17image-primary614efbe85-d4d9-42ff-aa35-0963a8746f30up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:20:17image-primary73f634cfd-f07a-4271-8c29-0ad3777a474cup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:20:17image-primary827e6f3f9-b993-4ad3-a6a6-74e8fc4feff4up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:20:17image-primary9b8a82bf7-89cf-4fad-b167-0553ebb15a5eup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:20:17' 2024-09-12T13:20:26.257 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:20:26.258 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary1'\'']/last_update' 2024-09-12T13:20:26.259 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+unknown 2024-09-12T13:20:26.262 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:20:26.263 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041924231mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary196ef9ec0-0a45-4e60-bcf0-79d28853db5eup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:20:17image-primary10a41c1371-bc6b-4e5a-b8a6-2d2252dfda67up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:20:17image-primary23ceee0af-566c-4f40-8fbb-b3b44ed7d010up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:20:17image-primary350e30e91-e1e0-44f3-8ad3-6fb3a5307bbaup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:20:17image-primary43d37a1b1-ea15-4b8b-8894-ae4f8b6c65cdup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:20:17image-primary5cfd4ea21-2703-4ce9-a4e3-e41c240a1165up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:20:17image-primary614efbe85-d4d9-42ff-aa35-0963a8746f30up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:20:17image-primary73f634cfd-f07a-4271-8c29-0ad3777a474cup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:20:17image-primary827e6f3f9-b993-4ad3-a6a6-74e8fc4feff4up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:20:17image-primary9b8a82bf7-89cf-4fad-b167-0553ebb15a5eup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:20:17' 2024-09-12T13:20:26.263 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:20:26.264 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+unknown 2024-09-12T13:20:26.266 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary4'\'']/last_update' 2024-09-12T13:20:26.271 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:20:26.273 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:20:17' 2024-09-12T13:20:26.274 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary1'\'']/state' 2024-09-12T13:20:26.275 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:20:26.275 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:20:26.275 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ promote_image cluster2 mirror image-primary6 2024-09-12T13:20:26.275 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:20:26.276 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:20:26.276 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:20:26.276 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local force= 2024-09-12T13:20:26.276 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 mirror image promote mirror/image-primary6 2024-09-12T13:20:26.279 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:20:17' 2024-09-12T13:20:26.281 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary4'\'']/state' 2024-09-12T13:20:26.289 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+unknown 2024-09-12T13:20:26.291 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+unknown 2024-09-12T13:20:26.293 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:20:26.294 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+unknown 2024-09-12T13:20:26.297 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:20:26.297 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:20:26.298 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:20:26.299 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:20:26.299 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ promote_image cluster2 mirror image-primary1 2024-09-12T13:20:26.299 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:20:26.299 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:20:26.299 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:20:26.299 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local force= 2024-09-12T13:20:26.299 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 mirror image promote mirror/image-primary1 2024-09-12T13:20:26.299 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+unknown 2024-09-12T13:20:26.303 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:20:26.304 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:20:26.305 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:20:26.305 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ promote_image cluster2 mirror image-primary4 2024-09-12T13:20:26.305 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:20:26.305 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:20:26.305 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:20:26.305 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local force= 2024-09-12T13:20:26.305 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 mirror image promote mirror/image-primary4 2024-09-12T13:20:26.497 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ dev=/dev/rbd5 2024-09-12T13:20:26.499 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo md5sum /dev/rbd5 2024-09-12T13:20:26.500 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ awk '{print $1}' 2024-09-12T13:20:26.561 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ dev=/dev/rbd6 2024-09-12T13:20:26.563 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo md5sum /dev/rbd6 2024-09-12T13:20:26.564 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ awk '{print $1}' 2024-09-12T13:20:27.099 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Image promoted to primary 2024-09-12T13:20:27.108 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Image promoted to primary 2024-09-12T13:20:27.117 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Image promoted to primary 2024-09-12T13:20:27.141 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \n\b\d ]] 2024-09-12T13:20:27.141 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \k\r\b\d ]] 2024-09-12T13:20:27.142 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo rbd --cluster cluster2 device map -t krbd mirror/image-primary4 2024-09-12T13:20:27.162 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \n\b\d ]] 2024-09-12T13:20:27.163 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \k\r\b\d ]] 2024-09-12T13:20:27.164 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo rbd --cluster cluster2 device map -t krbd mirror/image-primary6 2024-09-12T13:20:27.165 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \n\b\d ]] 2024-09-12T13:20:27.165 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \k\r\b\d ]] 2024-09-12T13:20:27.170 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo rbd --cluster cluster2 device map -t krbd mirror/image-primary1 2024-09-12T13:20:27.509 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ dev=/dev/rbd7 2024-09-12T13:20:27.580 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ awk '{print $1}' 2024-09-12T13:20:27.580 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo md5sum /dev/rbd7 2024-09-12T13:20:27.580 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ dev=/dev/rbd8 2024-09-12T13:20:27.580 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo md5sum /dev/rbd8 2024-09-12T13:20:27.580 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ awk '{print $1}' 2024-09-12T13:20:27.595 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ dev=/dev/rbd9 2024-09-12T13:20:27.600 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo md5sum /dev/rbd9 2024-09-12T13:20:27.601 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ awk '{print $1}' 2024-09-12T13:21:13.777 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ promote_md5=5d2c4017cbcdcb1cd6f6730465d75450 2024-09-12T13:21:13.777 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo rbd --cluster cluster2 device unmap -t krbd /dev/rbd1 2024-09-12T13:21:14.405 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ 5d2c4017cbcdcb1cd6f6730465d75450 != \5\d\2\c\4\0\1\7\c\b\c\d\c\b\1\c\d\6\f\6\7\3\0\4\6\5\d\7\5\4\5\0 ]] 2024-09-12T13:21:15.012 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ promote_md5=39408e54a2f5295dd5daa911a416fda4 2024-09-12T13:21:15.012 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo rbd --cluster cluster2 device unmap -t krbd /dev/rbd0 2024-09-12T13:21:15.257 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ 39408e54a2f5295dd5daa911a416fda4 != \3\9\4\0\8\e\5\4\a\2\f\5\2\9\5\d\d\5\d\a\a\9\1\1\a\4\1\6\f\d\a\4 ]] 2024-09-12T13:21:15.385 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ promote_md5=207063409a811f9645abeef41738206b 2024-09-12T13:21:15.385 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo rbd --cluster cluster2 device unmap -t krbd /dev/rbd3 2024-09-12T13:21:15.593 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ 207063409a811f9645abeef41738206b != \2\0\7\0\6\3\4\0\9\a\8\1\1\f\9\6\4\5\a\b\e\e\f\4\1\7\3\8\2\0\6\b ]] 2024-09-12T13:21:16.188 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ promote_md5=faa0aaae9753072b2cac78d3e2467795 2024-09-12T13:21:16.188 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo rbd --cluster cluster2 device unmap -t krbd /dev/rbd2 2024-09-12T13:21:16.333 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ faa0aaae9753072b2cac78d3e2467795 != \f\a\a\0\a\a\a\e\9\7\5\3\0\7\2\b\2\c\a\c\7\8\d\3\e\2\4\6\7\7\9\5 ]] 2024-09-12T13:21:16.826 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ promote_md5=4120d89c38b264f38b25861711499df2 2024-09-12T13:21:16.826 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo rbd --cluster cluster2 device unmap -t krbd /dev/rbd5 2024-09-12T13:21:16.965 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ 4120d89c38b264f38b25861711499df2 != \4\1\2\0\d\8\9\c\3\8\b\2\6\4\f\3\8\b\2\5\8\6\1\7\1\1\4\9\9\d\f\2 ]] 2024-09-12T13:21:18.989 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ promote_md5=15523d877809e4743f3932b22d38c387 2024-09-12T13:21:18.989 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo rbd --cluster cluster2 device unmap -t krbd /dev/rbd4 2024-09-12T13:21:19.123 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ 15523d877809e4743f3932b22d38c387 != \1\5\5\2\3\d\8\7\7\8\0\9\e\4\7\4\3\f\3\9\3\2\b\2\2\d\3\8\c\3\8\7 ]] 2024-09-12T13:21:19.335 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ promote_md5=e4c2be7299c521f6670ee00aa7fe149d 2024-09-12T13:21:19.336 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo rbd --cluster cluster2 device unmap -t krbd /dev/rbd6 2024-09-12T13:21:19.443 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ e4c2be7299c521f6670ee00aa7fe149d != \e\4\c\2\b\e\7\2\9\9\c\5\2\1\f\6\6\7\0\e\e\0\0\a\a\7\f\e\1\4\9\d ]] 2024-09-12T13:21:19.566 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ promote_md5=b639770233b76ecd8e91005571f6041c 2024-09-12T13:21:19.566 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo rbd --cluster cluster2 device unmap -t krbd /dev/rbd8 2024-09-12T13:21:19.877 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ b639770233b76ecd8e91005571f6041c != \b\6\3\9\7\7\0\2\3\3\b\7\6\e\c\d\8\e\9\1\0\0\5\5\7\1\f\6\0\4\1\c ]] 2024-09-12T13:21:19.877 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${COMPARE_PIDS[@]} 2024-09-12T13:21:19.877 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 66363 2024-09-12T13:21:19.878 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${COMPARE_PIDS[@]} 2024-09-12T13:21:19.878 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 66365 2024-09-12T13:21:19.878 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${COMPARE_PIDS[@]} 2024-09-12T13:21:19.878 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 66367 2024-09-12T13:21:20.865 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ promote_md5=ce4382640fa81da8b7b95c4017c84e83 2024-09-12T13:21:20.865 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo rbd --cluster cluster2 device unmap -t krbd /dev/rbd7 2024-09-12T13:21:20.978 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ ce4382640fa81da8b7b95c4017c84e83 != \c\e\4\3\8\2\6\4\0\f\a\8\1\d\a\8\b\7\b\9\5\c\4\0\1\7\c\8\4\e\8\3 ]] 2024-09-12T13:21:20.979 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${COMPARE_PIDS[@]} 2024-09-12T13:21:20.979 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 66369 2024-09-12T13:21:20.979 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${COMPARE_PIDS[@]} 2024-09-12T13:21:20.979 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 66371 2024-09-12T13:21:21.104 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ promote_md5=39dcba30863a72fe7cf39c1a605bbd65 2024-09-12T13:21:21.104 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo rbd --cluster cluster2 device unmap -t krbd /dev/rbd9 2024-09-12T13:21:21.193 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ 39dcba30863a72fe7cf39c1a605bbd65 != \3\9\d\c\b\a\3\0\8\6\3\a\7\2\f\e\7\c\f\3\9\c\1\a\6\0\5\b\b\d\6\5 ]] 2024-09-12T13:21:21.193 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${COMPARE_PIDS[@]} 2024-09-12T13:21:21.193 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 66373 2024-09-12T13:21:21.193 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${COMPARE_PIDS[@]} 2024-09-12T13:21:21.193 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 66375 2024-09-12T13:21:21.193 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${COMPARE_PIDS[@]} 2024-09-12T13:21:21.193 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 66377 2024-09-12T13:21:21.194 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${COMPARE_PIDS[@]} 2024-09-12T13:21:21.194 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 66379 2024-09-12T13:21:21.194 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ (( RET != 0 )) 2024-09-12T13:21:21.194 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:21:21.194 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ IMG=image-primary1 2024-09-12T13:21:21.194 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_replaying_status_in_pool_dir cluster1 mirror image-primary1 2024-09-12T13:21:21.194 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:21:21.194 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:21:21.194 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:21:21.195 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' snapshot = journal ']' 2024-09-12T13:21:21.195 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary1 up+replaying 2024-09-12T13:21:21.195 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:21:21.195 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:21:21.195 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:21:21.195 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:21:21.195 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:21:21.195 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:21:21.195 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:21:21.195 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:21:22.213 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary1 up+replaying '' '' 2024-09-12T13:21:22.213 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:21:22.213 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:21:22.213 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:21:22.213 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:21:22.213 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:21:22.213 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:21:22.213 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:21:22.214 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:21:22.214 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary1 2024-09-12T13:21:22.294 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary1: 2024-09-12T13:21:22.294 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 96ef9ec0-0a45-4e60-bcf0-79d28853db5e 2024-09-12T13:21:22.294 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+replaying 2024-09-12T13:21:22.295 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: replaying, {"bytes_per_second":3235225.6,"bytes_per_snapshot":64704512.0,"last_snapshot_bytes":64704512,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726147227,"remote_snapshot_timestamp":1726147227,"replay_state":"idle"} 2024-09-12T13:21:22.295 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:21:22.295 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:21:17' 2024-09-12T13:21:22.295 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+replaying' 2024-09-12T13:21:22.297 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+replaying 2024-09-12T13:21:22.298 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:21:22.300 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: replaying, {"bytes_per_second":3235225.6,"bytes_per_snapshot":64704512.0,"last_snapshot_bytes":64704512,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726147227,"remote_snapshot_timestamp":1726147227,"replay_state":"idle"} 2024-09-12T13:21:22.300 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:21:22.301 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+replaying 2024-09-12T13:21:22.301 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:21:22.303 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+replaying 2024-09-12T13:21:22.304 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:21:22.306 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:21:22.307 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:21:22.307 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:21:22.311 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:21:17' 2024-09-12T13:21:22.312 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary1 up+replaying '2024-09-12 13:21:17' 2024-09-12T13:21:22.312 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:21:22.312 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:21:22.312 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:21:22.312 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:21:22.312 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:21:17' 2024-09-12T13:21:22.312 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:21:22.312 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:21:22.313 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-09-12T13:21:22.418 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041934232mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary196ef9ec0-0a45-4e60-bcf0-79d28853db5eup+replayingreplaying, {"bytes_per_second":3235225.6,"bytes_per_snapshot":64704512.0,"last_snapshot_bytes":64704512,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726147227,"remote_snapshot_timestamp":1726147227,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:21:17image-primary10a41c1371-bc6b-4e5a-b8a6-2d2252dfda67up+replayingreplaying, {"bytes_per_second":3337420.8,"bytes_per_snapshot":66748416.0,"last_snapshot_bytes":66748416,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726147222,"remote_snapshot_timestamp":1726147222,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:21:17image-primary23ceee0af-566c-4f40-8fbb-b3b44ed7d010up+replayingreplaying, {"bytes_per_second":3348480.0,"bytes_per_snapshot":66969600.0,"last_snapshot_bytes":66969600,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726147226,"remote_snapshot_timestamp":1726147226,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:21:17image-primary350e30e91-e1e0-44f3-8ad3-6fb3a5307bbaup+replayingreplaying, {"bytes_per_second":3329433.6,"bytes_per_snapshot":66588672.0,"last_snapshot_bytes":66588672,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726147221,"remote_snapshot_timestamp":1726147221,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:21:17image-primary43d37a1b1-ea15-4b8b-8894-ae4f8b6c65cdup+replayingreplaying, {"bytes_per_second":3252633.6,"bytes_per_snapshot":65052672.0,"last_snapshot_bytes":65052672,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726147227,"remote_snapshot_timestamp":1726147227,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:21:17image-primary5cfd4ea21-2703-4ce9-a4e3-e41c240a1165up+replayingreplaying, {"bytes_per_second":3178496.0,"bytes_per_snapshot":63569920.0,"last_snapshot_bytes":63569920,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726147222,"remote_snapshot_timestamp":1726147222,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:21:17image-primary614efbe85-d4d9-42ff-aa35-0963a8746f30up+replayingreplaying, {"bytes_per_second":3246694.4,"bytes_per_snapshot":64933888.0,"last_snapshot_bytes":64933888,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726147227,"remote_snapshot_timestamp":1726147227,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:21:17image-primary73f634cfd-f07a-4271-8c29-0ad3777a474cup+replayingreplaying, {"bytes_per_second":3409510.4,"bytes_per_snapshot":68190208.0,"last_snapshot_bytes":68190208,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726147220,"remote_snapshot_timestamp":1726147220,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:21:17image-primary827e6f3f9-b993-4ad3-a6a6-74e8fc4feff4up+replayingreplaying, {"bytes_per_second":3413606.4,"bytes_per_snapshot":68272128.0,"last_snapshot_bytes":68272128,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726147225,"remote_snapshot_timestamp":1726147225,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:21:17image-primary9b8a82bf7-89cf-4fad-b167-0553ebb15a5eup+replayingreplaying, {"bytes_per_second":3309568.0,"bytes_per_snapshot":66191360.0,"last_snapshot_bytes":66191360,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726147226,"remote_snapshot_timestamp":1726147226,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:21:17' 2024-09-12T13:21:22.418 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:21:22.419 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary1'\'']/last_update' 2024-09-12T13:21:22.462 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:21:17' 2024-09-12T13:21:22.463 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary1'\'']/state' 2024-09-12T13:21:22.466 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+replaying 2024-09-12T13:21:22.467 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+replaying 2024-09-12T13:21:22.467 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+replaying 2024-09-12T13:21:22.469 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+replaying 2024-09-12T13:21:22.470 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:21:22.470 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:21:22.470 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ remove_image cluster2 mirror image-primary1 2024-09-12T13:21:22.471 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:21:22.471 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:21:22.471 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:21:22.471 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 snap purge mirror/image-primary1 2024-09-12T13:21:22.532 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 rm mirror/image-primary1 2024-09-12T13:21:22.666 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:21:22.666+0000 7f67e4800640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x56055bce9520 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-09-12T13:21:22.675 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:21:22.666+0000 7f67ed400640 -1 rbd::mirror::ImageReplayer: 0x5605561acf00 [3/96ef9ec0-0a45-4e60-bcf0-79d28853db5e] operator(): start failed: (85) Interrupted system call should be restarted 2024-09-12T13:21:24.260 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-09-12T13:21:24.266 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_image_removal cluster1 mirror image-primary1 2024-09-12T13:21:24.266 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:21:24.266 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:21:24.266 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:21:24.266 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:21:24.267 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 ls mirror 2024-09-12T13:21:24.267 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep -wq image-primary1 2024-09-12T13:21:24.318 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:21:24.319 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rm -rf test-primary1 2024-09-12T13:21:24.343 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:21:24.344 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ IMG=image-primary2 2024-09-12T13:21:24.344 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_replaying_status_in_pool_dir cluster1 mirror image-primary2 2024-09-12T13:21:24.344 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:21:24.344 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:21:24.344 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:21:24.344 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' snapshot = journal ']' 2024-09-12T13:21:24.344 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary2 up+replaying 2024-09-12T13:21:24.344 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:21:24.344 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:21:24.344 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:21:24.345 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:21:24.345 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:21:24.345 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:21:24.345 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:21:24.345 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:21:25.346 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary2 up+replaying '' '' 2024-09-12T13:21:25.346 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:21:25.346 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:21:25.347 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:21:25.347 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:21:25.347 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:21:25.347 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:21:25.347 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:21:25.347 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:21:25.348 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary2 2024-09-12T13:21:25.427 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary2: 2024-09-12T13:21:25.427 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 3ceee0af-566c-4f40-8fbb-b3b44ed7d010 2024-09-12T13:21:25.427 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+replaying 2024-09-12T13:21:25.427 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: replaying, {"bytes_per_second":3348480.0,"bytes_per_snapshot":66969600.0,"last_snapshot_bytes":66969600,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726147226,"remote_snapshot_timestamp":1726147226,"replay_state":"idle"} 2024-09-12T13:21:25.427 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:21:25.427 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:21:17' 2024-09-12T13:21:25.428 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+replaying' 2024-09-12T13:21:25.429 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+replaying 2024-09-12T13:21:25.430 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:21:25.432 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: replaying, {"bytes_per_second":3348480.0,"bytes_per_snapshot":66969600.0,"last_snapshot_bytes":66969600,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726147226,"remote_snapshot_timestamp":1726147226,"replay_state":"idle"} 2024-09-12T13:21:25.433 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:21:25.433 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+replaying 2024-09-12T13:21:25.434 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:21:25.436 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+replaying 2024-09-12T13:21:25.436 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:21:25.438 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:21:25.439 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:21:25.440 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:21:25.444 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:21:17' 2024-09-12T13:21:25.444 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary2 up+replaying '2024-09-12 13:21:17' 2024-09-12T13:21:25.444 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:21:25.444 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:21:25.444 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:21:25.444 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:21:25.444 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:21:17' 2024-09-12T13:21:25.444 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:21:25.444 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:21:25.445 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-09-12T13:21:25.549 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING941934232mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary10a41c1371-bc6b-4e5a-b8a6-2d2252dfda67up+replayingreplaying, {"bytes_per_second":3337420.8,"bytes_per_snapshot":66748416.0,"last_snapshot_bytes":66748416,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726147222,"remote_snapshot_timestamp":1726147222,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:21:17image-primary23ceee0af-566c-4f40-8fbb-b3b44ed7d010up+replayingreplaying, {"bytes_per_second":3348480.0,"bytes_per_snapshot":66969600.0,"last_snapshot_bytes":66969600,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726147226,"remote_snapshot_timestamp":1726147226,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:21:17image-primary350e30e91-e1e0-44f3-8ad3-6fb3a5307bbaup+replayingreplaying, {"bytes_per_second":3329433.6,"bytes_per_snapshot":66588672.0,"last_snapshot_bytes":66588672,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726147221,"remote_snapshot_timestamp":1726147221,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:21:17image-primary43d37a1b1-ea15-4b8b-8894-ae4f8b6c65cdup+replayingreplaying, {"bytes_per_second":3252633.6,"bytes_per_snapshot":65052672.0,"last_snapshot_bytes":65052672,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726147227,"remote_snapshot_timestamp":1726147227,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:21:17image-primary5cfd4ea21-2703-4ce9-a4e3-e41c240a1165up+replayingreplaying, {"bytes_per_second":3178496.0,"bytes_per_snapshot":63569920.0,"last_snapshot_bytes":63569920,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726147222,"remote_snapshot_timestamp":1726147222,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:21:17image-primary614efbe85-d4d9-42ff-aa35-0963a8746f30up+replayingreplaying, {"bytes_per_second":3246694.4,"bytes_per_snapshot":64933888.0,"last_snapshot_bytes":64933888,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726147227,"remote_snapshot_timestamp":1726147227,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:21:17image-primary73f634cfd-f07a-4271-8c29-0ad3777a474cup+replayingreplaying, {"bytes_per_second":3409510.4,"bytes_per_snapshot":68190208.0,"last_snapshot_bytes":68190208,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726147220,"remote_snapshot_timestamp":1726147220,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:21:17image-primary827e6f3f9-b993-4ad3-a6a6-74e8fc4feff4up+replayingreplaying, {"bytes_per_second":3413606.4,"bytes_per_snapshot":68272128.0,"last_snapshot_bytes":68272128,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726147225,"remote_snapshot_timestamp":1726147225,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:21:17image-primary9b8a82bf7-89cf-4fad-b167-0553ebb15a5eup+replayingreplaying, {"bytes_per_second":3309568.0,"bytes_per_snapshot":66191360.0,"last_snapshot_bytes":66191360,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726147226,"remote_snapshot_timestamp":1726147226,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:21:17' 2024-09-12T13:21:25.549 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:21:25.550 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary2'\'']/last_update' 2024-09-12T13:21:25.553 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:21:17' 2024-09-12T13:21:25.554 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary2'\'']/state' 2024-09-12T13:21:25.558 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+replaying 2024-09-12T13:21:25.558 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+replaying 2024-09-12T13:21:25.559 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+replaying 2024-09-12T13:21:25.561 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+replaying 2024-09-12T13:21:25.561 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:21:25.562 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:21:25.562 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ remove_image cluster2 mirror image-primary2 2024-09-12T13:21:25.562 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:21:25.562 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:21:25.562 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:21:25.562 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 snap purge mirror/image-primary2 2024-09-12T13:21:25.621 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 rm mirror/image-primary2 2024-09-12T13:21:25.752 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:21:25.753+0000 7f67e5200640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x560556582000 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-09-12T13:21:25.753 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:21:25.753+0000 7f67eca00640 -1 rbd::mirror::ImageReplayer: 0x56055c198780 [3/3ceee0af-566c-4f40-8fbb-b3b44ed7d010] operator(): start failed: (85) Interrupted system call should be restarted 2024-09-12T13:21:27.284 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-09-12T13:21:27.291 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_image_removal cluster1 mirror image-primary2 2024-09-12T13:21:27.291 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:21:27.291 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:21:27.291 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:21:27.291 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:21:27.292 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 ls mirror 2024-09-12T13:21:27.292 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep -wq image-primary2 2024-09-12T13:21:27.341 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:21:27.341 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rm -rf test-primary2 2024-09-12T13:21:27.343 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:21:27.343 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ IMG=image-primary3 2024-09-12T13:21:27.343 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_replaying_status_in_pool_dir cluster1 mirror image-primary3 2024-09-12T13:21:27.343 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:21:27.344 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:21:27.344 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:21:27.344 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' snapshot = journal ']' 2024-09-12T13:21:27.344 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary3 up+replaying 2024-09-12T13:21:27.344 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:21:27.344 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:21:27.344 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:21:27.344 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:21:27.344 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:21:27.344 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:21:27.344 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:21:27.344 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:21:28.345 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary3 up+replaying '' '' 2024-09-12T13:21:28.346 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:21:28.346 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:21:28.346 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:21:28.346 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:21:28.346 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:21:28.346 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:21:28.346 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:21:28.346 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:21:28.347 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary3 2024-09-12T13:21:28.424 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary3: 2024-09-12T13:21:28.424 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 50e30e91-e1e0-44f3-8ad3-6fb3a5307bba 2024-09-12T13:21:28.424 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+replaying 2024-09-12T13:21:28.424 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: replaying, {"bytes_per_second":3329433.6,"bytes_per_snapshot":66588672.0,"last_snapshot_bytes":66588672,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726147221,"remote_snapshot_timestamp":1726147221,"replay_state":"idle"} 2024-09-12T13:21:28.424 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:21:28.424 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:21:17' 2024-09-12T13:21:28.424 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+replaying' 2024-09-12T13:21:28.426 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+replaying 2024-09-12T13:21:28.426 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:21:28.429 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: replaying, {"bytes_per_second":3329433.6,"bytes_per_snapshot":66588672.0,"last_snapshot_bytes":66588672,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726147221,"remote_snapshot_timestamp":1726147221,"replay_state":"idle"} 2024-09-12T13:21:28.429 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:21:28.431 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+replaying 2024-09-12T13:21:28.431 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:21:28.432 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+replaying 2024-09-12T13:21:28.432 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:21:28.435 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:21:28.435 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:21:28.436 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:21:28.440 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:21:17' 2024-09-12T13:21:28.440 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary3 up+replaying '2024-09-12 13:21:17' 2024-09-12T13:21:28.440 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:21:28.440 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:21:28.440 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:21:28.440 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:21:28.440 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:21:17' 2024-09-12T13:21:28.440 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:21:28.441 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:21:28.441 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-09-12T13:21:28.540 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING841934232mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary10a41c1371-bc6b-4e5a-b8a6-2d2252dfda67up+replayingreplaying, {"bytes_per_second":3337420.8,"bytes_per_snapshot":66748416.0,"last_snapshot_bytes":66748416,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726147222,"remote_snapshot_timestamp":1726147222,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:21:17image-primary350e30e91-e1e0-44f3-8ad3-6fb3a5307bbaup+replayingreplaying, {"bytes_per_second":3329433.6,"bytes_per_snapshot":66588672.0,"last_snapshot_bytes":66588672,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726147221,"remote_snapshot_timestamp":1726147221,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:21:17image-primary43d37a1b1-ea15-4b8b-8894-ae4f8b6c65cdup+replayingreplaying, {"bytes_per_second":3252633.6,"bytes_per_snapshot":65052672.0,"last_snapshot_bytes":65052672,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726147227,"remote_snapshot_timestamp":1726147227,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:21:17image-primary5cfd4ea21-2703-4ce9-a4e3-e41c240a1165up+replayingreplaying, {"bytes_per_second":3178496.0,"bytes_per_snapshot":63569920.0,"last_snapshot_bytes":63569920,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726147222,"remote_snapshot_timestamp":1726147222,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:21:17image-primary614efbe85-d4d9-42ff-aa35-0963a8746f30up+replayingreplaying, {"bytes_per_second":3246694.4,"bytes_per_snapshot":64933888.0,"last_snapshot_bytes":64933888,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726147227,"remote_snapshot_timestamp":1726147227,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:21:17image-primary73f634cfd-f07a-4271-8c29-0ad3777a474cup+replayingreplaying, {"bytes_per_second":3409510.4,"bytes_per_snapshot":68190208.0,"last_snapshot_bytes":68190208,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726147220,"remote_snapshot_timestamp":1726147220,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:21:17image-primary827e6f3f9-b993-4ad3-a6a6-74e8fc4feff4up+replayingreplaying, {"bytes_per_second":3413606.4,"bytes_per_snapshot":68272128.0,"last_snapshot_bytes":68272128,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726147225,"remote_snapshot_timestamp":1726147225,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:21:17image-primary9b8a82bf7-89cf-4fad-b167-0553ebb15a5eup+replayingreplaying, {"bytes_per_second":3309568.0,"bytes_per_snapshot":66191360.0,"last_snapshot_bytes":66191360,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726147226,"remote_snapshot_timestamp":1726147226,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:21:17' 2024-09-12T13:21:28.540 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:21:28.541 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary3'\'']/last_update' 2024-09-12T13:21:28.544 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:21:17' 2024-09-12T13:21:28.545 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary3'\'']/state' 2024-09-12T13:21:28.549 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+replaying 2024-09-12T13:21:28.549 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+replaying 2024-09-12T13:21:28.550 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+replaying 2024-09-12T13:21:28.551 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+replaying 2024-09-12T13:21:28.552 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:21:28.552 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:21:28.552 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ remove_image cluster2 mirror image-primary3 2024-09-12T13:21:28.553 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:21:28.553 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:21:28.553 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:21:28.553 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 snap purge mirror/image-primary3 2024-09-12T13:21:28.613 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 rm mirror/image-primary3 2024-09-12T13:21:28.750 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:21:28.750+0000 7f67e5200640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x560556dfc9c0 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-09-12T13:21:28.750 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:21:28.750+0000 7f67eca00640 -1 rbd::mirror::ImageReplayer: 0x56055c198c80 [3/50e30e91-e1e0-44f3-8ad3-6fb3a5307bba] operator(): start failed: (85) Interrupted system call should be restarted 2024-09-12T13:21:29.991 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-09-12T13:21:29.997 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_image_removal cluster1 mirror image-primary3 2024-09-12T13:21:29.998 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:21:29.998 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:21:29.998 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:21:29.998 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:21:29.998 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 ls mirror 2024-09-12T13:21:29.998 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep -wq image-primary3 2024-09-12T13:21:30.051 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:21:30.051 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rm -rf test-primary3 2024-09-12T13:21:30.054 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:21:30.054 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ IMG=image-primary4 2024-09-12T13:21:30.054 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_replaying_status_in_pool_dir cluster1 mirror image-primary4 2024-09-12T13:21:30.054 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:21:30.054 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:21:30.054 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:21:30.054 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' snapshot = journal ']' 2024-09-12T13:21:30.054 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary4 up+replaying 2024-09-12T13:21:30.054 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:21:30.055 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:21:30.055 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:21:30.055 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:21:30.055 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:21:30.055 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:21:30.055 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:21:30.055 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:21:31.056 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary4 up+replaying '' '' 2024-09-12T13:21:31.056 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:21:31.056 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:21:31.056 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:21:31.057 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:21:31.057 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:21:31.057 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:21:31.057 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:21:31.057 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:21:31.057 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary4 2024-09-12T13:21:31.135 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary4: 2024-09-12T13:21:31.135 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 3d37a1b1-ea15-4b8b-8894-ae4f8b6c65cd 2024-09-12T13:21:31.135 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+replaying 2024-09-12T13:21:31.135 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: replaying, {"bytes_per_second":3252633.6,"bytes_per_snapshot":65052672.0,"last_snapshot_bytes":65052672,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726147227,"remote_snapshot_timestamp":1726147227,"replay_state":"idle"} 2024-09-12T13:21:31.135 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:21:31.135 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:21:17' 2024-09-12T13:21:31.135 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+replaying' 2024-09-12T13:21:31.140 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+replaying 2024-09-12T13:21:31.140 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:21:31.140 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: replaying, {"bytes_per_second":3252633.6,"bytes_per_snapshot":65052672.0,"last_snapshot_bytes":65052672,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726147227,"remote_snapshot_timestamp":1726147227,"replay_state":"idle"} 2024-09-12T13:21:31.141 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:21:31.141 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+replaying 2024-09-12T13:21:31.141 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:21:31.143 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+replaying 2024-09-12T13:21:31.143 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:21:31.146 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:21:31.146 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:21:31.147 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:21:31.151 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:21:17' 2024-09-12T13:21:31.151 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary4 up+replaying '2024-09-12 13:21:17' 2024-09-12T13:21:31.151 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:21:31.151 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:21:31.151 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:21:31.151 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:21:31.151 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:21:17' 2024-09-12T13:21:31.151 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:21:31.152 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:21:31.152 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-09-12T13:21:31.247 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING741934232mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary10a41c1371-bc6b-4e5a-b8a6-2d2252dfda67up+replayingreplaying, {"bytes_per_second":3337420.8,"bytes_per_snapshot":66748416.0,"last_snapshot_bytes":66748416,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726147222,"remote_snapshot_timestamp":1726147222,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:21:17image-primary43d37a1b1-ea15-4b8b-8894-ae4f8b6c65cdup+replayingreplaying, {"bytes_per_second":3252633.6,"bytes_per_snapshot":65052672.0,"last_snapshot_bytes":65052672,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726147227,"remote_snapshot_timestamp":1726147227,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:21:17image-primary5cfd4ea21-2703-4ce9-a4e3-e41c240a1165up+replayingreplaying, {"bytes_per_second":3178496.0,"bytes_per_snapshot":63569920.0,"last_snapshot_bytes":63569920,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726147222,"remote_snapshot_timestamp":1726147222,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:21:17image-primary614efbe85-d4d9-42ff-aa35-0963a8746f30up+replayingreplaying, {"bytes_per_second":3246694.4,"bytes_per_snapshot":64933888.0,"last_snapshot_bytes":64933888,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726147227,"remote_snapshot_timestamp":1726147227,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:21:17image-primary73f634cfd-f07a-4271-8c29-0ad3777a474cup+replayingreplaying, {"bytes_per_second":3409510.4,"bytes_per_snapshot":68190208.0,"last_snapshot_bytes":68190208,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726147220,"remote_snapshot_timestamp":1726147220,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:21:17image-primary827e6f3f9-b993-4ad3-a6a6-74e8fc4feff4up+replayingreplaying, {"bytes_per_second":3413606.4,"bytes_per_snapshot":68272128.0,"last_snapshot_bytes":68272128,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726147225,"remote_snapshot_timestamp":1726147225,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:21:17image-primary9b8a82bf7-89cf-4fad-b167-0553ebb15a5eup+replayingreplaying, {"bytes_per_second":3309568.0,"bytes_per_snapshot":66191360.0,"last_snapshot_bytes":66191360,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726147226,"remote_snapshot_timestamp":1726147226,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:21:17' 2024-09-12T13:21:31.248 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:21:31.248 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary4'\'']/last_update' 2024-09-12T13:21:31.252 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:21:17' 2024-09-12T13:21:31.252 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary4'\'']/state' 2024-09-12T13:21:31.257 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+replaying 2024-09-12T13:21:31.257 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+replaying 2024-09-12T13:21:31.257 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+replaying 2024-09-12T13:21:31.259 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+replaying 2024-09-12T13:21:31.260 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:21:31.260 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:21:31.260 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ remove_image cluster2 mirror image-primary4 2024-09-12T13:21:31.260 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:21:31.260 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:21:31.260 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:21:31.260 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 snap purge mirror/image-primary4 2024-09-12T13:21:31.320 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 rm mirror/image-primary4 2024-09-12T13:21:31.453 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:21:31.453+0000 7f67e5200640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x56055b864000 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-09-12T13:21:31.453 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:21:31.453+0000 7f67ed400640 -1 rbd::mirror::ImageReplayer: 0x56055c198500 [3/3d37a1b1-ea15-4b8b-8894-ae4f8b6c65cd] operator(): start failed: (85) Interrupted system call should be restarted 2024-09-12T13:21:33.302 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-09-12T13:21:33.309 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_image_removal cluster1 mirror image-primary4 2024-09-12T13:21:33.309 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:21:33.310 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:21:33.310 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:21:33.310 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:21:33.310 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 ls mirror 2024-09-12T13:21:33.310 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep -wq image-primary4 2024-09-12T13:21:33.359 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:21:33.359 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rm -rf test-primary4 2024-09-12T13:21:33.362 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:21:33.362 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ IMG=image-primary5 2024-09-12T13:21:33.362 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_replaying_status_in_pool_dir cluster1 mirror image-primary5 2024-09-12T13:21:33.362 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:21:33.362 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:21:33.362 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:21:33.362 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' snapshot = journal ']' 2024-09-12T13:21:33.362 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary5 up+replaying 2024-09-12T13:21:33.362 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:21:33.362 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:21:33.362 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:21:33.363 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:21:33.363 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:21:33.363 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:21:33.363 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:21:33.363 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:21:34.364 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary5 up+replaying '' '' 2024-09-12T13:21:34.365 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:21:34.365 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:21:34.365 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:21:34.365 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:21:34.365 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:21:34.365 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:21:34.365 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:21:34.365 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:21:34.365 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary5 2024-09-12T13:21:34.452 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary5: 2024-09-12T13:21:34.453 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: cfd4ea21-2703-4ce9-a4e3-e41c240a1165 2024-09-12T13:21:34.453 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+replaying 2024-09-12T13:21:34.453 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: replaying, {"bytes_per_second":3178496.0,"bytes_per_snapshot":63569920.0,"last_snapshot_bytes":63569920,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726147222,"remote_snapshot_timestamp":1726147222,"replay_state":"idle"} 2024-09-12T13:21:34.453 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:21:34.453 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:21:17' 2024-09-12T13:21:34.453 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+replaying' 2024-09-12T13:21:34.455 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+replaying 2024-09-12T13:21:34.455 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:21:34.458 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: replaying, {"bytes_per_second":3178496.0,"bytes_per_snapshot":63569920.0,"last_snapshot_bytes":63569920,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726147222,"remote_snapshot_timestamp":1726147222,"replay_state":"idle"} 2024-09-12T13:21:34.458 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:21:34.459 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+replaying 2024-09-12T13:21:34.459 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:21:34.461 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+replaying 2024-09-12T13:21:34.461 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:21:34.464 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:21:34.464 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:21:34.465 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:21:34.469 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:21:17' 2024-09-12T13:21:34.469 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary5 up+replaying '2024-09-12 13:21:17' 2024-09-12T13:21:34.469 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:21:34.470 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:21:34.470 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:21:34.470 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:21:34.470 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:21:17' 2024-09-12T13:21:34.470 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:21:34.470 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:21:34.470 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-09-12T13:21:34.567 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING641934232mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary10a41c1371-bc6b-4e5a-b8a6-2d2252dfda67up+replayingreplaying, {"bytes_per_second":3337420.8,"bytes_per_snapshot":66748416.0,"last_snapshot_bytes":66748416,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726147222,"remote_snapshot_timestamp":1726147222,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:21:17image-primary5cfd4ea21-2703-4ce9-a4e3-e41c240a1165up+replayingreplaying, {"bytes_per_second":3178496.0,"bytes_per_snapshot":63569920.0,"last_snapshot_bytes":63569920,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726147222,"remote_snapshot_timestamp":1726147222,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:21:17image-primary614efbe85-d4d9-42ff-aa35-0963a8746f30up+replayingreplaying, {"bytes_per_second":3246694.4,"bytes_per_snapshot":64933888.0,"last_snapshot_bytes":64933888,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726147227,"remote_snapshot_timestamp":1726147227,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:21:17image-primary73f634cfd-f07a-4271-8c29-0ad3777a474cup+replayingreplaying, {"bytes_per_second":3409510.4,"bytes_per_snapshot":68190208.0,"last_snapshot_bytes":68190208,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726147220,"remote_snapshot_timestamp":1726147220,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:21:17image-primary827e6f3f9-b993-4ad3-a6a6-74e8fc4feff4up+replayingreplaying, {"bytes_per_second":3413606.4,"bytes_per_snapshot":68272128.0,"last_snapshot_bytes":68272128,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726147225,"remote_snapshot_timestamp":1726147225,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:21:17image-primary9b8a82bf7-89cf-4fad-b167-0553ebb15a5eup+replayingreplaying, {"bytes_per_second":3309568.0,"bytes_per_snapshot":66191360.0,"last_snapshot_bytes":66191360,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726147226,"remote_snapshot_timestamp":1726147226,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:21:17' 2024-09-12T13:21:34.567 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:21:34.567 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary5'\'']/last_update' 2024-09-12T13:21:34.571 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:21:17' 2024-09-12T13:21:34.572 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary5'\'']/state' 2024-09-12T13:21:34.576 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+replaying 2024-09-12T13:21:34.576 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+replaying 2024-09-12T13:21:34.577 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+replaying 2024-09-12T13:21:34.579 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+replaying 2024-09-12T13:21:34.579 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:21:34.580 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:21:34.580 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ remove_image cluster2 mirror image-primary5 2024-09-12T13:21:34.580 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:21:34.580 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:21:34.580 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:21:34.580 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 snap purge mirror/image-primary5 2024-09-12T13:21:34.640 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 rm mirror/image-primary5 2024-09-12T13:21:34.776 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:21:34.776+0000 7f67e4800640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x56055be69a00 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-09-12T13:21:34.776 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:21:34.776+0000 7f67eca00640 -1 rbd::mirror::ImageReplayer: 0x5605561ad400 [3/cfd4ea21-2703-4ce9-a4e3-e41c240a1165] operator(): start failed: (85) Interrupted system call should be restarted 2024-09-12T13:21:35.990 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-09-12T13:21:35.996 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_image_removal cluster1 mirror image-primary5 2024-09-12T13:21:35.997 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:21:35.997 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:21:35.997 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:21:35.997 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:21:35.997 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 ls mirror 2024-09-12T13:21:35.997 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep -wq image-primary5 2024-09-12T13:21:36.046 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:21:36.046 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rm -rf test-primary5 2024-09-12T13:21:36.049 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:21:36.049 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ IMG=image-primary6 2024-09-12T13:21:36.049 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_replaying_status_in_pool_dir cluster1 mirror image-primary6 2024-09-12T13:21:36.049 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:21:36.049 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:21:36.049 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:21:36.049 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' snapshot = journal ']' 2024-09-12T13:21:36.049 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary6 up+replaying 2024-09-12T13:21:36.049 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:21:36.050 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:21:36.050 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:21:36.050 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:21:36.050 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:21:36.050 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:21:36.050 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:21:36.050 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:21:37.051 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary6 up+replaying '' '' 2024-09-12T13:21:37.051 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:21:37.051 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:21:37.051 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:21:37.051 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:21:37.051 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:21:37.051 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:21:37.052 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:21:37.052 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:21:37.052 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary6 2024-09-12T13:21:37.151 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary6: 2024-09-12T13:21:37.151 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 14efbe85-d4d9-42ff-aa35-0963a8746f30 2024-09-12T13:21:37.152 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+replaying 2024-09-12T13:21:37.152 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: replaying, {"bytes_per_second":3246694.4,"bytes_per_snapshot":64933888.0,"last_snapshot_bytes":64933888,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726147227,"remote_snapshot_timestamp":1726147227,"replay_state":"idle"} 2024-09-12T13:21:37.152 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:21:37.152 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:21:17' 2024-09-12T13:21:37.152 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+replaying' 2024-09-12T13:21:37.154 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+replaying 2024-09-12T13:21:37.154 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:21:37.157 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: replaying, {"bytes_per_second":3246694.4,"bytes_per_snapshot":64933888.0,"last_snapshot_bytes":64933888,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726147227,"remote_snapshot_timestamp":1726147227,"replay_state":"idle"} 2024-09-12T13:21:37.157 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:21:37.158 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+replaying 2024-09-12T13:21:37.158 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:21:37.160 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+replaying 2024-09-12T13:21:37.161 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:21:37.163 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:21:37.164 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:21:37.165 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:21:37.169 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:21:17' 2024-09-12T13:21:37.169 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary6 up+replaying '2024-09-12 13:21:17' 2024-09-12T13:21:37.169 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:21:37.169 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:21:37.169 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:21:37.169 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:21:37.169 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:21:17' 2024-09-12T13:21:37.169 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:21:37.170 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:21:37.170 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-09-12T13:21:37.298 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING541934232mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary10a41c1371-bc6b-4e5a-b8a6-2d2252dfda67up+replayingreplaying, {"bytes_per_second":3337420.8,"bytes_per_snapshot":66748416.0,"last_snapshot_bytes":66748416,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726147222,"remote_snapshot_timestamp":1726147222,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:21:17image-primary614efbe85-d4d9-42ff-aa35-0963a8746f30up+replayingreplaying, {"bytes_per_second":3246694.4,"bytes_per_snapshot":64933888.0,"last_snapshot_bytes":64933888,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726147227,"remote_snapshot_timestamp":1726147227,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:21:17image-primary73f634cfd-f07a-4271-8c29-0ad3777a474cup+replayingreplaying, {"bytes_per_second":3409510.4,"bytes_per_snapshot":68190208.0,"last_snapshot_bytes":68190208,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726147220,"remote_snapshot_timestamp":1726147220,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:21:17image-primary827e6f3f9-b993-4ad3-a6a6-74e8fc4feff4up+replayingreplaying, {"bytes_per_second":3413606.4,"bytes_per_snapshot":68272128.0,"last_snapshot_bytes":68272128,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726147225,"remote_snapshot_timestamp":1726147225,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:21:17image-primary9b8a82bf7-89cf-4fad-b167-0553ebb15a5eup+replayingreplaying, {"bytes_per_second":3309568.0,"bytes_per_snapshot":66191360.0,"last_snapshot_bytes":66191360,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726147226,"remote_snapshot_timestamp":1726147226,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:21:17' 2024-09-12T13:21:37.298 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:21:37.299 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary6'\'']/last_update' 2024-09-12T13:21:37.302 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:21:17' 2024-09-12T13:21:37.303 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary6'\'']/state' 2024-09-12T13:21:37.307 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+replaying 2024-09-12T13:21:37.308 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+replaying 2024-09-12T13:21:37.308 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+replaying 2024-09-12T13:21:37.310 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+replaying 2024-09-12T13:21:37.311 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:21:37.311 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:21:37.311 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ remove_image cluster2 mirror image-primary6 2024-09-12T13:21:37.311 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:21:37.311 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:21:37.311 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:21:37.311 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 snap purge mirror/image-primary6 2024-09-12T13:21:37.373 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 rm mirror/image-primary6 2024-09-12T13:21:37.576 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:21:37.575+0000 7f67e4800640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x56055c016b60 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-09-12T13:21:37.576 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:21:37.575+0000 7f67eca00640 -1 rbd::mirror::ImageReplayer: 0x5605561ac500 [3/14efbe85-d4d9-42ff-aa35-0963a8746f30] operator(): start failed: (85) Interrupted system call should be restarted 2024-09-12T13:21:38.987 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-09-12T13:21:38.993 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_image_removal cluster1 mirror image-primary6 2024-09-12T13:21:38.993 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:21:38.993 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:21:38.993 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:21:38.993 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:21:38.994 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 ls mirror 2024-09-12T13:21:38.994 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep -wq image-primary6 2024-09-12T13:21:39.043 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:21:39.043 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rm -rf test-primary6 2024-09-12T13:21:39.045 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:21:39.045 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ IMG=image-primary7 2024-09-12T13:21:39.046 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_replaying_status_in_pool_dir cluster1 mirror image-primary7 2024-09-12T13:21:39.046 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:21:39.046 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:21:39.046 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:21:39.046 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' snapshot = journal ']' 2024-09-12T13:21:39.046 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary7 up+replaying 2024-09-12T13:21:39.046 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:21:39.046 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:21:39.046 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:21:39.046 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:21:39.046 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:21:39.046 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:21:39.046 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:21:39.047 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:21:40.048 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary7 up+replaying '' '' 2024-09-12T13:21:40.048 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:21:40.048 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:21:40.048 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:21:40.048 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:21:40.048 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:21:40.049 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:21:40.049 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:21:40.049 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:21:40.049 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary7 2024-09-12T13:21:40.129 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary7: 2024-09-12T13:21:40.130 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 3f634cfd-f07a-4271-8c29-0ad3777a474c 2024-09-12T13:21:40.130 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+replaying 2024-09-12T13:21:40.130 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: replaying, {"bytes_per_second":3409510.4,"bytes_per_snapshot":68190208.0,"last_snapshot_bytes":68190208,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726147220,"remote_snapshot_timestamp":1726147220,"replay_state":"idle"} 2024-09-12T13:21:40.130 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:21:40.130 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:21:17' 2024-09-12T13:21:40.130 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+replaying' 2024-09-12T13:21:40.132 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+replaying 2024-09-12T13:21:40.132 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:21:40.135 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: replaying, {"bytes_per_second":3409510.4,"bytes_per_snapshot":68190208.0,"last_snapshot_bytes":68190208,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726147220,"remote_snapshot_timestamp":1726147220,"replay_state":"idle"} 2024-09-12T13:21:40.135 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:21:40.136 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+replaying 2024-09-12T13:21:40.136 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:21:40.138 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+replaying 2024-09-12T13:21:40.138 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:21:40.141 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:21:40.141 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:21:40.142 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:21:40.146 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:21:17' 2024-09-12T13:21:40.146 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary7 up+replaying '2024-09-12 13:21:17' 2024-09-12T13:21:40.146 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:21:40.146 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:21:40.146 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:21:40.147 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:21:40.147 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:21:17' 2024-09-12T13:21:40.147 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:21:40.147 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:21:40.147 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-09-12T13:21:40.237 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING441934232mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary10a41c1371-bc6b-4e5a-b8a6-2d2252dfda67up+replayingreplaying, {"bytes_per_second":3337420.8,"bytes_per_snapshot":66748416.0,"last_snapshot_bytes":66748416,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726147222,"remote_snapshot_timestamp":1726147222,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:21:17image-primary73f634cfd-f07a-4271-8c29-0ad3777a474cup+replayingreplaying, {"bytes_per_second":3409510.4,"bytes_per_snapshot":68190208.0,"last_snapshot_bytes":68190208,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726147220,"remote_snapshot_timestamp":1726147220,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:21:17image-primary827e6f3f9-b993-4ad3-a6a6-74e8fc4feff4up+replayingreplaying, {"bytes_per_second":3413606.4,"bytes_per_snapshot":68272128.0,"last_snapshot_bytes":68272128,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726147225,"remote_snapshot_timestamp":1726147225,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:21:17image-primary9b8a82bf7-89cf-4fad-b167-0553ebb15a5eup+replayingreplaying, {"bytes_per_second":3309568.0,"bytes_per_snapshot":66191360.0,"last_snapshot_bytes":66191360,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726147226,"remote_snapshot_timestamp":1726147226,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:21:17' 2024-09-12T13:21:40.237 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:21:40.238 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary7'\'']/last_update' 2024-09-12T13:21:40.242 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:21:17' 2024-09-12T13:21:40.242 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary7'\'']/state' 2024-09-12T13:21:40.246 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+replaying 2024-09-12T13:21:40.247 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+replaying 2024-09-12T13:21:40.247 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+replaying 2024-09-12T13:21:40.249 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+replaying 2024-09-12T13:21:40.250 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:21:40.250 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:21:40.250 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ remove_image cluster2 mirror image-primary7 2024-09-12T13:21:40.250 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:21:40.250 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:21:40.250 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:21:40.250 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 snap purge mirror/image-primary7 2024-09-12T13:21:40.318 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 rm mirror/image-primary7 2024-09-12T13:21:40.451 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:21:40.451+0000 7f67e5200640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x56055bafa000 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-09-12T13:21:40.451 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:21:40.451+0000 7f67ed400640 -1 rbd::mirror::ImageReplayer: 0x56055c199680 [3/3f634cfd-f07a-4271-8c29-0ad3777a474c] operator(): start failed: (85) Interrupted system call should be restarted 2024-09-12T13:21:42.013 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-09-12T13:21:42.019 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_image_removal cluster1 mirror image-primary7 2024-09-12T13:21:42.019 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:21:42.019 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:21:42.019 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:21:42.019 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:21:42.019 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 ls mirror 2024-09-12T13:21:42.020 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep -wq image-primary7 2024-09-12T13:21:42.070 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:21:42.070 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rm -rf test-primary7 2024-09-12T13:21:42.073 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:21:42.073 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ IMG=image-primary8 2024-09-12T13:21:42.073 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_replaying_status_in_pool_dir cluster1 mirror image-primary8 2024-09-12T13:21:42.073 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:21:42.073 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:21:42.074 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:21:42.074 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' snapshot = journal ']' 2024-09-12T13:21:42.074 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary8 up+replaying 2024-09-12T13:21:42.074 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:21:42.074 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:21:42.074 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:21:42.074 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:21:42.074 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:21:42.074 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:21:42.074 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:21:42.074 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:21:43.075 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary8 up+replaying '' '' 2024-09-12T13:21:43.075 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:21:43.075 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:21:43.075 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:21:43.075 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:21:43.075 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:21:43.075 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:21:43.076 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:21:43.076 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:21:43.076 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary8 2024-09-12T13:21:43.156 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary8: 2024-09-12T13:21:43.156 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 27e6f3f9-b993-4ad3-a6a6-74e8fc4feff4 2024-09-12T13:21:43.156 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+replaying 2024-09-12T13:21:43.156 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: replaying, {"bytes_per_second":3413606.4,"bytes_per_snapshot":68272128.0,"last_snapshot_bytes":68272128,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726147225,"remote_snapshot_timestamp":1726147225,"replay_state":"idle"} 2024-09-12T13:21:43.156 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:21:43.156 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:21:17' 2024-09-12T13:21:43.156 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+replaying' 2024-09-12T13:21:43.158 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+replaying 2024-09-12T13:21:43.159 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:21:43.161 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: replaying, {"bytes_per_second":3413606.4,"bytes_per_snapshot":68272128.0,"last_snapshot_bytes":68272128,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726147225,"remote_snapshot_timestamp":1726147225,"replay_state":"idle"} 2024-09-12T13:21:43.162 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:21:43.162 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+replaying 2024-09-12T13:21:43.162 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:21:43.164 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+replaying 2024-09-12T13:21:43.165 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:21:43.167 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:21:43.167 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:21:43.168 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:21:43.172 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:21:17' 2024-09-12T13:21:43.172 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary8 up+replaying '2024-09-12 13:21:17' 2024-09-12T13:21:43.172 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:21:43.172 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:21:43.172 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:21:43.172 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:21:43.172 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:21:17' 2024-09-12T13:21:43.172 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:21:43.173 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:21:43.173 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-09-12T13:21:43.258 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING341934232mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary10a41c1371-bc6b-4e5a-b8a6-2d2252dfda67up+replayingreplaying, {"bytes_per_second":3337420.8,"bytes_per_snapshot":66748416.0,"last_snapshot_bytes":66748416,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726147222,"remote_snapshot_timestamp":1726147222,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:21:17image-primary827e6f3f9-b993-4ad3-a6a6-74e8fc4feff4up+replayingreplaying, {"bytes_per_second":3413606.4,"bytes_per_snapshot":68272128.0,"last_snapshot_bytes":68272128,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726147225,"remote_snapshot_timestamp":1726147225,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:21:17image-primary9b8a82bf7-89cf-4fad-b167-0553ebb15a5eup+replayingreplaying, {"bytes_per_second":3309568.0,"bytes_per_snapshot":66191360.0,"last_snapshot_bytes":66191360,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726147226,"remote_snapshot_timestamp":1726147226,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:21:17' 2024-09-12T13:21:43.258 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:21:43.259 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary8'\'']/last_update' 2024-09-12T13:21:43.262 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:21:17' 2024-09-12T13:21:43.263 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary8'\'']/state' 2024-09-12T13:21:43.267 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+replaying 2024-09-12T13:21:43.267 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+replaying 2024-09-12T13:21:43.268 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+replaying 2024-09-12T13:21:43.269 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+replaying 2024-09-12T13:21:43.270 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:21:43.270 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:21:43.270 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ remove_image cluster2 mirror image-primary8 2024-09-12T13:21:43.271 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:21:43.271 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:21:43.271 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:21:43.271 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 snap purge mirror/image-primary8 2024-09-12T13:21:43.332 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 rm mirror/image-primary8 2024-09-12T13:21:43.471 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:21:43.471+0000 7f67e5200640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x56055be69040 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-09-12T13:21:43.472 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:21:43.471+0000 7f67eca00640 -1 rbd::mirror::ImageReplayer: 0x56055c199180 [3/27e6f3f9-b993-4ad3-a6a6-74e8fc4feff4] operator(): start failed: (85) Interrupted system call should be restarted 2024-09-12T13:21:45.005 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-09-12T13:21:45.012 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_image_removal cluster1 mirror image-primary8 2024-09-12T13:21:45.012 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:21:45.012 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:21:45.012 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:21:45.012 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:21:45.013 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 ls mirror 2024-09-12T13:21:45.013 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep -wq image-primary8 2024-09-12T13:21:45.069 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:21:45.069 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rm -rf test-primary8 2024-09-12T13:21:45.071 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:21:45.071 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ IMG=image-primary9 2024-09-12T13:21:45.071 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_replaying_status_in_pool_dir cluster1 mirror image-primary9 2024-09-12T13:21:45.071 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:21:45.071 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:21:45.071 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:21:45.072 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' snapshot = journal ']' 2024-09-12T13:21:45.072 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary9 up+replaying 2024-09-12T13:21:45.072 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:21:45.072 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:21:45.072 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:21:45.072 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:21:45.072 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:21:45.072 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:21:45.072 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:21:45.072 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:21:46.073 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary9 up+replaying '' '' 2024-09-12T13:21:46.073 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:21:46.074 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:21:46.074 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:21:46.074 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:21:46.074 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:21:46.074 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:21:46.074 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:21:46.074 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:21:46.074 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary9 2024-09-12T13:21:46.171 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary9: 2024-09-12T13:21:46.171 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: b8a82bf7-89cf-4fad-b167-0553ebb15a5e 2024-09-12T13:21:46.171 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+replaying 2024-09-12T13:21:46.172 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: replaying, {"bytes_per_second":3309568.0,"bytes_per_snapshot":66191360.0,"last_snapshot_bytes":66191360,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726147226,"remote_snapshot_timestamp":1726147226,"replay_state":"idle"} 2024-09-12T13:21:46.172 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:21:46.172 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:21:17' 2024-09-12T13:21:46.172 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+replaying' 2024-09-12T13:21:46.174 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+replaying 2024-09-12T13:21:46.174 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:21:46.176 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: replaying, {"bytes_per_second":3309568.0,"bytes_per_snapshot":66191360.0,"last_snapshot_bytes":66191360,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726147226,"remote_snapshot_timestamp":1726147226,"replay_state":"idle"} 2024-09-12T13:21:46.177 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:21:46.177 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+replaying 2024-09-12T13:21:46.178 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:21:46.180 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+replaying 2024-09-12T13:21:46.180 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:21:46.183 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:21:46.183 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:21:46.184 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:21:46.188 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:21:17' 2024-09-12T13:21:46.188 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary9 up+replaying '2024-09-12 13:21:17' 2024-09-12T13:21:46.188 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:21:46.188 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:21:46.188 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:21:46.188 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:21:46.189 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:21:17' 2024-09-12T13:21:46.189 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:21:46.189 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:21:46.189 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-09-12T13:21:46.308 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING241934232mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary10a41c1371-bc6b-4e5a-b8a6-2d2252dfda67up+replayingreplaying, {"bytes_per_second":3337420.8,"bytes_per_snapshot":66748416.0,"last_snapshot_bytes":66748416,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726147222,"remote_snapshot_timestamp":1726147222,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:21:17image-primary9b8a82bf7-89cf-4fad-b167-0553ebb15a5eup+replayingreplaying, {"bytes_per_second":3309568.0,"bytes_per_snapshot":66191360.0,"last_snapshot_bytes":66191360,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726147226,"remote_snapshot_timestamp":1726147226,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:21:17' 2024-09-12T13:21:46.308 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:21:46.309 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary9'\'']/last_update' 2024-09-12T13:21:46.313 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:21:17' 2024-09-12T13:21:46.313 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary9'\'']/state' 2024-09-12T13:21:46.317 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+replaying 2024-09-12T13:21:46.318 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+replaying 2024-09-12T13:21:46.318 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+replaying 2024-09-12T13:21:46.320 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+replaying 2024-09-12T13:21:46.321 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:21:46.321 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:21:46.321 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ remove_image cluster2 mirror image-primary9 2024-09-12T13:21:46.321 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:21:46.321 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:21:46.322 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:21:46.322 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 snap purge mirror/image-primary9 2024-09-12T13:21:46.382 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 rm mirror/image-primary9 2024-09-12T13:21:46.569 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:21:46.569+0000 7f67e5200640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x56055b7b5860 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-09-12T13:21:46.569 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:21:46.569+0000 7f67ed400640 -1 rbd::mirror::ImageReplayer: 0x5605573ca000 [3/b8a82bf7-89cf-4fad-b167-0553ebb15a5e] operator(): start failed: (85) Interrupted system call should be restarted 2024-09-12T13:21:48.163 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-09-12T13:21:48.169 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_image_removal cluster1 mirror image-primary9 2024-09-12T13:21:48.169 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:21:48.169 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:21:48.169 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:21:48.169 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:21:48.169 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 ls mirror 2024-09-12T13:21:48.169 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep -wq image-primary9 2024-09-12T13:21:48.220 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:21:48.220 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rm -rf test-primary9 2024-09-12T13:21:48.222 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:21:48.222 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ IMG=image-primary10 2024-09-12T13:21:48.223 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_replaying_status_in_pool_dir cluster1 mirror image-primary10 2024-09-12T13:21:48.223 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:21:48.223 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:21:48.223 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:21:48.223 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' snapshot = journal ']' 2024-09-12T13:21:48.223 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary10 up+replaying 2024-09-12T13:21:48.223 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:21:48.223 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:21:48.223 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:21:48.223 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:21:48.223 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:21:48.223 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:21:48.223 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:21:48.224 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:21:49.225 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary10 up+replaying '' '' 2024-09-12T13:21:49.225 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:21:49.225 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:21:49.225 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:21:49.225 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:21:49.225 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:21:49.225 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:21:49.225 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:21:49.226 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:21:49.226 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary10 2024-09-12T13:21:49.299 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary10: 2024-09-12T13:21:49.300 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: a41c1371-bc6b-4e5a-b8a6-2d2252dfda67 2024-09-12T13:21:49.300 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+replaying 2024-09-12T13:21:49.300 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: replaying, {"bytes_per_second":0.0,"bytes_per_snapshot":66748416.0,"last_snapshot_bytes":66748416,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726147222,"remote_snapshot_timestamp":1726147222,"replay_state":"idle"} 2024-09-12T13:21:49.300 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:21:49.300 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:21:47' 2024-09-12T13:21:49.300 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+replaying' 2024-09-12T13:21:49.302 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+replaying 2024-09-12T13:21:49.302 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:21:49.304 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: replaying, {"bytes_per_second":0.0,"bytes_per_snapshot":66748416.0,"last_snapshot_bytes":66748416,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726147222,"remote_snapshot_timestamp":1726147222,"replay_state":"idle"} 2024-09-12T13:21:49.305 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:21:49.305 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+replaying 2024-09-12T13:21:49.306 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:21:49.307 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+replaying 2024-09-12T13:21:49.308 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:21:49.310 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:21:49.311 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:21:49.311 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:21:49.315 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:21:47' 2024-09-12T13:21:49.315 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary10 up+replaying '2024-09-12 13:21:47' 2024-09-12T13:21:49.315 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:21:49.316 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:21:49.316 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:21:49.316 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:21:49.316 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:21:47' 2024-09-12T13:21:49.316 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:21:49.316 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:21:49.316 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-09-12T13:21:49.399 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING141934232mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary10a41c1371-bc6b-4e5a-b8a6-2d2252dfda67up+replayingreplaying, {"bytes_per_second":0.0,"bytes_per_snapshot":66748416.0,"last_snapshot_bytes":66748416,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726147222,"remote_snapshot_timestamp":1726147222,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:21:47' 2024-09-12T13:21:49.399 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:21:49.399 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary10'\'']/last_update' 2024-09-12T13:21:49.403 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:21:47' 2024-09-12T13:21:49.404 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary10'\'']/state' 2024-09-12T13:21:49.408 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+replaying 2024-09-12T13:21:49.408 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+replaying 2024-09-12T13:21:49.409 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+replaying 2024-09-12T13:21:49.411 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+replaying 2024-09-12T13:21:49.411 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:21:49.411 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:21:49.411 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ remove_image cluster2 mirror image-primary10 2024-09-12T13:21:49.412 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:21:49.412 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:21:49.412 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:21:49.412 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 snap purge mirror/image-primary10 2024-09-12T13:21:49.472 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 rm mirror/image-primary10 2024-09-12T13:21:49.609 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:21:49.609+0000 7f67e4800640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x5605557d4680 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-09-12T13:21:49.610 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:21:49.609+0000 7f67ed400640 -1 rbd::mirror::ImageReplayer: 0x560556cef180 [3/a41c1371-bc6b-4e5a-b8a6-2d2252dfda67] operator(): start failed: (85) Interrupted system call should be restarted 2024-09-12T13:21:51.019 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-09-12T13:21:51.025 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_image_removal cluster1 mirror image-primary10 2024-09-12T13:21:51.025 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:21:51.025 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:21:51.025 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:21:51.026 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:21:51.026 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 ls mirror 2024-09-12T13:21:51.026 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep -wq image-primary10 2024-09-12T13:21:51.075 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:21:51.075 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rm -rf test-primary10 2024-09-12T13:21:51.079 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..10} 2024-09-12T13:21:51.079 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEVS=() 2024-09-12T13:21:51.079 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ SNAP_PIDS=() 2024-09-12T13:21:51.079 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ COMPARE_PIDS=() 2024-09-12T13:21:51.080 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ WORKLOAD_PIDS=() 2024-09-12T13:21:51.080 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ RET=0 2024-09-12T13:21:51.080 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:21:51.080 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ IMG=image-primary1 2024-09-12T13:21:51.080 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ MNTPT=test-primary1 2024-09-12T13:21:51.080 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ create_image_and_enable_mirror cluster1 mirror image-primary1 snapshot 10G 2024-09-12T13:21:51.080 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:21:51.081 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:21:51.081 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:21:51.081 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:21:51.081 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:21:51.081 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:21:51.081 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mode=snapshot 2024-09-12T13:21:51.082 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n snapshot ']' 2024-09-12T13:21:51.082 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:21:51.082 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ create_image cluster1 mirror image-primary1 10G 2024-09-12T13:21:51.082 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:21:51.082 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:21:51.082 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:21:51.082 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:21:51.083 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:21:51.083 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:21:51.083 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local size=128 2024-09-12T13:21:51.083 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n 10G ']' 2024-09-12T13:21:51.083 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ size=10G 2024-09-12T13:21:51.083 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:21:51.084 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 create --size 10G --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image-primary1 2024-09-12T13:21:51.148 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ is_pool_mirror_mode_image mirror 2024-09-12T13:21:51.148 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:21:51.148 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' image = image ']' 2024-09-12T13:21:51.148 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:21:51.149 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ enable_mirror cluster1 mirror image-primary1 snapshot 2024-09-12T13:21:51.149 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:21:51.149 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:21:51.149 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:21:51.149 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mode=snapshot 2024-09-12T13:21:51.149 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 mirror image enable mirror/image-primary1 snapshot 2024-09-12T13:21:51.995 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Mirroring enabled 2024-09-12T13:21:52.010 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 info mirror/image-primary1 2024-09-12T13:21:52.073 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:rbd image 'image-primary1': 2024-09-12T13:21:52.073 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: size 10 GiB in 2560 objects 2024-09-12T13:21:52.073 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: order 22 (4 MiB objects) 2024-09-12T13:21:52.073 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: snapshot_count: 1 2024-09-12T13:21:52.073 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: id: 1f6d492a3024 2024-09-12T13:21:52.073 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: block_name_prefix: rbd_data.1f6d492a3024 2024-09-12T13:21:52.073 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: format: 2 2024-09-12T13:21:52.073 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: features: layering, exclusive-lock, object-map, fast-diff 2024-09-12T13:21:52.073 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: op_features: 2024-09-12T13:21:52.073 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: flags: 2024-09-12T13:21:52.074 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: create_timestamp: Thu Sep 12 13:21:51 2024 2024-09-12T13:21:52.074 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: access_timestamp: Thu Sep 12 13:21:51 2024 2024-09-12T13:21:52.074 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: modify_timestamp: Thu Sep 12 13:21:51 2024 2024-09-12T13:21:52.074 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring state: enabled 2024-09-12T13:21:52.074 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring mode: snapshot 2024-09-12T13:21:52.074 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring global id: 8bc720a8-a8a7-45cb-ab9a-5646a2cc4cc4 2024-09-12T13:21:52.074 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring primary: true 2024-09-12T13:21:52.079 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \n\b\d ]] 2024-09-12T13:21:52.079 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \k\r\b\d ]] 2024-09-12T13:21:52.080 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo rbd --cluster cluster1 device map -t krbd mirror/image-primary1 2024-09-12T13:21:52.223 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEV=/dev/rbd0 2024-09-12T13:21:52.223 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEVS+=($DEV) 2024-09-12T13:21:52.223 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo mkfs.ext4 /dev/rbd0 2024-09-12T13:21:52.265 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:mke2fs 1.46.5 (30-Dec-2021) 2024-09-12T13:21:52.276 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Discarding device blocks: 0/2621440 done 2024-09-12T13:21:52.276 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Creating filesystem with 2621440 4k blocks and 655360 inodes 2024-09-12T13:21:52.277 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Filesystem UUID: da1087eb-fc3d-4f8f-b2fb-5805995f9067 2024-09-12T13:21:52.277 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Superblock backups stored on blocks: 2024-09-12T13:21:52.277 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 2024-09-12T13:21:52.277 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 2024-09-12T13:21:52.277 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Allocating group tables: 0/80 done 2024-09-12T13:21:52.277 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Writing inode tables: 0/80 done 2024-09-12T13:21:52.281 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Creating journal (16384 blocks): done 2024-09-12T13:21:52.356 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Writing superblocks and filesystem accounting information: 0/80 done 2024-09-12T13:21:52.356 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 2024-09-12T13:21:52.359 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mkdir test-primary1 2024-09-12T13:21:52.374 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo mount /dev/rbd0 test-primary1 2024-09-12T13:21:52.453 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ whoami 2024-09-12T13:21:52.459 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo chown ubuntu test-primary1 2024-09-12T13:21:52.491 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ SNAP_PIDS+=($!) 2024-09-12T13:21:52.491 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ take_mirror_snapshots cluster1 mirror image-primary1 2024-09-12T13:21:52.491 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:21:52.491 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:21:52.491 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:21:52.491 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:21:52.491 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:21:52.491 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:21:52.491 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:21:52.492 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:21:52.492 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:21:52.492 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ WORKLOAD_PIDS+=($!) 2024-09-12T13:21:52.492 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:21:52.492 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ IMG=image-primary2 2024-09-12T13:21:52.492 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ MNTPT=test-primary2 2024-09-12T13:21:52.492 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ slow_untar_workload test-primary1 2024-09-12T13:21:52.492 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ create_image_and_enable_mirror cluster1 mirror image-primary2 snapshot 10G 2024-09-12T13:21:52.492 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:21:52.492 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:21:52.493 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:21:52.493 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:21:52.493 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:21:52.493 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:21:52.493 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mode=snapshot 2024-09-12T13:21:52.493 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n snapshot ']' 2024-09-12T13:21:52.493 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:21:52.493 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mountpt=test-primary1 2024-09-12T13:21:52.493 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ create_image cluster1 mirror image-primary2 10G 2024-09-12T13:21:52.493 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ cp linux-5.4.tar.gz test-primary1 2024-09-12T13:21:52.493 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:21:52.493 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:21:52.493 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:21:52.493 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:21:52.494 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:21:52.494 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:21:52.494 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local size=128 2024-09-12T13:21:52.494 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n 10G ']' 2024-09-12T13:21:52.494 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ size=10G 2024-09-12T13:21:52.494 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:21:52.494 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 create --size 10G --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image-primary2 2024-09-12T13:21:52.560 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ is_pool_mirror_mode_image mirror 2024-09-12T13:21:52.560 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:21:52.560 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' image = image ']' 2024-09-12T13:21:52.561 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:21:52.561 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ enable_mirror cluster1 mirror image-primary2 snapshot 2024-09-12T13:21:52.561 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:21:52.561 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:21:52.561 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:21:52.561 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mode=snapshot 2024-09-12T13:21:52.561 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 mirror image enable mirror/image-primary2 snapshot 2024-09-12T13:21:52.776 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local ret=0 2024-09-12T13:21:52.776 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ timeout 5m bash -c 'zcat test-primary1/linux-5.4.tar.gz | pv -L 256K | tar xf - -C test-primary1' 2024-09-12T13:21:53.005 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Mirroring enabled 2024-09-12T13:21:53.018 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 994 2024-09-12T13:21:53.026 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 info mirror/image-primary2 2024-09-12T13:21:53.030 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:21:53.102 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:rbd image 'image-primary2': 2024-09-12T13:21:53.102 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: size 10 GiB in 2560 objects 2024-09-12T13:21:53.102 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: order 22 (4 MiB objects) 2024-09-12T13:21:53.103 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: snapshot_count: 1 2024-09-12T13:21:53.103 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: id: 1f7d2b3c9581 2024-09-12T13:21:53.103 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: block_name_prefix: rbd_data.1f7d2b3c9581 2024-09-12T13:21:53.103 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: format: 2 2024-09-12T13:21:53.103 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: features: layering, exclusive-lock, object-map, fast-diff 2024-09-12T13:21:53.103 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: op_features: 2024-09-12T13:21:53.103 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: flags: 2024-09-12T13:21:53.103 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: create_timestamp: Thu Sep 12 13:21:52 2024 2024-09-12T13:21:53.103 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: access_timestamp: Thu Sep 12 13:21:52 2024 2024-09-12T13:21:53.103 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: modify_timestamp: Thu Sep 12 13:21:52 2024 2024-09-12T13:21:53.103 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring state: enabled 2024-09-12T13:21:53.103 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring mode: snapshot 2024-09-12T13:21:53.103 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring global id: b9571de3-33f9-41ad-aae9-a4841d1fab32 2024-09-12T13:21:53.103 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring primary: true 2024-09-12T13:21:53.108 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \n\b\d ]] 2024-09-12T13:21:53.108 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \k\r\b\d ]] 2024-09-12T13:21:53.108 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo rbd --cluster cluster1 device map -t krbd mirror/image-primary2 2024-09-12T13:21:53.259 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEV=/dev/rbd1 2024-09-12T13:21:53.259 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEVS+=($DEV) 2024-09-12T13:21:53.259 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo mkfs.ext4 /dev/rbd1 2024-09-12T13:21:53.280 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:mke2fs 1.46.5 (30-Dec-2021) 2024-09-12T13:21:53.292 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Discarding device blocks: 0/2621440 done 2024-09-12T13:21:53.292 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Creating filesystem with 2621440 4k blocks and 655360 inodes 2024-09-12T13:21:53.292 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Filesystem UUID: 5f409cd2-d790-4e1b-8581-a6b304767f8b 2024-09-12T13:21:53.293 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Superblock backups stored on blocks: 2024-09-12T13:21:53.293 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 2024-09-12T13:21:53.293 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 2024-09-12T13:21:53.293 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Allocating group tables: 0/80 done 2024-09-12T13:21:53.293 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Writing inode tables: 0/80 done 2024-09-12T13:21:53.298 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Creating journal (16384 blocks): done 2024-09-12T13:21:53.377 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Writing superblocks and filesystem accounting information: 0/80 done 2024-09-12T13:21:53.377 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 2024-09-12T13:21:53.380 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mkdir test-primary2 2024-09-12T13:21:53.383 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo mount /dev/rbd1 test-primary2 2024-09-12T13:21:53.480 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ whoami 2024-09-12T13:21:53.482 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo chown ubuntu test-primary2 2024-09-12T13:21:53.506 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ SNAP_PIDS+=($!) 2024-09-12T13:21:53.506 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ take_mirror_snapshots cluster1 mirror image-primary2 2024-09-12T13:21:53.506 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:21:53.507 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:21:53.507 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:21:53.507 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:21:53.507 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ WORKLOAD_PIDS+=($!) 2024-09-12T13:21:53.507 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:21:53.507 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:21:53.507 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:21:53.507 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:21:53.507 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:21:53.507 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:21:53.507 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ IMG=image-primary3 2024-09-12T13:21:53.507 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ MNTPT=test-primary3 2024-09-12T13:21:53.507 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ create_image_and_enable_mirror cluster1 mirror image-primary3 snapshot 10G 2024-09-12T13:21:53.507 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ slow_untar_workload test-primary2 2024-09-12T13:21:53.508 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:21:53.508 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:21:53.508 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:21:53.508 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:21:53.508 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:21:53.508 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:21:53.508 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mode=snapshot 2024-09-12T13:21:53.508 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n snapshot ']' 2024-09-12T13:21:53.508 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:21:53.509 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ create_image cluster1 mirror image-primary3 10G 2024-09-12T13:21:53.509 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mountpt=test-primary2 2024-09-12T13:21:53.509 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:21:53.509 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:21:53.509 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:21:53.509 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ cp linux-5.4.tar.gz test-primary2 2024-09-12T13:21:53.509 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:21:53.509 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:21:53.509 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:21:53.510 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local size=128 2024-09-12T13:21:53.510 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n 10G ']' 2024-09-12T13:21:53.510 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ size=10G 2024-09-12T13:21:53.510 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:21:53.510 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 create --size 10G --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image-primary3 2024-09-12T13:21:53.603 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ is_pool_mirror_mode_image mirror 2024-09-12T13:21:53.603 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:21:53.603 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' image = image ']' 2024-09-12T13:21:53.603 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:21:53.603 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ enable_mirror cluster1 mirror image-primary3 snapshot 2024-09-12T13:21:53.603 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:21:53.604 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:21:53.604 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:21:53.604 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mode=snapshot 2024-09-12T13:21:53.604 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 mirror image enable mirror/image-primary3 snapshot 2024-09-12T13:21:53.814 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local ret=0 2024-09-12T13:21:53.815 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ timeout 5m bash -c 'zcat test-primary2/linux-5.4.tar.gz | pv -L 256K | tar xf - -C test-primary2' 2024-09-12T13:21:54.003 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Mirroring enabled 2024-09-12T13:21:54.025 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 996 2024-09-12T13:21:54.036 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 info mirror/image-primary3 2024-09-12T13:21:54.042 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:21:54.110 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:rbd image 'image-primary3': 2024-09-12T13:21:54.110 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: size 10 GiB in 2560 objects 2024-09-12T13:21:54.110 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: order 22 (4 MiB objects) 2024-09-12T13:21:54.110 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: snapshot_count: 1 2024-09-12T13:21:54.110 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: id: 1f8bd7b5df56 2024-09-12T13:21:54.110 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: block_name_prefix: rbd_data.1f8bd7b5df56 2024-09-12T13:21:54.110 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: format: 2 2024-09-12T13:21:54.111 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: features: layering, exclusive-lock, object-map, fast-diff 2024-09-12T13:21:54.111 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: op_features: 2024-09-12T13:21:54.111 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: flags: 2024-09-12T13:21:54.111 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: create_timestamp: Thu Sep 12 13:21:53 2024 2024-09-12T13:21:54.111 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: access_timestamp: Thu Sep 12 13:21:53 2024 2024-09-12T13:21:54.111 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: modify_timestamp: Thu Sep 12 13:21:53 2024 2024-09-12T13:21:54.111 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring state: enabled 2024-09-12T13:21:54.111 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring mode: snapshot 2024-09-12T13:21:54.111 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring global id: 1d6661c9-3d05-4fd7-9ee6-aad6901ddd33 2024-09-12T13:21:54.111 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring primary: true 2024-09-12T13:21:54.115 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \n\b\d ]] 2024-09-12T13:21:54.116 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \k\r\b\d ]] 2024-09-12T13:21:54.116 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo rbd --cluster cluster1 device map -t krbd mirror/image-primary3 2024-09-12T13:21:54.279 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEV=/dev/rbd2 2024-09-12T13:21:54.279 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEVS+=($DEV) 2024-09-12T13:21:54.279 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo mkfs.ext4 /dev/rbd2 2024-09-12T13:21:54.300 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:mke2fs 1.46.5 (30-Dec-2021) 2024-09-12T13:21:54.311 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Discarding device blocks: 0/2621440 done 2024-09-12T13:21:54.311 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Creating filesystem with 2621440 4k blocks and 655360 inodes 2024-09-12T13:21:54.311 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Filesystem UUID: f9b821dc-8df8-4a45-aa53-3e44003842c5 2024-09-12T13:21:54.311 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Superblock backups stored on blocks: 2024-09-12T13:21:54.311 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 2024-09-12T13:21:54.311 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 2024-09-12T13:21:54.311 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Allocating group tables: 0/80 done 2024-09-12T13:21:54.312 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Writing inode tables: 0/80 done 2024-09-12T13:21:54.316 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Creating journal (16384 blocks): done 2024-09-12T13:21:54.403 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Writing superblocks and filesystem accounting information: 0/80 done 2024-09-12T13:21:54.403 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 2024-09-12T13:21:54.405 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mkdir test-primary3 2024-09-12T13:21:54.408 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo mount /dev/rbd2 test-primary3 2024-09-12T13:21:54.494 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ whoami 2024-09-12T13:21:54.496 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo chown ubuntu test-primary3 2024-09-12T13:21:54.521 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ SNAP_PIDS+=($!) 2024-09-12T13:21:54.521 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ take_mirror_snapshots cluster1 mirror image-primary3 2024-09-12T13:21:54.521 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:21:54.522 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:21:54.522 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:21:54.522 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:21:54.522 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:21:54.522 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:21:54.522 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:21:54.522 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:21:54.522 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:21:54.522 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ WORKLOAD_PIDS+=($!) 2024-09-12T13:21:54.522 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:21:54.522 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ slow_untar_workload test-primary3 2024-09-12T13:21:54.523 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ IMG=image-primary4 2024-09-12T13:21:54.523 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ MNTPT=test-primary4 2024-09-12T13:21:54.523 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ create_image_and_enable_mirror cluster1 mirror image-primary4 snapshot 10G 2024-09-12T13:21:54.523 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mountpt=test-primary3 2024-09-12T13:21:54.523 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:21:54.523 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:21:54.523 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:21:54.523 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:21:54.523 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:21:54.523 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:21:54.524 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mode=snapshot 2024-09-12T13:21:54.524 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ cp linux-5.4.tar.gz test-primary3 2024-09-12T13:21:54.524 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n snapshot ']' 2024-09-12T13:21:54.524 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:21:54.524 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ create_image cluster1 mirror image-primary4 10G 2024-09-12T13:21:54.524 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:21:54.524 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:21:54.524 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:21:54.524 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:21:54.524 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:21:54.524 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:21:54.524 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local size=128 2024-09-12T13:21:54.524 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n 10G ']' 2024-09-12T13:21:54.524 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ size=10G 2024-09-12T13:21:54.525 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:21:54.525 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 create --size 10G --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image-primary4 2024-09-12T13:21:54.613 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ is_pool_mirror_mode_image mirror 2024-09-12T13:21:54.613 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:21:54.613 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' image = image ']' 2024-09-12T13:21:54.613 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:21:54.613 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ enable_mirror cluster1 mirror image-primary4 snapshot 2024-09-12T13:21:54.613 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:21:54.613 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:21:54.613 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:21:54.614 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mode=snapshot 2024-09-12T13:21:54.614 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 mirror image enable mirror/image-primary4 snapshot 2024-09-12T13:21:54.820 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local ret=0 2024-09-12T13:21:54.820 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ timeout 5m bash -c 'zcat test-primary3/linux-5.4.tar.gz | pv -L 256K | tar xf - -C test-primary3' 2024-09-12T13:21:54.982 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Mirroring enabled 2024-09-12T13:21:54.987 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 998 2024-09-12T13:21:54.996 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 info mirror/image-primary4 2024-09-12T13:21:54.996 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:21:55.056 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:rbd image 'image-primary4': 2024-09-12T13:21:55.056 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: size 10 GiB in 2560 objects 2024-09-12T13:21:55.056 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: order 22 (4 MiB objects) 2024-09-12T13:21:55.056 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: snapshot_count: 1 2024-09-12T13:21:55.056 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: id: 1f983edbbc0b 2024-09-12T13:21:55.056 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: block_name_prefix: rbd_data.1f983edbbc0b 2024-09-12T13:21:55.056 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: format: 2 2024-09-12T13:21:55.057 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: features: layering, exclusive-lock, object-map, fast-diff 2024-09-12T13:21:55.057 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: op_features: 2024-09-12T13:21:55.057 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: flags: 2024-09-12T13:21:55.057 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: create_timestamp: Thu Sep 12 13:21:54 2024 2024-09-12T13:21:55.057 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: access_timestamp: Thu Sep 12 13:21:54 2024 2024-09-12T13:21:55.057 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: modify_timestamp: Thu Sep 12 13:21:54 2024 2024-09-12T13:21:55.057 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring state: enabled 2024-09-12T13:21:55.057 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring mode: snapshot 2024-09-12T13:21:55.057 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring global id: ce400cdd-ba29-4924-9381-6e78dcb9b581 2024-09-12T13:21:55.057 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring primary: true 2024-09-12T13:21:55.061 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \n\b\d ]] 2024-09-12T13:21:55.061 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \k\r\b\d ]] 2024-09-12T13:21:55.062 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo rbd --cluster cluster1 device map -t krbd mirror/image-primary4 2024-09-12T13:21:55.176 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEV=/dev/rbd3 2024-09-12T13:21:55.176 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEVS+=($DEV) 2024-09-12T13:21:55.176 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo mkfs.ext4 /dev/rbd3 2024-09-12T13:21:55.196 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:mke2fs 1.46.5 (30-Dec-2021) 2024-09-12T13:21:55.208 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Discarding device blocks: 0/2621440 done 2024-09-12T13:21:55.208 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Creating filesystem with 2621440 4k blocks and 655360 inodes 2024-09-12T13:21:55.208 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Filesystem UUID: f6d68da7-fb4a-4a22-a8b7-fe4e6e30298c 2024-09-12T13:21:55.208 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Superblock backups stored on blocks: 2024-09-12T13:21:55.208 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 2024-09-12T13:21:55.208 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 2024-09-12T13:21:55.208 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Allocating group tables: 0/80 done 2024-09-12T13:21:55.208 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Writing inode tables: 0/80 done 2024-09-12T13:21:55.213 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Creating journal (16384 blocks): done 2024-09-12T13:21:55.267 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Writing superblocks and filesystem accounting information: 0/80 done 2024-09-12T13:21:55.267 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 2024-09-12T13:21:55.269 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mkdir test-primary4 2024-09-12T13:21:55.272 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo mount /dev/rbd3 test-primary4 2024-09-12T13:21:55.335 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ whoami 2024-09-12T13:21:55.337 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo chown ubuntu test-primary4 2024-09-12T13:21:55.362 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ SNAP_PIDS+=($!) 2024-09-12T13:21:55.362 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ take_mirror_snapshots cluster1 mirror image-primary4 2024-09-12T13:21:55.362 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:21:55.362 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:21:55.362 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:21:55.362 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ WORKLOAD_PIDS+=($!) 2024-09-12T13:21:55.362 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:21:55.363 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:21:55.363 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:21:55.363 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:21:55.363 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:21:55.363 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ IMG=image-primary5 2024-09-12T13:21:55.363 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:21:55.363 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ MNTPT=test-primary5 2024-09-12T13:21:55.363 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ create_image_and_enable_mirror cluster1 mirror image-primary5 snapshot 10G 2024-09-12T13:21:55.363 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ slow_untar_workload test-primary4 2024-09-12T13:21:55.363 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:21:55.363 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:21:55.363 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:21:55.363 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:21:55.365 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:21:55.365 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:21:55.366 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:21:55.366 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mode=snapshot 2024-09-12T13:21:55.366 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n snapshot ']' 2024-09-12T13:21:55.366 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:21:55.366 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ create_image cluster1 mirror image-primary5 10G 2024-09-12T13:21:55.366 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mountpt=test-primary4 2024-09-12T13:21:55.366 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:21:55.366 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:21:55.366 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ cp linux-5.4.tar.gz test-primary4 2024-09-12T13:21:55.366 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:21:55.366 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:21:55.366 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:21:55.367 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:21:55.367 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local size=128 2024-09-12T13:21:55.367 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n 10G ']' 2024-09-12T13:21:55.367 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ size=10G 2024-09-12T13:21:55.367 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:21:55.367 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 create --size 10G --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image-primary5 2024-09-12T13:21:55.439 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ is_pool_mirror_mode_image mirror 2024-09-12T13:21:55.439 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:21:55.439 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' image = image ']' 2024-09-12T13:21:55.440 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:21:55.440 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ enable_mirror cluster1 mirror image-primary5 snapshot 2024-09-12T13:21:55.440 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:21:55.440 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:21:55.440 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:21:55.440 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mode=snapshot 2024-09-12T13:21:55.440 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 mirror image enable mirror/image-primary5 snapshot 2024-09-12T13:21:55.651 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local ret=0 2024-09-12T13:21:55.652 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ timeout 5m bash -c 'zcat test-primary4/linux-5.4.tar.gz | pv -L 256K | tar xf - -C test-primary4' 2024-09-12T13:21:55.992 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Mirroring enabled 2024-09-12T13:21:55.998 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1000 2024-09-12T13:21:56.007 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 info mirror/image-primary5 2024-09-12T13:21:56.008 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:21:56.033 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:21:56.033 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:21:56.033 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:21:56.033 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:21:56.033 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:21:56.033 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:21:56.086 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:rbd image 'image-primary5': 2024-09-12T13:21:56.086 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: size 10 GiB in 2560 objects 2024-09-12T13:21:56.086 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: order 22 (4 MiB objects) 2024-09-12T13:21:56.086 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: snapshot_count: 1 2024-09-12T13:21:56.086 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: id: 1fa542db0fc9 2024-09-12T13:21:56.086 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: block_name_prefix: rbd_data.1fa542db0fc9 2024-09-12T13:21:56.086 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: format: 2 2024-09-12T13:21:56.087 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: features: layering, exclusive-lock, object-map, fast-diff 2024-09-12T13:21:56.087 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: op_features: 2024-09-12T13:21:56.087 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: flags: 2024-09-12T13:21:56.087 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: create_timestamp: Thu Sep 12 13:21:55 2024 2024-09-12T13:21:56.087 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: access_timestamp: Thu Sep 12 13:21:55 2024 2024-09-12T13:21:56.087 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: modify_timestamp: Thu Sep 12 13:21:55 2024 2024-09-12T13:21:56.087 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring state: enabled 2024-09-12T13:21:56.087 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring mode: snapshot 2024-09-12T13:21:56.087 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring global id: 1d3a6d29-f46e-4dfe-ba1d-2e1bda9e7bc4 2024-09-12T13:21:56.087 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring primary: true 2024-09-12T13:21:56.091 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \n\b\d ]] 2024-09-12T13:21:56.091 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \k\r\b\d ]] 2024-09-12T13:21:56.092 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo rbd --cluster cluster1 device map -t krbd mirror/image-primary5 2024-09-12T13:21:56.230 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEV=/dev/rbd4 2024-09-12T13:21:56.230 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEVS+=($DEV) 2024-09-12T13:21:56.230 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo mkfs.ext4 /dev/rbd4 2024-09-12T13:21:56.250 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:mke2fs 1.46.5 (30-Dec-2021) 2024-09-12T13:21:56.262 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Discarding device blocks: 0/2621440 done 2024-09-12T13:21:56.262 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Creating filesystem with 2621440 4k blocks and 655360 inodes 2024-09-12T13:21:56.262 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Filesystem UUID: 65ae1fd0-ad58-4214-9923-dbc2d001087d 2024-09-12T13:21:56.262 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Superblock backups stored on blocks: 2024-09-12T13:21:56.262 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 2024-09-12T13:21:56.263 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 2024-09-12T13:21:56.263 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Allocating group tables: 0/80 done 2024-09-12T13:21:56.263 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Writing inode tables: 0/80 done 2024-09-12T13:21:56.267 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Creating journal (16384 blocks): done 2024-09-12T13:21:56.321 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Writing superblocks and filesystem accounting information: 0/80 done 2024-09-12T13:21:56.321 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 2024-09-12T13:21:56.324 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mkdir test-primary5 2024-09-12T13:21:56.326 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo mount /dev/rbd4 test-primary5 2024-09-12T13:21:56.381 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ whoami 2024-09-12T13:21:56.383 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo chown ubuntu test-primary5 2024-09-12T13:21:56.408 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ SNAP_PIDS+=($!) 2024-09-12T13:21:56.408 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ take_mirror_snapshots cluster1 mirror image-primary5 2024-09-12T13:21:56.408 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:21:56.408 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:21:56.408 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:21:56.408 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:21:56.409 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:21:56.409 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:21:56.409 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:21:56.409 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:21:56.409 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:21:56.409 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ WORKLOAD_PIDS+=($!) 2024-09-12T13:21:56.409 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:21:56.409 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ slow_untar_workload test-primary5 2024-09-12T13:21:56.409 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ IMG=image-primary6 2024-09-12T13:21:56.409 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ MNTPT=test-primary6 2024-09-12T13:21:56.409 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ create_image_and_enable_mirror cluster1 mirror image-primary6 snapshot 10G 2024-09-12T13:21:56.409 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mountpt=test-primary5 2024-09-12T13:21:56.409 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:21:56.410 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ cp linux-5.4.tar.gz test-primary5 2024-09-12T13:21:56.410 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:21:56.410 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:21:56.410 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:21:56.410 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:21:56.410 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:21:56.410 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mode=snapshot 2024-09-12T13:21:56.410 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n snapshot ']' 2024-09-12T13:21:56.410 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:21:56.410 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ create_image cluster1 mirror image-primary6 10G 2024-09-12T13:21:56.410 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:21:56.410 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:21:56.410 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:21:56.411 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:21:56.411 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:21:56.411 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:21:56.411 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local size=128 2024-09-12T13:21:56.411 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n 10G ']' 2024-09-12T13:21:56.411 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ size=10G 2024-09-12T13:21:56.411 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:21:56.411 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 create --size 10G --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image-primary6 2024-09-12T13:21:56.481 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ is_pool_mirror_mode_image mirror 2024-09-12T13:21:56.481 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:21:56.481 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' image = image ']' 2024-09-12T13:21:56.481 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:21:56.481 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ enable_mirror cluster1 mirror image-primary6 snapshot 2024-09-12T13:21:56.481 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:21:56.482 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:21:56.482 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:21:56.482 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mode=snapshot 2024-09-12T13:21:56.482 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 mirror image enable mirror/image-primary6 snapshot 2024-09-12T13:21:56.698 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local ret=0 2024-09-12T13:21:56.699 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ timeout 5m bash -c 'zcat test-primary5/linux-5.4.tar.gz | pv -L 256K | tar xf - -C test-primary5' 2024-09-12T13:21:57.006 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1003 2024-09-12T13:21:57.010 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Mirroring enabled 2024-09-12T13:21:57.015 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:21:57.021 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1002 2024-09-12T13:21:57.028 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 info mirror/image-primary6 2024-09-12T13:21:57.032 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:21:57.045 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:21:57.045 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:21:57.045 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:21:57.045 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:21:57.045 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:21:57.045 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:21:57.097 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:rbd image 'image-primary6': 2024-09-12T13:21:57.097 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: size 10 GiB in 2560 objects 2024-09-12T13:21:57.097 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: order 22 (4 MiB objects) 2024-09-12T13:21:57.097 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: snapshot_count: 1 2024-09-12T13:21:57.098 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: id: 1fb6e562f985 2024-09-12T13:21:57.098 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: block_name_prefix: rbd_data.1fb6e562f985 2024-09-12T13:21:57.098 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: format: 2 2024-09-12T13:21:57.098 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: features: layering, exclusive-lock, object-map, fast-diff 2024-09-12T13:21:57.098 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: op_features: 2024-09-12T13:21:57.098 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: flags: 2024-09-12T13:21:57.099 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: create_timestamp: Thu Sep 12 13:21:56 2024 2024-09-12T13:21:57.099 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: access_timestamp: Thu Sep 12 13:21:56 2024 2024-09-12T13:21:57.099 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: modify_timestamp: Thu Sep 12 13:21:56 2024 2024-09-12T13:21:57.099 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring state: enabled 2024-09-12T13:21:57.100 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring mode: snapshot 2024-09-12T13:21:57.100 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring global id: b580eb06-ffeb-44e0-8354-b02afae3f6a4 2024-09-12T13:21:57.100 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring primary: true 2024-09-12T13:21:57.101 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \n\b\d ]] 2024-09-12T13:21:57.101 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \k\r\b\d ]] 2024-09-12T13:21:57.102 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo rbd --cluster cluster1 device map -t krbd mirror/image-primary6 2024-09-12T13:21:57.218 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEV=/dev/rbd5 2024-09-12T13:21:57.218 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEVS+=($DEV) 2024-09-12T13:21:57.218 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo mkfs.ext4 /dev/rbd5 2024-09-12T13:21:57.239 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:mke2fs 1.46.5 (30-Dec-2021) 2024-09-12T13:21:57.251 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Discarding device blocks: 0/2621440 done 2024-09-12T13:21:57.251 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Creating filesystem with 2621440 4k blocks and 655360 inodes 2024-09-12T13:21:57.252 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Filesystem UUID: b502b5c9-f7d2-4e0f-9078-084df114755f 2024-09-12T13:21:57.252 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Superblock backups stored on blocks: 2024-09-12T13:21:57.252 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 2024-09-12T13:21:57.252 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 2024-09-12T13:21:57.252 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Allocating group tables: 0/80 done 2024-09-12T13:21:57.252 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Writing inode tables: 0/80 done 2024-09-12T13:21:57.257 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Creating journal (16384 blocks): done 2024-09-12T13:21:57.305 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Writing superblocks and filesystem accounting information: 0/80 done 2024-09-12T13:21:57.305 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 2024-09-12T13:21:57.308 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mkdir test-primary6 2024-09-12T13:21:57.310 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo mount /dev/rbd5 test-primary6 2024-09-12T13:21:57.365 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ whoami 2024-09-12T13:21:57.367 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo chown ubuntu test-primary6 2024-09-12T13:21:57.392 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ SNAP_PIDS+=($!) 2024-09-12T13:21:57.392 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ take_mirror_snapshots cluster1 mirror image-primary6 2024-09-12T13:21:57.392 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:21:57.392 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:21:57.392 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:21:57.392 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:21:57.392 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:21:57.392 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:21:57.392 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:21:57.392 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ WORKLOAD_PIDS+=($!) 2024-09-12T13:21:57.392 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:21:57.393 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:21:57.393 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:21:57.393 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ IMG=image-primary7 2024-09-12T13:21:57.393 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ MNTPT=test-primary7 2024-09-12T13:21:57.393 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ create_image_and_enable_mirror cluster1 mirror image-primary7 snapshot 10G 2024-09-12T13:21:57.393 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:21:57.393 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:21:57.393 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:21:57.393 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:21:57.393 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:21:57.393 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:21:57.393 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mode=snapshot 2024-09-12T13:21:57.393 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n snapshot ']' 2024-09-12T13:21:57.394 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:21:57.394 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ create_image cluster1 mirror image-primary7 10G 2024-09-12T13:21:57.394 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ slow_untar_workload test-primary6 2024-09-12T13:21:57.394 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:21:57.394 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:21:57.394 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:21:57.394 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:21:57.394 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:21:57.394 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:21:57.394 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local size=128 2024-09-12T13:21:57.394 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n 10G ']' 2024-09-12T13:21:57.394 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ size=10G 2024-09-12T13:21:57.394 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:21:57.394 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 create --size 10G --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image-primary7 2024-09-12T13:21:57.395 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mountpt=test-primary6 2024-09-12T13:21:57.395 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ cp linux-5.4.tar.gz test-primary6 2024-09-12T13:21:57.460 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ is_pool_mirror_mode_image mirror 2024-09-12T13:21:57.461 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:21:57.461 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' image = image ']' 2024-09-12T13:21:57.461 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:21:57.461 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ enable_mirror cluster1 mirror image-primary7 snapshot 2024-09-12T13:21:57.461 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:21:57.461 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:21:57.461 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:21:57.461 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mode=snapshot 2024-09-12T13:21:57.461 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 mirror image enable mirror/image-primary7 snapshot 2024-09-12T13:21:57.687 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local ret=0 2024-09-12T13:21:57.687 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ timeout 5m bash -c 'zcat test-primary6/linux-5.4.tar.gz | pv -L 256K | tar xf - -C test-primary6' 2024-09-12T13:21:57.956 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1005 2024-09-12T13:21:57.958 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Mirroring enabled 2024-09-12T13:21:57.964 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1006 2024-09-12T13:21:57.968 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:21:57.979 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 info mirror/image-primary7 2024-09-12T13:21:57.981 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:21:57.999 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:21:57.999 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:21:57.999 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:21:57.999 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:21:57.999 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:21:57.999 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:21:58.041 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:rbd image 'image-primary7': 2024-09-12T13:21:58.041 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: size 10 GiB in 2560 objects 2024-09-12T13:21:58.041 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: order 22 (4 MiB objects) 2024-09-12T13:21:58.042 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: snapshot_count: 1 2024-09-12T13:21:58.042 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: id: 1fc7363450a8 2024-09-12T13:21:58.042 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: block_name_prefix: rbd_data.1fc7363450a8 2024-09-12T13:21:58.042 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: format: 2 2024-09-12T13:21:58.042 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: features: layering, exclusive-lock, object-map, fast-diff 2024-09-12T13:21:58.042 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: op_features: 2024-09-12T13:21:58.042 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: flags: 2024-09-12T13:21:58.042 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: create_timestamp: Thu Sep 12 13:21:57 2024 2024-09-12T13:21:58.042 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: access_timestamp: Thu Sep 12 13:21:57 2024 2024-09-12T13:21:58.042 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: modify_timestamp: Thu Sep 12 13:21:57 2024 2024-09-12T13:21:58.042 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring state: enabled 2024-09-12T13:21:58.042 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring mode: snapshot 2024-09-12T13:21:58.042 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring global id: 43088135-2ee1-48a6-8f76-ae96116f16b1 2024-09-12T13:21:58.042 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring primary: true 2024-09-12T13:21:58.047 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \n\b\d ]] 2024-09-12T13:21:58.048 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \k\r\b\d ]] 2024-09-12T13:21:58.048 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo rbd --cluster cluster1 device map -t krbd mirror/image-primary7 2024-09-12T13:21:58.168 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEV=/dev/rbd6 2024-09-12T13:21:58.168 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEVS+=($DEV) 2024-09-12T13:21:58.168 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo mkfs.ext4 /dev/rbd6 2024-09-12T13:21:58.189 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:mke2fs 1.46.5 (30-Dec-2021) 2024-09-12T13:21:58.200 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Discarding device blocks: 0/2621440 done 2024-09-12T13:21:58.200 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Creating filesystem with 2621440 4k blocks and 655360 inodes 2024-09-12T13:21:58.200 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Filesystem UUID: 0a974dbd-cd10-499a-9edb-c2a402058648 2024-09-12T13:21:58.200 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Superblock backups stored on blocks: 2024-09-12T13:21:58.200 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 2024-09-12T13:21:58.200 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 2024-09-12T13:21:58.200 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Allocating group tables: 0/80 done 2024-09-12T13:21:58.201 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Writing inode tables: 0/80 done 2024-09-12T13:21:58.205 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Creating journal (16384 blocks): done 2024-09-12T13:21:58.250 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Writing superblocks and filesystem accounting information: 0/80 done 2024-09-12T13:21:58.250 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 2024-09-12T13:21:58.253 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mkdir test-primary7 2024-09-12T13:21:58.255 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo mount /dev/rbd6 test-primary7 2024-09-12T13:21:58.309 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ whoami 2024-09-12T13:21:58.311 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo chown ubuntu test-primary7 2024-09-12T13:21:58.336 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ SNAP_PIDS+=($!) 2024-09-12T13:21:58.336 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ take_mirror_snapshots cluster1 mirror image-primary7 2024-09-12T13:21:58.336 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:21:58.336 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:21:58.336 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:21:58.336 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:21:58.337 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:21:58.337 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:21:58.337 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:21:58.337 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:21:58.337 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:21:58.337 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ WORKLOAD_PIDS+=($!) 2024-09-12T13:21:58.337 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:21:58.337 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ slow_untar_workload test-primary7 2024-09-12T13:21:58.337 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ IMG=image-primary8 2024-09-12T13:21:58.337 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ MNTPT=test-primary8 2024-09-12T13:21:58.337 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ create_image_and_enable_mirror cluster1 mirror image-primary8 snapshot 10G 2024-09-12T13:21:58.337 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mountpt=test-primary7 2024-09-12T13:21:58.337 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ cp linux-5.4.tar.gz test-primary7 2024-09-12T13:21:58.338 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:21:58.338 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:21:58.338 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:21:58.338 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:21:58.338 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:21:58.338 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:21:58.338 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mode=snapshot 2024-09-12T13:21:58.338 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n snapshot ']' 2024-09-12T13:21:58.338 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:21:58.338 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ create_image cluster1 mirror image-primary8 10G 2024-09-12T13:21:58.338 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:21:58.338 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:21:58.338 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:21:58.338 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:21:58.339 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:21:58.339 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:21:58.339 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local size=128 2024-09-12T13:21:58.339 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n 10G ']' 2024-09-12T13:21:58.339 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ size=10G 2024-09-12T13:21:58.339 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:21:58.339 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 create --size 10G --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image-primary8 2024-09-12T13:21:58.405 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ is_pool_mirror_mode_image mirror 2024-09-12T13:21:58.405 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:21:58.405 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' image = image ']' 2024-09-12T13:21:58.405 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:21:58.405 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ enable_mirror cluster1 mirror image-primary8 snapshot 2024-09-12T13:21:58.405 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:21:58.405 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:21:58.405 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:21:58.406 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mode=snapshot 2024-09-12T13:21:58.406 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 mirror image enable mirror/image-primary8 snapshot 2024-09-12T13:21:58.630 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local ret=0 2024-09-12T13:21:58.630 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ timeout 5m bash -c 'zcat test-primary7/linux-5.4.tar.gz | pv -L 256K | tar xf - -C test-primary7' 2024-09-12T13:21:58.945 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Mirroring enabled 2024-09-12T13:21:58.953 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1009 2024-09-12T13:21:58.954 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1008 2024-09-12T13:21:58.970 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:21:58.972 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 info mirror/image-primary8 2024-09-12T13:21:58.980 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:21:59.012 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:21:59.012 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:21:59.012 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:21:59.012 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:21:59.012 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:21:59.012 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:21:59.068 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:rbd image 'image-primary8': 2024-09-12T13:21:59.068 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: size 10 GiB in 2560 objects 2024-09-12T13:21:59.068 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: order 22 (4 MiB objects) 2024-09-12T13:21:59.068 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: snapshot_count: 1 2024-09-12T13:21:59.069 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: id: 1fd93ba82474 2024-09-12T13:21:59.069 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: block_name_prefix: rbd_data.1fd93ba82474 2024-09-12T13:21:59.069 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: format: 2 2024-09-12T13:21:59.069 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: features: layering, exclusive-lock, object-map, fast-diff 2024-09-12T13:21:59.069 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: op_features: 2024-09-12T13:21:59.069 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: flags: 2024-09-12T13:21:59.069 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: create_timestamp: Thu Sep 12 13:21:58 2024 2024-09-12T13:21:59.069 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: access_timestamp: Thu Sep 12 13:21:58 2024 2024-09-12T13:21:59.069 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: modify_timestamp: Thu Sep 12 13:21:58 2024 2024-09-12T13:21:59.069 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring state: enabled 2024-09-12T13:21:59.069 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring mode: snapshot 2024-09-12T13:21:59.069 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring global id: e5469b72-dbc8-4a77-9c96-ab995f3cc6f1 2024-09-12T13:21:59.069 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring primary: true 2024-09-12T13:21:59.074 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \n\b\d ]] 2024-09-12T13:21:59.075 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \k\r\b\d ]] 2024-09-12T13:21:59.076 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo rbd --cluster cluster1 device map -t krbd mirror/image-primary8 2024-09-12T13:21:59.204 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEV=/dev/rbd7 2024-09-12T13:21:59.204 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEVS+=($DEV) 2024-09-12T13:21:59.204 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo mkfs.ext4 /dev/rbd7 2024-09-12T13:21:59.224 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:mke2fs 1.46.5 (30-Dec-2021) 2024-09-12T13:21:59.238 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Discarding device blocks: 0/2621440 done 2024-09-12T13:21:59.238 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Creating filesystem with 2621440 4k blocks and 655360 inodes 2024-09-12T13:21:59.238 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Filesystem UUID: 451881fb-bd8f-46d7-944c-2ac13aed5090 2024-09-12T13:21:59.238 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Superblock backups stored on blocks: 2024-09-12T13:21:59.238 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 2024-09-12T13:21:59.238 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 2024-09-12T13:21:59.238 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Allocating group tables: 0/80 done 2024-09-12T13:21:59.238 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Writing inode tables: 0/80 done 2024-09-12T13:21:59.243 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Creating journal (16384 blocks): done 2024-09-12T13:21:59.288 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Writing superblocks and filesystem accounting information: 0/80 done 2024-09-12T13:21:59.288 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 2024-09-12T13:21:59.291 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mkdir test-primary8 2024-09-12T13:21:59.293 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo mount /dev/rbd7 test-primary8 2024-09-12T13:21:59.351 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ whoami 2024-09-12T13:21:59.352 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo chown ubuntu test-primary8 2024-09-12T13:21:59.378 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ SNAP_PIDS+=($!) 2024-09-12T13:21:59.378 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ take_mirror_snapshots cluster1 mirror image-primary8 2024-09-12T13:21:59.378 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:21:59.378 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:21:59.378 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:21:59.378 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:21:59.378 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:21:59.378 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:21:59.378 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:21:59.378 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:21:59.378 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:21:59.379 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ WORKLOAD_PIDS+=($!) 2024-09-12T13:21:59.379 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:21:59.379 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ IMG=image-primary9 2024-09-12T13:21:59.379 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ MNTPT=test-primary9 2024-09-12T13:21:59.379 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ slow_untar_workload test-primary8 2024-09-12T13:21:59.379 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ create_image_and_enable_mirror cluster1 mirror image-primary9 snapshot 10G 2024-09-12T13:21:59.379 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:21:59.379 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:21:59.379 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:21:59.379 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:21:59.379 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:21:59.379 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:21:59.379 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mode=snapshot 2024-09-12T13:21:59.380 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n snapshot ']' 2024-09-12T13:21:59.380 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:21:59.380 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ create_image cluster1 mirror image-primary9 10G 2024-09-12T13:21:59.380 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mountpt=test-primary8 2024-09-12T13:21:59.380 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:21:59.380 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:21:59.380 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:21:59.380 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:21:59.380 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:21:59.380 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ cp linux-5.4.tar.gz test-primary8 2024-09-12T13:21:59.380 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:21:59.380 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local size=128 2024-09-12T13:21:59.380 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n 10G ']' 2024-09-12T13:21:59.380 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ size=10G 2024-09-12T13:21:59.381 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:21:59.381 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 create --size 10G --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image-primary9 2024-09-12T13:21:59.446 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ is_pool_mirror_mode_image mirror 2024-09-12T13:21:59.447 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:21:59.447 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' image = image ']' 2024-09-12T13:21:59.447 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:21:59.447 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ enable_mirror cluster1 mirror image-primary9 snapshot 2024-09-12T13:21:59.447 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:21:59.447 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:21:59.447 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:21:59.447 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mode=snapshot 2024-09-12T13:21:59.447 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 mirror image enable mirror/image-primary9 snapshot 2024-09-12T13:21:59.682 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local ret=0 2024-09-12T13:21:59.683 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ timeout 5m bash -c 'zcat test-primary8/linux-5.4.tar.gz | pv -L 256K | tar xf - -C test-primary8' 2024-09-12T13:21:59.953 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Mirroring enabled 2024-09-12T13:21:59.955 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1012 2024-09-12T13:21:59.960 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1011 2024-09-12T13:21:59.972 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:21:59.978 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 info mirror/image-primary9 2024-09-12T13:21:59.978 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:22:00.018 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:22:00.018 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:22:00.018 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:22:00.018 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:22:00.018 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:22:00.018 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:22:00.035 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:22:00.035 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:22:00.035 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:22:00.035 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:22:00.035 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:22:00.035 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:22:00.036 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:rbd image 'image-primary9': 2024-09-12T13:22:00.036 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: size 10 GiB in 2560 objects 2024-09-12T13:22:00.036 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: order 22 (4 MiB objects) 2024-09-12T13:22:00.036 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: snapshot_count: 1 2024-09-12T13:22:00.036 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: id: 1fea63106e98 2024-09-12T13:22:00.037 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: block_name_prefix: rbd_data.1fea63106e98 2024-09-12T13:22:00.037 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: format: 2 2024-09-12T13:22:00.037 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: features: layering, exclusive-lock, object-map, fast-diff 2024-09-12T13:22:00.037 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: op_features: 2024-09-12T13:22:00.037 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: flags: 2024-09-12T13:22:00.037 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: create_timestamp: Thu Sep 12 13:21:59 2024 2024-09-12T13:22:00.037 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: access_timestamp: Thu Sep 12 13:21:59 2024 2024-09-12T13:22:00.037 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: modify_timestamp: Thu Sep 12 13:21:59 2024 2024-09-12T13:22:00.037 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring state: enabled 2024-09-12T13:22:00.037 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring mode: snapshot 2024-09-12T13:22:00.037 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring global id: cabafe0d-591b-466b-8d57-50e60dd6fa48 2024-09-12T13:22:00.037 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring primary: true 2024-09-12T13:22:00.047 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \n\b\d ]] 2024-09-12T13:22:00.047 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \k\r\b\d ]] 2024-09-12T13:22:00.048 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo rbd --cluster cluster1 device map -t krbd mirror/image-primary9 2024-09-12T13:22:00.179 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEV=/dev/rbd8 2024-09-12T13:22:00.180 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEVS+=($DEV) 2024-09-12T13:22:00.180 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo mkfs.ext4 /dev/rbd8 2024-09-12T13:22:00.200 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:mke2fs 1.46.5 (30-Dec-2021) 2024-09-12T13:22:00.215 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Discarding device blocks: 0/2621440 done 2024-09-12T13:22:00.215 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Creating filesystem with 2621440 4k blocks and 655360 inodes 2024-09-12T13:22:00.215 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Filesystem UUID: dd393fef-344a-4844-8fbb-116025358da7 2024-09-12T13:22:00.215 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Superblock backups stored on blocks: 2024-09-12T13:22:00.215 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 2024-09-12T13:22:00.215 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 2024-09-12T13:22:00.215 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Allocating group tables: 0/80 done 2024-09-12T13:22:00.215 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Writing inode tables: 0/80 done 2024-09-12T13:22:00.220 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Creating journal (16384 blocks): done 2024-09-12T13:22:00.263 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Writing superblocks and filesystem accounting information: 0/80 done 2024-09-12T13:22:00.263 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 2024-09-12T13:22:00.266 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mkdir test-primary9 2024-09-12T13:22:00.268 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo mount /dev/rbd8 test-primary9 2024-09-12T13:22:00.323 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ whoami 2024-09-12T13:22:00.324 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo chown ubuntu test-primary9 2024-09-12T13:22:00.350 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ SNAP_PIDS+=($!) 2024-09-12T13:22:00.350 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ take_mirror_snapshots cluster1 mirror image-primary9 2024-09-12T13:22:00.350 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:22:00.350 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:22:00.350 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:22:00.351 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:22:00.351 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:22:00.351 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:22:00.351 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:22:00.351 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:22:00.351 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ WORKLOAD_PIDS+=($!) 2024-09-12T13:22:00.351 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:22:00.351 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:22:00.351 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ IMG=image-primary10 2024-09-12T13:22:00.351 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ MNTPT=test-primary10 2024-09-12T13:22:00.351 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ create_image_and_enable_mirror cluster1 mirror image-primary10 snapshot 10G 2024-09-12T13:22:00.351 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:22:00.351 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:22:00.351 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:22:00.352 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:22:00.352 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:22:00.352 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:22:00.352 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mode=snapshot 2024-09-12T13:22:00.352 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n snapshot ']' 2024-09-12T13:22:00.352 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:22:00.352 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ create_image cluster1 mirror image-primary10 10G 2024-09-12T13:22:00.352 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:22:00.352 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:22:00.352 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ slow_untar_workload test-primary9 2024-09-12T13:22:00.352 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:22:00.352 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:22:00.352 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:22:00.352 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:22:00.353 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local size=128 2024-09-12T13:22:00.353 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n 10G ']' 2024-09-12T13:22:00.353 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ size=10G 2024-09-12T13:22:00.353 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:22:00.353 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 create --size 10G --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image-primary10 2024-09-12T13:22:00.353 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mountpt=test-primary9 2024-09-12T13:22:00.353 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ cp linux-5.4.tar.gz test-primary9 2024-09-12T13:22:00.422 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ is_pool_mirror_mode_image mirror 2024-09-12T13:22:00.422 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:22:00.423 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' image = image ']' 2024-09-12T13:22:00.423 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:22:00.423 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ enable_mirror cluster1 mirror image-primary10 snapshot 2024-09-12T13:22:00.423 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:22:00.423 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:22:00.423 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:22:00.423 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mode=snapshot 2024-09-12T13:22:00.423 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 mirror image enable mirror/image-primary10 snapshot 2024-09-12T13:22:00.644 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local ret=0 2024-09-12T13:22:00.644 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ timeout 5m bash -c 'zcat test-primary9/linux-5.4.tar.gz | pv -L 256K | tar xf - -C test-primary9' 2024-09-12T13:22:00.972 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:22:00.972 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:22:00.973 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:22:00.973 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:22:00.973 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:22:00.973 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:22:00.976 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1014 2024-09-12T13:22:00.977 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Mirroring enabled 2024-09-12T13:22:00.986 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:22:00.987 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:22:00.987 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:22:00.987 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:22:00.987 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:22:00.987 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:22:01.000 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1015 2024-09-12T13:22:01.000 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1016 2024-09-12T13:22:01.010 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:22:01.015 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 info mirror/image-primary10 2024-09-12T13:22:01.019 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:22:01.019 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:22:01.107 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:rbd image 'image-primary10': 2024-09-12T13:22:01.107 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: size 10 GiB in 2560 objects 2024-09-12T13:22:01.107 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: order 22 (4 MiB objects) 2024-09-12T13:22:01.107 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: snapshot_count: 1 2024-09-12T13:22:01.107 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: id: 1fffad224f18 2024-09-12T13:22:01.107 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: block_name_prefix: rbd_data.1fffad224f18 2024-09-12T13:22:01.108 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: format: 2 2024-09-12T13:22:01.108 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: features: layering, exclusive-lock, object-map, fast-diff 2024-09-12T13:22:01.108 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: op_features: 2024-09-12T13:22:01.108 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: flags: 2024-09-12T13:22:01.108 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: create_timestamp: Thu Sep 12 13:22:00 2024 2024-09-12T13:22:01.108 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: access_timestamp: Thu Sep 12 13:22:00 2024 2024-09-12T13:22:01.108 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: modify_timestamp: Thu Sep 12 13:22:00 2024 2024-09-12T13:22:01.108 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring state: enabled 2024-09-12T13:22:01.108 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring mode: snapshot 2024-09-12T13:22:01.108 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring global id: a020c6a3-79f5-4f05-82bf-2803d0febb5c 2024-09-12T13:22:01.108 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring primary: true 2024-09-12T13:22:01.112 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \n\b\d ]] 2024-09-12T13:22:01.112 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \k\r\b\d ]] 2024-09-12T13:22:01.113 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo rbd --cluster cluster1 device map -t krbd mirror/image-primary10 2024-09-12T13:22:01.270 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEV=/dev/rbd9 2024-09-12T13:22:01.270 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEVS+=($DEV) 2024-09-12T13:22:01.270 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo mkfs.ext4 /dev/rbd9 2024-09-12T13:22:01.291 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:mke2fs 1.46.5 (30-Dec-2021) 2024-09-12T13:22:01.303 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Discarding device blocks: 0/2621440 done 2024-09-12T13:22:01.303 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Creating filesystem with 2621440 4k blocks and 655360 inodes 2024-09-12T13:22:01.303 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Filesystem UUID: 45e44a45-7840-4340-9731-a61964959f25 2024-09-12T13:22:01.303 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Superblock backups stored on blocks: 2024-09-12T13:22:01.303 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 2024-09-12T13:22:01.303 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 2024-09-12T13:22:01.303 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Allocating group tables: 0/80 done 2024-09-12T13:22:01.304 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Writing inode tables: 0/80 done 2024-09-12T13:22:01.308 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Creating journal (16384 blocks): done 2024-09-12T13:22:01.395 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Writing superblocks and filesystem accounting information: 0/80 done 2024-09-12T13:22:01.395 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 2024-09-12T13:22:01.398 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mkdir test-primary10 2024-09-12T13:22:01.401 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo mount /dev/rbd9 test-primary10 2024-09-12T13:22:01.478 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ whoami 2024-09-12T13:22:01.480 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo chown ubuntu test-primary10 2024-09-12T13:22:01.505 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ SNAP_PIDS+=($!) 2024-09-12T13:22:01.505 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ take_mirror_snapshots cluster1 mirror image-primary10 2024-09-12T13:22:01.505 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:22:01.505 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:22:01.505 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:22:01.505 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:22:01.505 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:22:01.505 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:22:01.505 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:22:01.505 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:22:01.505 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ WORKLOAD_PIDS+=($!) 2024-09-12T13:22:01.505 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:22:01.506 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${SNAP_PIDS[@]} 2024-09-12T13:22:01.506 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 70860 2024-09-12T13:22:01.506 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ slow_untar_workload test-primary10 2024-09-12T13:22:01.506 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mountpt=test-primary10 2024-09-12T13:22:01.506 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ cp linux-5.4.tar.gz test-primary10 2024-09-12T13:22:01.810 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local ret=0 2024-09-12T13:22:01.810 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ timeout 5m bash -c 'zcat test-primary10/linux-5.4.tar.gz | pv -L 256K | tar xf - -C test-primary10' 2024-09-12T13:22:01.977 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:22:01.977 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:22:01.977 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:22:01.977 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:22:01.977 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:22:01.977 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:22:01.987 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:22:01.987 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:22:01.987 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:22:01.987 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:22:01.987 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:22:01.987 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:22:02.012 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1020 2024-09-12T13:22:02.032 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1019 2024-09-12T13:22:02.045 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:22:02.069 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1018 2024-09-12T13:22:02.076 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:22:02.115 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:22:02.976 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:22:02.976 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:22:02.976 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:22:02.976 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:22:02.977 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:22:02.977 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:22:02.982 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:22:02.982 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:22:02.982 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:22:02.982 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:22:02.983 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:22:02.983 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:22:03.014 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1022 2024-09-12T13:22:03.018 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1021 2024-09-12T13:22:03.065 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:22:03.078 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:22:03.966 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1024 2024-09-12T13:22:03.990 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:22:03.991 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1023 2024-09-12T13:22:04.016 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:22:04.016 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:22:04.016 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:22:04.017 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:22:04.017 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:22:04.017 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:22:04.023 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:22:04.025 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:22:04.025 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:22:04.029 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:22:04.029 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:22:04.029 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:22:04.029 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:22:04.031 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:22:04.031 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:22:04.031 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:22:04.031 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:22:04.031 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:22:04.031 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:22:05.011 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1026 2024-09-12T13:22:05.031 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1027 2024-09-12T13:22:05.031 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1025 2024-09-12T13:22:05.038 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:22:05.046 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:22:05.053 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:22:05.053 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:22:05.053 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:22:05.053 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:22:05.053 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:22:05.053 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:22:05.058 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:22:05.081 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:22:05.082 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:22:05.082 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:22:05.082 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:22:05.082 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:22:05.083 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:22:05.120 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:22:05.120 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:22:05.120 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:22:05.120 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:22:05.120 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:22:05.120 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:22:06.006 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1028 2024-09-12T13:22:06.018 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1030 2024-09-12T13:22:06.057 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:22:06.059 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:22:06.070 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:22:06.070 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:22:06.071 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:22:06.071 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:22:06.071 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:22:06.071 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:22:06.071 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1029 2024-09-12T13:22:06.088 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:22:06.088 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:22:06.089 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:22:06.089 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:22:06.089 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:22:06.090 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:22:06.118 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:22:07.000 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:22:07.001 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:22:07.001 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:22:07.001 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:22:07.001 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:22:07.001 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:22:07.001 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1031 2024-09-12T13:22:07.024 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1032 2024-09-12T13:22:07.027 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:22:07.028 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:22:07.029 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:22:07.030 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:22:07.030 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:22:07.030 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:22:07.050 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:22:07.072 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:22:07.954 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1033 2024-09-12T13:22:07.974 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1034 2024-09-12T13:22:07.993 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:22:08.036 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:22:08.043 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:22:08.043 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:22:08.043 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:22:08.043 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:22:08.043 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:22:08.043 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:22:08.051 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:22:08.051 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:22:08.051 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:22:08.051 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:22:08.052 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:22:08.052 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:22:08.063 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:22:08.064 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:22:08.064 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:22:08.064 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:22:08.064 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:22:08.064 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:22:09.032 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1037 2024-09-12T13:22:09.043 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1035 2024-09-12T13:22:09.063 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:22:09.063 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:22:09.065 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:22:09.065 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:22:09.065 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:22:09.065 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:22:09.065 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:22:09.065 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:22:09.065 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:22:09.065 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:22:09.065 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:22:09.065 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:22:09.066 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:22:09.090 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:22:09.130 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:22:09.130 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:22:09.130 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:22:09.130 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:22:09.130 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:22:09.131 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:22:09.189 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1036 2024-09-12T13:22:09.220 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:22:10.041 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1039 2024-09-12T13:22:10.115 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:22:10.115 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:22:10.115 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:22:10.115 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:22:10.115 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:22:10.115 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:22:10.116 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1040 2024-09-12T13:22:10.116 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:22:10.116 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:22:10.117 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:22:10.117 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:22:10.117 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:22:10.117 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:22:10.117 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:22:10.117 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1038 2024-09-12T13:22:10.117 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:22:10.132 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:22:10.997 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:22:10.997 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:22:10.997 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:22:10.997 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:22:10.998 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:22:10.998 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:22:11.042 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:22:11.042 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:22:11.042 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:22:11.043 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:22:11.043 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:22:11.043 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:22:11.082 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1041 2024-09-12T13:22:11.132 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:22:11.168 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1042 2024-09-12T13:22:11.192 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:22:12.071 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:22:12.072 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:22:12.072 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:22:12.072 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:22:12.072 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:22:12.072 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:22:12.096 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:22:12.097 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:22:12.097 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:22:12.097 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:22:12.097 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:22:12.097 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:22:12.129 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1043 2024-09-12T13:22:12.154 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1044 2024-09-12T13:22:12.155 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:22:12.177 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:22:12.230 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:22:12.230 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:22:12.231 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:22:12.231 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:22:12.231 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:22:12.231 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:22:13.034 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1046 2024-09-12T13:22:13.049 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:22:13.051 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1045 2024-09-12T13:22:13.075 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:22:13.075 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:22:13.075 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:22:13.075 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:22:13.076 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:22:13.076 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:22:13.087 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:22:13.108 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:22:13.108 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:22:13.108 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:22:13.109 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:22:13.109 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:22:13.109 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:22:13.136 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:22:13.136 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:22:13.136 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:22:13.136 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:22:13.136 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:22:13.136 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:22:14.038 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1048 2024-09-12T13:22:14.074 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:22:14.106 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1049 2024-09-12T13:22:14.119 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:22:14.137 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:22:14.137 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:22:14.137 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:22:14.137 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:22:14.138 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:22:14.138 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:22:14.195 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:22:14.195 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:22:14.195 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:22:14.195 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:22:14.195 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:22:14.195 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:22:15.002 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1047 2024-09-12T13:22:15.045 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:22:15.067 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1051 2024-09-12T13:22:15.099 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:22:15.161 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:22:15.161 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:22:15.161 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:22:15.161 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:22:15.161 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:22:15.161 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:22:15.180 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:22:15.180 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:22:15.180 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:22:15.180 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:22:15.180 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:22:15.181 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:22:15.985 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1050 2024-09-12T13:22:16.024 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:22:16.054 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:22:16.054 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:22:16.055 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:22:16.055 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:22:16.055 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:22:16.055 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:22:16.092 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:22:16.092 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:22:16.093 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:22:16.093 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:22:16.093 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:22:16.093 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:22:16.199 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1053 2024-09-12T13:22:16.222 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:22:16.985 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1052 2024-09-12T13:22:17.022 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:22:17.048 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:22:17.049+0000 7f67dc600640 -1 rbd::mirror::image_replayer::snapshot::Replayer: 0x56055c2f8400 handle_unlink_peer: failed to unlink local peer from remote image: (22) Invalid argument 2024-09-12T13:22:17.078 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:22:17.078 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:22:17.078 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:22:17.079 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1056 2024-09-12T13:22:17.079 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:22:17.079 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:22:17.079 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:22:17.096 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:22:17.122 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:22:17.123 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:22:17.123 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:22:17.123 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:22:17.123 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:22:17.123 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:22:17.932 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1054 2024-09-12T13:22:17.971 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1055 2024-09-12T13:22:17.983 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:22:18.004 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:22:18.025 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1057 2024-09-12T13:22:18.048 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:22:18.048 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:22:18.048 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:22:18.048 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:22:18.048 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:22:18.048 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:22:18.074 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:22:18.102 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:22:18.102 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:22:18.102 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:22:18.103 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:22:18.103 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:22:18.103 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:22:19.027 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:22:19.027 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:22:19.027 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:22:19.027 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:22:19.027 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:22:19.028 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:22:19.101 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1059 2024-09-12T13:22:19.143 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:22:19.225 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:22:19.226 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:22:19.226 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:22:19.226 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:22:19.226 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:22:19.226 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:22:20.014 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1058 2024-09-12T13:22:20.025 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:22:20.026 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:22:20.026 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:22:20.026 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:22:20.026 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:22:20.026 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:22:20.080 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:22:20.099 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:22:20.099 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:22:20.099 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:22:20.099 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:22:20.099 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:22:20.099 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:22:20.166 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1061 2024-09-12T13:22:20.183 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:22:20.986 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:22:20.986 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:22:20.987 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:22:20.987 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:22:20.987 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:22:20.987 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:22:21.008 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:22:21.008 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:22:21.008 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:22:21.008 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:22:21.008 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:22:21.008 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:22:21.016 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1060 2024-09-12T13:22:21.081 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:22:21.081 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:22:21.081 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:22:21.081 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:22:21.081 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:22:21.081 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:22:21.121 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:22:21.122+0000 7f67dd000640 -1 rbd::mirror::image_replayer::snapshot::Replayer: 0x56055c2db400 handle_unlink_peer: failed to unlink local peer from remote image: (22) Invalid argument 2024-09-12T13:22:21.130 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:22:21.240 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1063 2024-09-12T13:22:21.286 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:22:22.013 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1062 2024-09-12T13:22:22.089 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:22:22.117 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1066 2024-09-12T13:22:22.136 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:22:22.146 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:22:22.146 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:22:22.146 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:22:22.146 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:22:22.146 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:22:22.146 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:22:23.026 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1064 2024-09-12T13:22:23.087 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:22:23.087 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:22:23.087 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:22:23.087 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:22:23.087 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:22:23.087 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:22:23.092 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:22:23.185 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:22:23.186 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:22:23.186 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:22:23.186 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:22:23.186 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:22:23.186 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:22:24.135 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:22:24.135 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:22:24.135 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:22:24.135 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:22:24.135 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:22:24.135 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:22:24.149 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1067 2024-09-12T13:22:24.160 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1065 2024-09-12T13:22:24.199 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:22:24.199 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:22:24.289 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:22:24.289 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:22:24.289 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:22:24.290 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:22:24.290 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:22:24.290 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:22:24.295 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1069 2024-09-12T13:22:24.317 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:22:25.093 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:22:25.093 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:22:25.093 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:22:25.093 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:22:25.093 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:22:25.094 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:22:25.139 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:22:25.139 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:22:25.139 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:22:25.139 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:22:25.139 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:22:25.139 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:22:25.161 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1068 2024-09-12T13:22:25.200 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:22:25.247 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1071 2024-09-12T13:22:25.262 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:22:26.095 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:22:26.095 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:22:26.095 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:22:26.096 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:22:26.096 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:22:26.096 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:22:26.153 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1070 2024-09-12T13:22:26.193 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:22:26.334 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1073 2024-09-12T13:22:26.356 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:22:27.164 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1072 2024-09-12T13:22:27.199 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:22:27.203 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:22:27.203 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:22:27.203 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:22:27.203 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:22:27.204 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:22:27.204 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:22:27.204 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:22:27.204 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:22:27.204 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:22:27.204 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:22:27.205 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:22:27.205 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:22:27.319 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:22:27.320 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:22:27.320 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:22:27.320 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:22:27.320 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:22:27.320 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:22:27.383 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1075 2024-09-12T13:22:27.400 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:22:27.908 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1074 2024-09-12T13:22:27.967 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:22:28.054 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1077 2024-09-12T13:22:28.057 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1076 2024-09-12T13:22:28.109 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:22:28.129 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:22:28.202 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:22:28.203 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:22:28.203 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:22:28.203 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:22:28.203 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:22:28.203 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:22:28.265 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:22:28.265 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:22:28.265 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:22:28.265 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:22:28.266 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:22:28.266 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:22:29.197 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:22:29.197 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:22:29.197 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:22:29.197 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:22:29.197 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:22:29.197 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:22:29.359 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:22:29.359 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:22:29.360 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:22:29.360 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:22:29.360 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:22:29.360 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:22:29.675 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1079 2024-09-12T13:22:29.727 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:22:30.180 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1078 2024-09-12T13:22:30.202 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:22:30.202 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:22:30.202 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:22:30.202 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:22:30.202 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:22:30.202 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:22:30.227 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:22:30.313 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1082 2024-09-12T13:22:30.313 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1081 2024-09-12T13:22:30.328 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:22:30.328 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:22:30.402 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:22:30.403 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:22:30.403 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:22:30.403 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:22:30.403 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:22:30.403 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:22:30.970 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:22:30.970 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:22:30.970 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:22:30.971 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:22:30.971 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:22:30.971 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:22:31.114 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:22:31.114 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:22:31.114 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:22:31.114 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:22:31.114 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:22:31.114 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:22:31.133 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:22:31.133 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:22:31.133 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:22:31.133 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:22:31.133 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:22:31.133 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:22:31.156 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1080 2024-09-12T13:22:31.196 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:22:31.247 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1083 2024-09-12T13:22:31.293 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:22:31.293 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1085 2024-09-12T13:22:31.302 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:22:32.730 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:22:32.730 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:22:32.730 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:22:32.730 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:22:32.730 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:22:32.730 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:22:32.928 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1084 2024-09-12T13:22:32.964 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:22:32.976 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1086 2024-09-12T13:22:32.999 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1087 2024-09-12T13:22:33.013 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:22:33.035 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:22:33.230 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:22:33.230 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:22:33.231 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:22:33.231 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:22:33.231 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:22:33.231 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:22:33.330 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:22:33.330 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:22:33.330 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:22:33.330 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:22:33.330 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:22:33.331 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:22:33.331 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:22:33.331 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:22:33.331 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:22:33.331 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:22:33.331 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:22:33.331 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:22:33.973 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1088 2024-09-12T13:22:33.981 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1091 2024-09-12T13:22:33.982 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:22:33.985 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1089 2024-09-12T13:22:33.997 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:22:34.000 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:22:34.199 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:22:34.199 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:22:34.199 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:22:34.199 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:22:34.200 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:22:34.200 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:22:34.296 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:22:34.296 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:22:34.296 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:22:34.296 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:22:34.296 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:22:34.296 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:22:34.305 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:22:34.305 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:22:34.305 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:22:34.305 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:22:34.305 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:22:34.305 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:22:35.012 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1092 2024-09-12T13:22:35.023 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:22:35.925 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1090 2024-09-12T13:22:35.957 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:22:35.963 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:22:35.963 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:22:35.963 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:22:35.964 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:22:35.964 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:22:35.964 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:22:36.019 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:22:36.019 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:22:36.019 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:22:36.019 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:22:36.019 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:22:36.020 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:22:36.040 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:22:36.040 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:22:36.040 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:22:36.040 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:22:36.040 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:22:36.040 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:22:36.938 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1093 2024-09-12T13:22:36.950 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1094 2024-09-12T13:22:36.985 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:22:36.985 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:22:36.985 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:22:36.986 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:22:36.986 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:22:36.986 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:22:36.994 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:22:36.999 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:22:36.999 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:22:37.000 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:22:37.000 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:22:37.000 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:22:37.000 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:22:37.003 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:22:37.003 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:22:37.003 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:22:37.003 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:22:37.003 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:22:37.003 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:22:37.009 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:22:37.088 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1095 2024-09-12T13:22:37.108 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:22:38.026 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:22:38.026 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:22:38.027 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:22:38.027 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:22:38.027 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:22:38.027 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:22:38.148 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1098 2024-09-12T13:22:38.163 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:22:38.957 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1096 2024-09-12T13:22:38.959 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:22:38.959 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:22:38.959 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:22:38.959 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:22:38.959 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:22:38.959 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:22:38.965 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1097 2024-09-12T13:22:39.016 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:22:39.016 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:22:39.017 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:22:39.017+0000 7f67dc600640 -1 rbd::mirror::image_replayer::snapshot::Replayer: 0x56055c37ac00 handle_unlink_peer: failed to unlink local peer from remote image: (22) Invalid argument 2024-09-12T13:22:39.943 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1100 2024-09-12T13:22:39.949 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1099 2024-09-12T13:22:39.983 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:22:39.998 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:22:39.998 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:22:39.998 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:22:39.998 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:22:39.998 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:22:39.998 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:22:40.001 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:22:40.012 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:22:40.012 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:22:40.012 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:22:40.012 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:22:40.013 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:22:40.013 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:22:40.033 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1102 2024-09-12T13:22:40.044 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:22:40.110 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:22:40.110 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:22:40.111 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:22:40.111 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:22:40.111 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:22:40.111 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:22:40.934 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1101 2024-09-12T13:22:40.982 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:22:41.020 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1104 2024-09-12T13:22:41.026 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1103 2024-09-12T13:22:41.038 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:22:41.038 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:22:41.166 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:22:41.166 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:22:41.166 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:22:41.166 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:22:41.166 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:22:41.166 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:22:42.019 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:22:42.019 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:22:42.019 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:22:42.019 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:22:42.019 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:22:42.020 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:22:42.020 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:22:42.020 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:22:42.021 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:22:42.021 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:22:42.021 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:22:42.021 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:22:42.953 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1106 2024-09-12T13:22:42.960 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1105 2024-09-12T13:22:42.986 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:22:42.986 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:22:42.986 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:22:42.986 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:22:42.986 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:22:42.987 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:22:42.992 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:22:43.003 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:22:43.003 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:22:43.003 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:22:43.003 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:22:43.003 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:22:43.004 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:22:43.007 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:22:43.042 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1108 2024-09-12T13:22:43.046 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1107 2024-09-12T13:22:43.047 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:22:43.047 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:22:43.047 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:22:43.047 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:22:43.047 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:22:43.047 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:22:43.058 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:22:43.065 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:22:43.985 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:22:43.985 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:22:43.985 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:22:43.985 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:22:43.985 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:22:43.985 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:22:44.025 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1110 2024-09-12T13:22:44.027 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1109 2024-09-12T13:22:44.035 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:22:44.036 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:22:44.041 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:22:44.041 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:22:44.041 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:22:44.041 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:22:44.041 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:22:44.041 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:22:44.041 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:22:44.041 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:22:44.041 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:22:44.041 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:22:44.042 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:22:44.042 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:22:45.051 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1112 2024-09-12T13:22:45.071 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:22:45.951 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1111 2024-09-12T13:22:45.995 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:22:45.995 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:22:45.995 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:22:45.995 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:22:45.995 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:22:45.995 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:22:46.011 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:22:46.011 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:22:46.011 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:22:46.011 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:22:46.011 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:22:46.011 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:22:46.019 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:22:46.061 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:22:46.062 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:22:46.062 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:22:46.062 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:22:46.062 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:22:46.062 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:22:46.067 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:22:46.067 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:22:46.067 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:22:46.067 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:22:46.067 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:22:46.067 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:22:46.984 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1113 2024-09-12T13:22:47.003 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1114 2024-09-12T13:22:47.038 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:22:47.038 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:22:47.038 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:22:47.038 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:22:47.038 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:22:47.039 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:22:47.039 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:22:47.039 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:22:47.039 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:22:47.040 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:22:47.040 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:22:47.040 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:22:47.044 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:22:47.067 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:22:47.141 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1116 2024-09-12T13:22:47.154 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:22:47.974 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1115 2024-09-12T13:22:48.039 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:22:48.039+0000 7f67dd000640 -1 rbd::mirror::image_replayer::snapshot::Replayer: 0x56055b900c00 handle_unlink_peer: failed to unlink local peer from remote image: (22) Invalid argument 2024-09-12T13:22:48.071 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:22:48.074 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:22:48.074 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:22:48.074 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:22:48.074 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:22:48.074 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:22:48.074 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:22:48.097 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1119 2024-09-12T13:22:48.168 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:22:48.970 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1118 2024-09-12T13:22:48.980 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1117 2024-09-12T13:22:49.022 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:22:49.022 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:22:49.022 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:22:49.022 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:22:49.022 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:22:49.022 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:22:49.029 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:22:49.035 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:22:49.961 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1120 2024-09-12T13:22:50.004 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:22:50.038 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1122 2024-09-12T13:22:50.046 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:22:50.046 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:22:50.047 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:22:50.047 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:22:50.047 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:22:50.047 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:22:50.055 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:22:50.070 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:22:50.070 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:22:50.070 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:22:50.070 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:22:50.070 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:22:50.070 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:22:50.156 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:22:50.157 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:22:50.157 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:22:50.157 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:22:50.157 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:22:50.157 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:22:50.987 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1121 2024-09-12T13:22:51.011 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:22:51.040 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1124 2024-09-12T13:22:51.050 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:22:51.051 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1123 2024-09-12T13:22:51.061 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:22:51.074 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:22:51.074 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:22:51.074 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:22:51.074 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:22:51.075 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:22:51.075 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:22:51.170 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:22:51.171 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:22:51.171 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:22:51.171 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:22:51.171 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:22:51.171 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:22:52.026 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1125 2024-09-12T13:22:52.032 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:22:52.033 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:22:52.033 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:22:52.033 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:22:52.033 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:22:52.033 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:22:52.037 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:22:52.040 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:22:52.040 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:22:52.040 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:22:52.040 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:22:52.040 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:22:52.040 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:22:53.007 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:22:53.007 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:22:53.007 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:22:53.008 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:22:53.008 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:22:53.008 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:22:53.052 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1129 2024-09-12T13:22:53.057 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:22:53.058 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:22:53.058 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:22:53.058 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:22:53.058 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:22:53.058 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:22:53.072 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:22:53.075 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1128 2024-09-12T13:22:53.086 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:22:54.016 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:22:54.016 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:22:54.016 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:22:54.016 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:22:54.016 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:22:54.017 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:22:54.046 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1127 2024-09-12T13:22:54.053 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:22:54.053 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:22:54.053 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:22:54.053 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:22:54.053 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:22:54.053 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:22:54.054 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1126 2024-09-12T13:22:54.064 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:22:54.064 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:22:54.064 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:22:54.064 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:22:54.064 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:22:54.064 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:22:54.065 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:22:54.076 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:22:54.080 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1130 2024-09-12T13:22:54.100 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:22:55.040 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1132 2024-09-12T13:22:55.041 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:22:55.041 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:22:55.041 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:22:55.041 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:22:55.042 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:22:55.042 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:22:55.056 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:22:56.007 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1131 2024-09-12T13:22:56.047 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:22:56.075 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:22:56.075 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:22:56.075 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:22:56.076 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:22:56.076 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:22:56.076 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:22:56.089 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:22:56.089 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:22:56.089 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:22:56.089 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:22:56.089 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:22:56.089 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:22:57.000 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1133 2024-09-12T13:22:57.020 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1134 2024-09-12T13:22:57.045 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:22:57.055 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:22:57.068 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:22:57.068 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:22:57.068 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:22:57.069 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:22:57.069 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:22:57.069 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:22:57.079 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:22:57.079 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:22:57.079 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:22:57.080 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:22:57.080 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:22:57.080 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:22:57.085 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1136 2024-09-12T13:22:57.100 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:22:57.103 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:22:57.103 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:22:57.103 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:22:57.103 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:22:57.103 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:22:57.103 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:22:57.937 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1135 2024-09-12T13:22:57.984 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:22:58.059 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:22:58.059 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:22:58.059 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:22:58.059 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:22:58.059 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:22:58.060 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:22:59.024 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1137 2024-09-12T13:22:59.049 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:22:59.050 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:22:59.050 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:22:59.050 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:22:59.050 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:22:59.050 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:22:59.068 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:23:00.013 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1140 2024-09-12T13:23:00.031 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1138 2024-09-12T13:23:00.046 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:23:00.048 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:23:00.048 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:23:00.048 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:23:00.048 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:23:00.048 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:23:00.048 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:23:00.060 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:23:00.060 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:23:00.060 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:23:00.060 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:23:00.060 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:23:00.061 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:23:00.066 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:23:00.103 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:23:00.104 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:23:00.104 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:23:00.104 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:23:00.104 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:23:00.104 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:23:00.106 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1142 2024-09-12T13:23:00.138 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:23:00.987 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:23:00.987 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:23:00.987 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:23:00.987 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:23:00.988 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:23:00.988 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:23:01.021 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1141 2024-09-12T13:23:01.059 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:23:01.082 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1144 2024-09-12T13:23:01.099 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1143 2024-09-12T13:23:01.105 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:23:01.109 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:23:01.111 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1145 2024-09-12T13:23:01.120 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:23:02.017 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1139 2024-09-12T13:23:02.039 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:23:02.050 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1146 2024-09-12T13:23:02.070 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:23:02.070 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:23:02.070 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:23:02.071 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:23:02.071 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:23:02.071 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:23:02.071 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:23:03.045 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1147 2024-09-12T13:23:03.049 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:23:03.049 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:23:03.049 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:23:03.049 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:23:03.049 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:23:03.049 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:23:03.054 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:23:03.069 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:23:03.069 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:23:03.069 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:23:03.069 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:23:03.070 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:23:03.070 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:23:03.140 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:23:03.141 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:23:03.141 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:23:03.141 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:23:03.141 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:23:03.141 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:23:04.053 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1149 2024-09-12T13:23:04.062 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:23:04.062 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:23:04.062 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:23:04.062 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:23:04.062 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:23:04.063 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:23:04.069 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1148 2024-09-12T13:23:04.071 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:23:04.078 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:23:04.108 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:23:04.108 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:23:04.108 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:23:04.108 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:23:04.108 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:23:04.108 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:23:04.112 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:23:04.113 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:23:04.113 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:23:04.113 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:23:04.113 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:23:04.113 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:23:04.123 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:23:04.123 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:23:04.123 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:23:04.123 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:23:04.123 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:23:04.123 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:23:05.042 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:23:05.042 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:23:05.043 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:23:05.043 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:23:05.043 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:23:05.043 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:23:05.073 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:23:05.073 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:23:05.073 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:23:05.073 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:23:05.073 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:23:05.073 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:23:05.087 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1151 2024-09-12T13:23:05.099 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1154 2024-09-12T13:23:05.099 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:23:05.108 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:23:06.032 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1150 2024-09-12T13:23:06.057 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:23:06.057 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:23:06.057 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:23:06.057 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:23:06.057 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:23:06.057 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:23:06.082 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:23:06.122 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1155 2024-09-12T13:23:06.136 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:23:07.031 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1152 2024-09-12T13:23:07.050 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1153 2024-09-12T13:23:07.068 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:23:07.075 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:23:07.075 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:23:07.075 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:23:07.075 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:23:07.075 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:23:07.075 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:23:07.080 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:23:07.081 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:23:07.081 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:23:07.082 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:23:07.082 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:23:07.082 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:23:07.082 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:23:07.940 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1156 2024-09-12T13:23:07.978 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1157 2024-09-12T13:23:07.990 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:23:07.991 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:23:08.102 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:23:08.102 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:23:08.102 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:23:08.102 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:23:08.102 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:23:08.102 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:23:08.110 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:23:08.110 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:23:08.110 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:23:08.110 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:23:08.111 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:23:08.111 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:23:09.070 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:23:09.070+0000 7f67dc600640 -1 rbd::mirror::image_replayer::snapshot::Replayer: 0x56055623a000 handle_unlink_peer: failed to unlink local peer from remote image: (22) Invalid argument 2024-09-12T13:23:09.085 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:23:09.085 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:23:09.085 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:23:09.085 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:23:09.085 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:23:09.085 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:23:09.139 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:23:09.139 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:23:09.139 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:23:09.139 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:23:09.139 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:23:09.139 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:23:10.038 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1158 2024-09-12T13:23:10.051 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1159 2024-09-12T13:23:10.057 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1161 2024-09-12T13:23:10.071 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:23:10.071 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:23:10.071 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:23:10.071 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:23:10.071 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:23:10.071 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:23:10.083 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:23:10.083 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:23:10.083 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:23:10.083 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:23:10.083 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:23:10.083 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:23:10.085 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:23:10.090 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:23:10.102 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:23:10.127 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1162 2024-09-12T13:23:10.148 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:23:10.994 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:23:10.994 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:23:10.994 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:23:10.994 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:23:10.994 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:23:10.994 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:23:10.994 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:23:10.994 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:23:10.994 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:23:10.994 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:23:10.994 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:23:10.995 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:23:11.048 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1160 2024-09-12T13:23:11.095 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:23:11.111 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1165 2024-09-12T13:23:11.136 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:23:11.138 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1164 2024-09-12T13:23:11.146 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:23:12.036 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1163 2024-09-12T13:23:12.060 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:23:12.090 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1167 2024-09-12T13:23:12.098 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:23:13.088 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:23:13.088 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:23:13.088 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:23:13.088 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:23:13.088 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:23:13.089 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:23:13.093 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:23:13.093 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:23:13.093 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:23:13.093 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:23:13.093 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:23:13.093 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:23:13.106 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:23:13.106 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:23:13.106 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:23:13.106 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:23:13.106 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:23:13.106 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:23:13.151 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:23:13.151 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:23:13.151 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:23:13.151 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:23:13.151 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:23:13.151 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:23:14.047 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1166 2024-09-12T13:23:14.077 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:23:14.098 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:23:14.098 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:23:14.098 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:23:14.098 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:23:14.098 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:23:14.098 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:23:14.104 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1170 2024-09-12T13:23:14.110 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1169 2024-09-12T13:23:14.125 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:23:14.127 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:23:14.139 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:23:14.139 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:23:14.139 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:23:14.139 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:23:14.139 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:23:14.139 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:23:14.149 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:23:14.149 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:23:14.149 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:23:14.149 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:23:14.149 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:23:14.149 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:23:15.059 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1168 2024-09-12T13:23:15.063 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:23:15.063 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:23:15.063 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:23:15.063 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:23:15.063 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:23:15.063 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:23:15.087 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:23:15.101 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:23:15.101 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:23:15.101 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:23:15.101 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:23:15.102 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:23:15.102 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:23:15.116 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1172 2024-09-12T13:23:15.125 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:23:16.059 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1171 2024-09-12T13:23:16.115 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:23:16.154 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1175 2024-09-12T13:23:16.163 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:23:17.058 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1173 2024-09-12T13:23:17.063 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1174 2024-09-12T13:23:17.080 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:23:17.080 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:23:17.080 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:23:17.080 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:23:17.080 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:23:17.080 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:23:17.098 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:23:17.100 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:23:17.128 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:23:17.128 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:23:17.128 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:23:17.129 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:23:17.129 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:23:17.129 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:23:17.130 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:23:17.130 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:23:17.130 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:23:17.130 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:23:17.130 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:23:17.131 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:23:17.942 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1176 2024-09-12T13:23:17.986 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:23:18.018 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1177 2024-09-12T13:23:18.022 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1179 2024-09-12T13:23:18.028 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:23:18.043 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:23:18.090 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:23:18.090 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:23:18.090 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:23:18.090 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:23:18.090 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:23:18.090 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:23:18.129 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:23:18.129 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:23:18.129 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:23:18.130 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:23:18.130 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:23:18.130 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:23:19.118 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:23:19.118 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:23:19.118 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:23:19.119 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:23:19.119 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:23:19.119 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:23:19.166 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:23:19.166 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:23:19.166 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:23:19.166 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:23:19.166 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:23:19.166 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:23:20.080 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1180 2024-09-12T13:23:20.085 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1178 2024-09-12T13:23:20.101 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:23:20.101 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:23:20.101 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:23:20.101 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:23:20.101 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:23:20.101 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:23:20.102 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:23:20.103 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:23:20.103 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:23:20.103 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:23:20.103 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:23:20.103 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:23:20.131 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:23:20.134 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:23:20.142 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1182 2024-09-12T13:23:20.164 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:23:20.990 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:23:20.990 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:23:20.990 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:23:20.990 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:23:20.990 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:23:20.990 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:23:21.031 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:23:21.031 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:23:21.031 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:23:21.031 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:23:21.032 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:23:21.032 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:23:21.046 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:23:21.046 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:23:21.046 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:23:21.046 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:23:21.046 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:23:21.046 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:23:21.097 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1181 2024-09-12T13:23:21.134 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:23:21.161 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1185 2024-09-12T13:23:21.174 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1184 2024-09-12T13:23:21.183 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:23:21.183 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:23:22.090 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1183 2024-09-12T13:23:22.110 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:23:22.130 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1186 2024-09-12T13:23:22.143 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:23:23.135 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:23:23.135 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:23:23.135 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:23:23.135 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:23:23.135 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:23:23.135 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:23:23.137 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:23:23.137 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:23:23.137 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:23:23.137 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:23:23.137 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:23:23.137 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:23:23.167 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:23:23.167 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:23:23.167 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:23:23.167 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:23:23.167 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:23:23.168 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:23:24.105 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1187 2024-09-12T13:23:24.137 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:23:24.137 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:23:24.137 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:23:24.137 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:23:24.137 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:23:24.137 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:23:24.153 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:23:24.153+0000 7f67dc600640 -1 rbd::mirror::image_replayer::snapshot::Replayer: 0x560556d0fc00 handle_unlink_peer: failed to unlink local peer from remote image: (22) Invalid argument 2024-09-12T13:23:24.157 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:23:24.173 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1189 2024-09-12T13:23:24.188 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:23:24.189 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:23:24.189 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:23:24.189 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:23:24.189 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:23:24.189 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:23:24.189 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:23:24.189 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:23:24.189 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:23:24.189 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:23:24.189 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:23:24.189 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:23:24.190 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:23:25.089 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1188 2024-09-12T13:23:25.113 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:23:25.113 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:23:25.114 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:23:25.114 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:23:25.114 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:23:25.114 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:23:25.145 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:23:25.146 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:23:25.146 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:23:25.146 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:23:25.146 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:23:25.146 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:23:25.155 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:23:25.172 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1192 2024-09-12T13:23:25.197 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:23:26.110 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1191 2024-09-12T13:23:26.166 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:23:26.210 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1195 2024-09-12T13:23:26.224 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:23:27.098 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1194 2024-09-12T13:23:27.112 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1193 2024-09-12T13:23:27.138 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:23:27.145 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:23:27.161 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:23:27.161 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:23:27.161 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:23:27.161 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:23:27.161 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:23:27.162 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:23:27.191 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:23:27.192 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:23:27.192 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:23:27.192 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:23:27.192 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:23:27.192 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:23:27.936 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1196 2024-09-12T13:23:27.971 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:23:27.975 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:23:27.975+0000 7f67dd000640 -1 rbd::mirror::image_replayer::snapshot::Replayer: 0x56055b999000 handle_unlink_peer: failed to unlink local peer from remote image: (22) Invalid argument 2024-09-12T13:23:28.004 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1197 2024-09-12T13:23:28.018 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:23:28.158 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:23:28.158 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:23:28.158 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:23:28.158 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:23:28.159 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:23:28.159 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:23:28.201 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:23:28.201 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:23:28.201 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:23:28.201 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:23:28.201 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:23:28.201 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:23:28.934 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1190 2024-09-12T13:23:28.970 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:23:29.002 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1199 2024-09-12T13:23:29.016 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:23:29.168 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:23:29.169 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:23:29.169 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:23:29.169 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:23:29.169 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:23:29.169 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:23:29.227 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:23:29.227 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:23:29.227 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:23:29.227 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:23:29.227 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:23:29.227 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:23:29.938 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1198 2024-09-12T13:23:29.979 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:23:29.980 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:23:29.980+0000 7f67dd000640 -1 rbd::mirror::image_replayer::snapshot::Replayer: 0x56055c2b2000 handle_unlink_peer: failed to unlink local peer from remote image: (22) Invalid argument 2024-09-12T13:23:29.998 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1201 2024-09-12T13:23:30.013 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:23:30.141 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:23:30.141 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:23:30.142 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:23:30.142 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:23:30.142 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:23:30.142 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:23:30.147 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:23:30.147 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:23:30.148 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:23:30.148 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:23:30.148 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:23:30.148 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:23:30.957 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1200 2024-09-12T13:23:30.974 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:23:30.974 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:23:30.974 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:23:30.974 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:23:30.975 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:23:30.975 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:23:30.982 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:23:30.997 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1204 2024-09-12T13:23:31.005 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1203 2024-09-12T13:23:31.014 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:23:31.014 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:23:31.020 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:23:31.021 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:23:31.021 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:23:31.021 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:23:31.021 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:23:31.021 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:23:31.953 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1202 2024-09-12T13:23:31.973 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:23:31.974 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:23:31.974 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:23:31.974 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:23:31.974 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:23:31.974 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:23:31.980 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:23:31.998 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1205 2024-09-12T13:23:32.014 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:23:32.018 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:23:32.019 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:23:32.019 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:23:32.019 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:23:32.019 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:23:32.019 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:23:32.979 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1206 2024-09-12T13:23:32.982 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:23:32.982 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:23:32.982 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:23:32.982 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:23:32.982 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:23:32.983 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:23:33.002 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1208 2024-09-12T13:23:33.003 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1207 2024-09-12T13:23:33.005 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:23:33.016 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:23:33.017 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:23:33.017 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:23:33.017 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:23:33.017 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:23:33.017 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:23:33.022 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:23:33.023 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:23:33.982 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1209 2024-09-12T13:23:33.985 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:23:33.985 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:23:33.985 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:23:33.985 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:23:33.985 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:23:33.985 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:23:33.998 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:23:34.017 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:23:34.017 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:23:34.017 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:23:34.017 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:23:34.017 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:23:34.017 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:23:34.017 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:23:34.017 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:23:34.018 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:23:34.018 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:23:34.018 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:23:34.018 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:23:34.983 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:23:34.983 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:23:34.983 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:23:34.983 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:23:34.983 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:23:34.983 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:23:35.008 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1211 2024-09-12T13:23:35.017 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:23:35.017 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:23:35.018 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:23:35.018 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:23:35.018 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:23:35.018 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:23:35.022 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:23:35.963 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1210 2024-09-12T13:23:36.008 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:23:36.009 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:23:36.009 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:23:36.009 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:23:36.009 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:23:36.009 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:23:36.009 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:23:36.023 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:23:36.023+0000 7f67dc600640 -1 rbd::mirror::image_replayer::snapshot::Replayer: 0x56055c300400 handle_unlink_peer: failed to unlink local peer from remote image: (22) Invalid argument 2024-09-12T13:23:36.026 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:23:36.026 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:23:36.026 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:23:36.026 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:23:36.026 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:23:36.026 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:23:36.027 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:23:36.027 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:23:36.027 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:23:36.027 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:23:36.027 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:23:36.027 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:23:36.065 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1214 2024-09-12T13:23:36.106 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:23:36.984 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1212 2024-09-12T13:23:36.987 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1213 2024-09-12T13:23:37.001 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:23:37.001 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:23:37.001 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:23:37.001 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:23:37.001 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:23:37.001 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:23:37.001 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:23:37.001+0000 7f67dd000640 -1 rbd::mirror::image_replayer::snapshot::Replayer: 0x560556a8a800 handle_unlink_peer: failed to unlink local peer from remote image: (22) Invalid argument 2024-09-12T13:23:37.027 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:23:37.031 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:23:37.071 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1217 2024-09-12T13:23:37.080 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1218 2024-09-12T13:23:37.092 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:23:37.105 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:23:37.941 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1215 2024-09-12T13:23:37.970 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:23:38.025 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:23:38.025 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:23:38.026 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:23:38.026 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:23:38.026 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:23:38.026 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:23:38.976 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1216 2024-09-12T13:23:39.009 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:23:39.011 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:23:39.011 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:23:39.011 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:23:39.011 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:23:39.011 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:23:39.012 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:23:39.027 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:23:39.027+0000 7f67dd000640 -1 rbd::mirror::image_replayer::snapshot::Replayer: 0x56055c37a800 handle_unlink_peer: failed to unlink local peer from remote image: (22) Invalid argument 2024-09-12T13:23:39.048 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1220 2024-09-12T13:23:39.063 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:23:39.108 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:23:39.108 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:23:39.108 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:23:39.109 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:23:39.109 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:23:39.109 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:23:39.963 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1219 2024-09-12T13:23:39.998 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:23:40.020 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1221 2024-09-12T13:23:40.030 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:23:40.030 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:23:40.030 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:23:40.030 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:23:40.031 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:23:40.031 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:23:40.033 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:23:40.034 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:23:40.034 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:23:40.034 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:23:40.034 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:23:40.034 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:23:40.034 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:23:40.095 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:23:40.095 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:23:40.095 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:23:40.096 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:23:40.096 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:23:40.096 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:23:40.108 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:23:40.108 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:23:40.108 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:23:40.108 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:23:40.108 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:23:40.109 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:23:40.973 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:23:40.973 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:23:40.974 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:23:40.974 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:23:40.974 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:23:40.974 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:23:41.010 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:23:41.009+0000 7f67dc600640 -1 rbd::mirror::image_replayer::snapshot::Replayer: 0x560556854c00 handle_unlink_peer: failed to unlink local peer from remote image: (22) Invalid argument 2024-09-12T13:23:41.036 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1223 2024-09-12T13:23:41.043 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1226 2024-09-12T13:23:41.045 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1225 2024-09-12T13:23:41.055 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1224 2024-09-12T13:23:41.055 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:23:41.063 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:23:41.066 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:23:41.067 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:23:41.989 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1222 2024-09-12T13:23:42.012 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:23:42.012 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:23:42.013 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:23:42.013 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:23:42.044 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:23:42.045 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:23:42.045 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:23:42.056 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1227 2024-09-12T13:23:42.065 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:23:42.065 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:23:42.066 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:23:42.066 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:23:42.066 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:23:42.066 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:23:42.073 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:23:43.001 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:23:43.001 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:23:43.001 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:23:43.001 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:23:43.001 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:23:43.001 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:23:43.009 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1228 2024-09-12T13:23:43.011 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1229 2024-09-12T13:23:43.028 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:23:43.028 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:23:43.036 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:23:43.036 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:23:43.036 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:23:43.036 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:23:43.036 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:23:43.036 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:23:44.029 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1230 2024-09-12T13:23:44.049 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:23:44.058 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:23:44.058 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:23:44.058 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:23:44.058 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:23:44.058 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:23:44.058 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:23:44.065 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:23:44.065 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:23:44.065 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:23:44.065 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:23:44.066 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:23:44.066 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:23:44.068 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:23:44.068 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:23:44.069 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:23:44.069 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:23:44.069 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:23:44.069 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:23:44.070 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:23:44.070 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:23:44.070 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:23:44.070 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:23:44.070 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:23:44.070 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:23:45.027 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:23:45.027 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:23:45.027 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:23:45.027 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:23:45.027 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:23:45.027 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:23:45.041 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1233 2024-09-12T13:23:45.063 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:23:45.063 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1235 2024-09-12T13:23:45.072 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:23:45.077 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:23:45.077 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:23:45.078 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:23:45.078 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:23:45.078 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:23:45.078 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:23:46.006 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1231 2024-09-12T13:23:46.009 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1232 2024-09-12T13:23:46.031 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:23:46.031 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:23:46.031 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:23:46.031 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:23:46.031 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:23:46.032 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:23:46.032 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:23:46.032 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:23:46.032 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:23:46.032 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:23:46.032 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:23:46.032 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:23:46.046 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:23:46.066 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:23:46.091 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1237 2024-09-12T13:23:46.093 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1236 2024-09-12T13:23:46.113 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:23:46.114 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:23:47.002 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1234 2024-09-12T13:23:47.031 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:23:47.051 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:23:47.052 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:23:47.052 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:23:47.052 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:23:47.052 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:23:47.052 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:23:47.054 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1239 2024-09-12T13:23:47.068 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:23:47.975 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1238 2024-09-12T13:23:48.001 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:23:48.011 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1240 2024-09-12T13:23:48.036 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:23:48.068 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:23:48.068 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:23:48.068 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:23:48.069 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:23:48.069 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:23:48.069 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:23:48.074 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:23:48.075 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:23:48.075 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:23:48.075 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:23:48.075 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:23:48.075 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:23:49.032 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1241 2024-09-12T13:23:49.050 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:23:49.050 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:23:49.050 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:23:49.050 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:23:49.050 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:23:49.050 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:23:49.056 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:23:49.071 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:23:49.073 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:23:49.073 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:23:49.073 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:23:49.073 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:23:49.073 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:23:49.115 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:23:49.116 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:23:49.116 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:23:49.116 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:23:49.116 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:23:49.116 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:23:49.117 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:23:49.117 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:23:49.117 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:23:49.118 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:23:49.118 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:23:49.118 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:23:50.011 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1242 2024-09-12T13:23:50.034 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:23:50.034 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:23:50.034 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:23:50.034 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:23:50.034 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:23:50.034 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:23:50.057 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:23:50.071 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:23:50.071 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:23:50.071 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:23:50.071 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:23:50.071 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:23:50.071 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:23:50.071 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1244 2024-09-12T13:23:50.096 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:23:51.007 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:23:51.007 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:23:51.007 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:23:51.007 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:23:51.007 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:23:51.007 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:23:51.040 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:23:51.040 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:23:51.041 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:23:51.041 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:23:51.041 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:23:51.041 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:23:51.083 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1246 2024-09-12T13:23:51.091 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1243 2024-09-12T13:23:51.096 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1245 2024-09-12T13:23:51.125 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:23:51.129 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1247 2024-09-12T13:23:51.136 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:23:51.141 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:23:51.145 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:23:52.060 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1248 2024-09-12T13:23:52.061 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:23:52.061 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:23:52.061 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:23:52.061 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:23:52.061 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:23:52.061 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:23:52.089 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:23:53.063 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:23:53.063 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:23:53.063 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:23:53.063 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:23:53.063 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:23:53.063 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:23:53.067 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1249 2024-09-12T13:23:53.095 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:23:53.098 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:23:53.099 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:23:53.100 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:23:53.100 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:23:53.100 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:23:53.100 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:23:54.085 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1250 2024-09-12T13:23:54.127 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:23:54.129 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:23:54.130 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:23:54.130 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:23:54.130 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:23:54.130 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:23:54.130 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:23:54.142 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:23:54.142 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:23:54.142 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:23:54.142 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:23:54.142 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:23:54.142 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:23:54.144 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:23:54.144 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:23:54.144 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:23:54.145 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:23:54.145 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:23:54.145 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:23:54.153 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:23:54.153 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:23:54.153 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:23:54.153 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:23:54.153 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:23:54.153 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:23:55.093 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:23:55.094 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:23:55.094 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:23:55.094 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:23:55.094 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:23:55.094 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:23:56.071 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1253 2024-09-12T13:23:56.100 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:23:56.100 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:23:56.100 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:23:56.100 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:23:56.100 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:23:56.100 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:23:56.133 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:23:57.050 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1255 2024-09-12T13:23:57.060 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1251 2024-09-12T13:23:57.067 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1257 2024-09-12T13:23:57.097 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:23:57.107 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:23:57.119 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:23:57.130 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:23:57.130 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:23:57.130 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:23:57.130 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:23:57.130 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:23:57.131 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:23:57.954 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1254 2024-09-12T13:23:58.022 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:23:59.050 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1259 2024-09-12T13:23:59.057 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1252 2024-09-12T13:23:59.065 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1256 2024-09-12T13:23:59.093 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:23:59.095 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:23:59.110 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:23:59.136 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:23:59.136 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:23:59.136 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:23:59.136 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:23:59.136 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:23:59.136 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:24:00.058 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1260 2024-09-12T13:24:00.094 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:24:00.100 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:24:00.100 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:24:00.100 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:24:00.100 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:24:00.100 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:24:00.100 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:24:00.110 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:24:00.110 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:24:00.110 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:24:00.110 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:24:00.110 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:24:00.111 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:24:00.114 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1261 2024-09-12T13:24:00.122 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:24:00.122 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:24:00.122 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:24:00.122 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:24:00.122 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:24:00.122 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:24:00.137 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:24:01.025 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:24:01.025 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:24:01.025 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:24:01.026 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:24:01.026 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:24:01.026 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:24:01.056 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1258 2024-09-12T13:24:01.080 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:24:01.107 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1264 2024-09-12T13:24:01.109 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1262 2024-09-12T13:24:01.116 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1263 2024-09-12T13:24:01.122 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:24:01.139 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:24:01.142 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:24:02.091 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1265 2024-09-12T13:24:02.096 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:24:02.097 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:24:02.097 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:24:02.097 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:24:02.097 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:24:02.097 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:24:02.097 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:24:02.098 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:24:02.098 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:24:02.098 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:24:02.098 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:24:02.098 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:24:02.106 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:24:02.113 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:24:02.113 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:24:02.113 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:24:02.113 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:24:02.113 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:24:02.113 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:24:03.089 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1266 2024-09-12T13:24:03.097 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:24:03.097 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:24:03.097 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:24:03.097 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:24:03.097 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:24:03.097 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:24:03.097 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1267 2024-09-12T13:24:03.106 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:24:03.113 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:24:03.140 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:24:03.140 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:24:03.140 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:24:03.140 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:24:03.140 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:24:03.140 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:24:04.083 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:24:04.084 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:24:04.084 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:24:04.084 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:24:04.084 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:24:04.084 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:24:04.107 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1269 2024-09-12T13:24:04.119 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:24:04.125 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:24:04.125 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:24:04.125 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:24:04.125 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:24:04.125 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:24:04.125 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:24:04.141 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:24:04.141 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:24:04.142 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:24:04.142 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:24:04.142 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:24:04.142 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:24:04.145 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:24:04.145 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:24:04.145 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:24:04.145 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:24:04.145 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:24:04.145 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:24:05.068 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1268 2024-09-12T13:24:05.109 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:24:05.109 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:24:05.109 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:24:05.109 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:24:05.109 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:24:05.109 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:24:05.110 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:24:05.144 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1271 2024-09-12T13:24:05.159 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:24:06.071 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1270 2024-09-12T13:24:06.102 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:24:06.109 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:24:06.109 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:24:06.109 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:24:06.109 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:24:06.109 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:24:06.109 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:24:06.115 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:24:06.116 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:24:06.116 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:24:06.116 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:24:06.116 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:24:06.116 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:24:07.074 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1273 2024-09-12T13:24:07.081 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1274 2024-09-12T13:24:07.090 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1272 2024-09-12T13:24:07.118 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:24:07.122 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:24:07.122 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:24:07.122 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:24:07.122 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:24:07.122 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:24:07.122 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:24:07.131 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:24:07.141 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:24:07.941 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1275 2024-09-12T13:24:07.976 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:24:08.036 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:24:08.037+0000 7f67dd000640 -1 rbd::mirror::image_replayer::snapshot::Replayer: 0x56055b859400 handle_unlink_peer: failed to unlink local peer from remote image: (22) Invalid argument 2024-09-12T13:24:08.113 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:24:08.113 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:24:08.113 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:24:08.113 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:24:08.113 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:24:08.113 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:24:08.162 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:24:08.162 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:24:08.162 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:24:08.163 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:24:08.163 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:24:08.163 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:24:08.940 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1277 2024-09-12T13:24:08.952 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1276 2024-09-12T13:24:08.979 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:24:09.002 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:24:09.024 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:24:09.025+0000 7f67dd000640 -1 rbd::mirror::image_replayer::snapshot::Replayer: 0x560556c9ac00 handle_unlink_peer: failed to unlink local peer from remote image: (22) Invalid argument 2024-09-12T13:24:09.046 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1279 2024-09-12T13:24:09.061 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:24:09.105 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:24:09.105 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:24:09.106 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:24:09.106 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:24:09.106 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:24:09.106 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:24:09.955 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1278 2024-09-12T13:24:09.985 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:24:10.004 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:24:10.005+0000 7f67dd000640 -1 rbd::mirror::image_replayer::snapshot::Replayer: 0x56055c2db400 handle_unlink_peer: failed to unlink local peer from remote image: (22) Invalid argument 2024-09-12T13:24:10.006 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1281 2024-09-12T13:24:10.021 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:24:10.121 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:24:10.121 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:24:10.121 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:24:10.121 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:24:10.121 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:24:10.121 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:24:10.134 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:24:10.134 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:24:10.134 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:24:10.134 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:24:10.134 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:24:10.134 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:24:10.144 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:24:10.144 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:24:10.144 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:24:10.144 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:24:10.144 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:24:10.145 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:24:10.959 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1280 2024-09-12T13:24:10.979 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:24:10.979 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:24:10.979 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:24:10.979 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:24:10.979 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:24:10.979 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:24:10.981 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:24:11.000 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1283 2024-09-12T13:24:11.001 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1282 2024-09-12T13:24:11.004 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1284 2024-09-12T13:24:11.017 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:24:11.019 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:24:11.020 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:24:11.977 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1285 2024-09-12T13:24:11.982 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:24:11.982 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:24:11.982 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:24:11.982 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:24:11.982 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:24:11.982 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:24:11.996 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:24:12.004 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:24:12.004 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:24:12.004 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:24:12.004 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:24:12.004 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:24:12.004 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:24:12.064 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:24:12.064 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:24:12.064 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:24:12.064 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:24:12.064 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:24:12.064 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:24:12.989 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:24:12.989 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:24:12.989 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:24:12.989 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:24:12.989 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:24:12.989 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:24:12.999 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1287 2024-09-12T13:24:12.999 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1286 2024-09-12T13:24:13.021 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:24:13.024 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:24:13.024 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:24:13.024 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:24:13.024 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:24:13.025 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:24:13.025 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:24:13.026 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:24:13.985 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:24:13.985 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:24:13.985 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:24:13.985 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:24:13.985 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:24:13.985 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:24:13.985 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1288 2024-09-12T13:24:14.004 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:24:14.015 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1289 2024-09-12T13:24:14.019 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:24:14.019 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:24:14.019 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:24:14.019 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:24:14.020 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:24:14.020 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:24:14.022 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:24:14.022 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:24:14.022 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:24:14.022 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:24:14.022 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:24:14.022 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:24:14.023 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:24:14.023 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:24:14.023 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:24:14.023 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:24:14.023 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:24:14.023 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:24:14.043 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:24:14.999 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:24:14.999 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:24:15.000 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:24:15.000 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:24:15.000 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:24:15.000 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:24:15.014 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1291 2024-09-12T13:24:15.032 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:24:15.989 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1290 2024-09-12T13:24:16.024 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:24:16.024 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:24:16.024 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:24:16.024 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:24:16.024 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:24:16.024 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:24:16.029 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:24:16.029 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:24:16.029 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:24:16.029 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:24:16.030 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:24:16.030 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:24:16.037 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1293 2024-09-12T13:24:16.039 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:24:16.071 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:24:16.111 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:24:16.111+0000 7f67dd000640 -1 rbd::mirror::image_replayer::snapshot::Replayer: 0x56055b858c00 handle_unlink_peer: failed to unlink local peer from remote image: (22) Invalid argument 2024-09-12T13:24:16.983 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1292 2024-09-12T13:24:16.997 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1294 2024-09-12T13:24:17.008 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:24:17.008 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:24:17.008 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:24:17.008 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:24:17.008 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:24:17.008 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:24:17.023 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:24:17.042 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:24:17.047 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${SNAP_PIDS[@]} 2024-09-12T13:24:17.047 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 70992 2024-09-12T13:24:17.081 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1297 2024-09-12T13:24:17.102 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:24:17.969 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1295 2024-09-12T13:24:18.008 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1296 2024-09-12T13:24:18.012 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:24:18.024 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:24:18.035 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:24:18.035 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:24:18.035 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:24:18.035 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:24:18.035 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:24:18.036 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:24:18.998 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1298 2024-09-12T13:24:19.023 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:24:19.024 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1299 2024-09-12T13:24:19.042 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${SNAP_PIDS[@]} 2024-09-12T13:24:19.043 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 71124 2024-09-12T13:24:19.048 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:24:19.074 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:24:19.074 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:24:19.075 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:24:19.075 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:24:19.075 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:24:19.075 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:24:20.045 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${SNAP_PIDS[@]} 2024-09-12T13:24:20.045 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 71256 2024-09-12T13:24:20.045 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${SNAP_PIDS[@]} 2024-09-12T13:24:20.045 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 71408 2024-09-12T13:24:20.104 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:24:20.104 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:24:20.104 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:24:20.104 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:24:20.104 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:24:20.104 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:24:21.007 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1300 2024-09-12T13:24:21.017 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:24:21.017 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:24:21.017 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:24:21.017 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:24:21.017 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:24:21.017 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:24:21.028 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${SNAP_PIDS[@]} 2024-09-12T13:24:21.028 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 71561 2024-09-12T13:24:21.033 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:24:22.026 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${SNAP_PIDS[@]} 2024-09-12T13:24:22.026 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 71714 2024-09-12T13:24:22.028 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1301 2024-09-12T13:24:22.038 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1302 2024-09-12T13:24:22.048 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:24:22.052 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${SNAP_PIDS[@]} 2024-09-12T13:24:22.052 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 71867 2024-09-12T13:24:22.055 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:24:24.038 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${SNAP_PIDS[@]} 2024-09-12T13:24:24.038 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 72040 2024-09-12T13:24:25.058 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${SNAP_PIDS[@]} 2024-09-12T13:24:25.059 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 72214 2024-09-12T13:24:25.059 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ (( RET != 0 )) 2024-09-12T13:24:25.059 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${WORKLOAD_PIDS[@]} 2024-09-12T13:24:25.059 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 70861 2024-09-12T13:26:52.796 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ ret=124 2024-09-12T13:26:52.797 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ (( ret != 124 )) 2024-09-12T13:26:52.797 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${WORKLOAD_PIDS[@]} 2024-09-12T13:26:52.797 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 70993 2024-09-12T13:26:53.818 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ ret=124 2024-09-12T13:26:53.818 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ (( ret != 124 )) 2024-09-12T13:26:53.819 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${WORKLOAD_PIDS[@]} 2024-09-12T13:26:53.819 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 71125 2024-09-12T13:26:54.823 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ ret=124 2024-09-12T13:26:54.823 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ (( ret != 124 )) 2024-09-12T13:26:54.823 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${WORKLOAD_PIDS[@]} 2024-09-12T13:26:54.824 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 71257 2024-09-12T13:26:55.655 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ ret=124 2024-09-12T13:26:55.655 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ (( ret != 124 )) 2024-09-12T13:26:55.655 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${WORKLOAD_PIDS[@]} 2024-09-12T13:26:55.655 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 71409 2024-09-12T13:26:56.702 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ ret=124 2024-09-12T13:26:56.702 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ (( ret != 124 )) 2024-09-12T13:26:56.702 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${WORKLOAD_PIDS[@]} 2024-09-12T13:26:56.702 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 71562 2024-09-12T13:26:57.690 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ ret=124 2024-09-12T13:26:57.690 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ (( ret != 124 )) 2024-09-12T13:26:57.690 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${WORKLOAD_PIDS[@]} 2024-09-12T13:26:57.690 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 71715 2024-09-12T13:26:58.633 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ ret=124 2024-09-12T13:26:58.633 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ (( ret != 124 )) 2024-09-12T13:26:58.634 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${WORKLOAD_PIDS[@]} 2024-09-12T13:26:58.634 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 71868 2024-09-12T13:26:59.686 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ ret=124 2024-09-12T13:26:59.686 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ (( ret != 124 )) 2024-09-12T13:26:59.686 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${WORKLOAD_PIDS[@]} 2024-09-12T13:26:59.686 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 72041 2024-09-12T13:27:00.647 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ ret=124 2024-09-12T13:27:00.647 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ (( ret != 124 )) 2024-09-12T13:27:00.648 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${WORKLOAD_PIDS[@]} 2024-09-12T13:27:00.648 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 72215 2024-09-12T13:27:01.813 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ ret=124 2024-09-12T13:27:01.814 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ (( ret != 124 )) 2024-09-12T13:27:01.814 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ (( RET != 0 )) 2024-09-12T13:27:01.814 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:27:01.815 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ COMPARE_PIDS+=($!) 2024-09-12T13:27:01.816 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:27:01.816 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ compare_demoted_promoted_image 1 2024-09-12T13:27:01.816 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local dev=/dev/rbd0 2024-09-12T13:27:01.816 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local img=image-primary1 2024-09-12T13:27:01.816 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mntpt=test-primary1 2024-09-12T13:27:01.816 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local demote_md5 promote_md5 2024-09-12T13:27:01.816 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo umount test-primary1 2024-09-12T13:27:01.816 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ COMPARE_PIDS+=($!) 2024-09-12T13:27:01.816 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:27:01.816 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ compare_demoted_promoted_image 2 2024-09-12T13:27:01.816 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local dev=/dev/rbd1 2024-09-12T13:27:01.817 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local img=image-primary2 2024-09-12T13:27:01.817 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mntpt=test-primary2 2024-09-12T13:27:01.817 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local demote_md5 promote_md5 2024-09-12T13:27:01.817 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo umount test-primary2 2024-09-12T13:27:01.817 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ COMPARE_PIDS+=($!) 2024-09-12T13:27:01.817 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:27:01.817 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ compare_demoted_promoted_image 3 2024-09-12T13:27:01.818 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local dev=/dev/rbd2 2024-09-12T13:27:01.818 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local img=image-primary3 2024-09-12T13:27:01.818 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mntpt=test-primary3 2024-09-12T13:27:01.818 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local demote_md5 promote_md5 2024-09-12T13:27:01.818 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo umount test-primary3 2024-09-12T13:27:01.819 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ COMPARE_PIDS+=($!) 2024-09-12T13:27:01.819 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:27:01.819 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ compare_demoted_promoted_image 4 2024-09-12T13:27:01.819 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local dev=/dev/rbd3 2024-09-12T13:27:01.819 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local img=image-primary4 2024-09-12T13:27:01.819 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mntpt=test-primary4 2024-09-12T13:27:01.819 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local demote_md5 promote_md5 2024-09-12T13:27:01.819 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo umount test-primary4 2024-09-12T13:27:01.819 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ COMPARE_PIDS+=($!) 2024-09-12T13:27:01.820 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ compare_demoted_promoted_image 5 2024-09-12T13:27:01.820 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:27:01.820 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local dev=/dev/rbd4 2024-09-12T13:27:01.820 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local img=image-primary5 2024-09-12T13:27:01.821 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mntpt=test-primary5 2024-09-12T13:27:01.821 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local demote_md5 promote_md5 2024-09-12T13:27:01.821 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo umount test-primary5 2024-09-12T13:27:01.821 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ COMPARE_PIDS+=($!) 2024-09-12T13:27:01.821 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:27:01.821 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ compare_demoted_promoted_image 6 2024-09-12T13:27:01.821 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local dev=/dev/rbd5 2024-09-12T13:27:01.821 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local img=image-primary6 2024-09-12T13:27:01.821 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mntpt=test-primary6 2024-09-12T13:27:01.821 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local demote_md5 promote_md5 2024-09-12T13:27:01.821 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo umount test-primary6 2024-09-12T13:27:01.821 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ compare_demoted_promoted_image 7 2024-09-12T13:27:01.822 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ COMPARE_PIDS+=($!) 2024-09-12T13:27:01.822 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:27:01.823 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local dev=/dev/rbd6 2024-09-12T13:27:01.823 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local img=image-primary7 2024-09-12T13:27:01.823 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mntpt=test-primary7 2024-09-12T13:27:01.823 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local demote_md5 promote_md5 2024-09-12T13:27:01.823 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo umount test-primary7 2024-09-12T13:27:01.823 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ COMPARE_PIDS+=($!) 2024-09-12T13:27:01.823 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:27:01.823 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ compare_demoted_promoted_image 8 2024-09-12T13:27:01.824 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local dev=/dev/rbd7 2024-09-12T13:27:01.825 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ compare_demoted_promoted_image 9 2024-09-12T13:27:01.825 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ COMPARE_PIDS+=($!) 2024-09-12T13:27:01.826 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local img=image-primary8 2024-09-12T13:27:01.826 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mntpt=test-primary8 2024-09-12T13:27:01.826 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local demote_md5 promote_md5 2024-09-12T13:27:01.826 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo umount test-primary8 2024-09-12T13:27:01.827 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:27:01.829 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ COMPARE_PIDS+=($!) 2024-09-12T13:27:01.829 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${COMPARE_PIDS[@]} 2024-09-12T13:27:01.829 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 78170 2024-09-12T13:27:01.830 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local dev=/dev/rbd8 2024-09-12T13:27:01.831 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local img=image-primary9 2024-09-12T13:27:01.831 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ compare_demoted_promoted_image 10 2024-09-12T13:27:01.831 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mntpt=test-primary9 2024-09-12T13:27:01.831 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local demote_md5 promote_md5 2024-09-12T13:27:01.831 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo umount test-primary9 2024-09-12T13:27:01.831 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local dev=/dev/rbd9 2024-09-12T13:27:01.831 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local img=image-primary10 2024-09-12T13:27:01.831 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mntpt=test-primary10 2024-09-12T13:27:01.831 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local demote_md5 promote_md5 2024-09-12T13:27:01.831 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo umount test-primary10 2024-09-12T13:27:02.548 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo md5sum /dev/rbd3 2024-09-12T13:27:02.549 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ awk '{print $1}' 2024-09-12T13:27:02.622 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo md5sum /dev/rbd5 2024-09-12T13:27:02.629 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ awk '{print $1}' 2024-09-12T13:27:02.657 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo md5sum /dev/rbd4 2024-09-12T13:27:02.661 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ awk '{print $1}' 2024-09-12T13:27:02.675 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo md5sum /dev/rbd9 2024-09-12T13:27:02.679 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ awk '{print $1}' 2024-09-12T13:27:02.681 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo md5sum /dev/rbd0 2024-09-12T13:27:02.684 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ awk '{print $1}' 2024-09-12T13:27:02.705 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo md5sum /dev/rbd7 2024-09-12T13:27:02.706 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ awk '{print $1}' 2024-09-12T13:27:02.727 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo md5sum /dev/rbd1 2024-09-12T13:27:02.728 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ awk '{print $1}' 2024-09-12T13:27:02.742 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo md5sum /dev/rbd6 2024-09-12T13:27:02.743 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ awk '{print $1}' 2024-09-12T13:27:02.825 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ awk '{print $1}' 2024-09-12T13:27:02.825 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo md5sum /dev/rbd2 2024-09-12T13:27:02.871 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo md5sum /dev/rbd8 2024-09-12T13:27:02.874 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ awk '{print $1}' 2024-09-12T13:27:52.072 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ demote_md5=e88d359832a0849246f40e77fa6e5870 2024-09-12T13:27:52.072 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo rbd --cluster cluster1 device unmap -t krbd mirror/image-primary1 2024-09-12T13:27:52.733 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ demote_image cluster1 mirror image-primary1 2024-09-12T13:27:52.733 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:27:52.733 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:27:52.733 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:27:52.733 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 mirror image demote mirror/image-primary1 2024-09-12T13:27:54.236 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Image demoted to non-primary 2024-09-12T13:27:54.328 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary1 up+unknown 2024-09-12T13:27:54.328 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:27:54.328 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:27:54.328 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:27:54.328 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:27:54.328 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:27:54.328 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:27:54.328 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:27:54.329 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:27:55.347 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary1 up+unknown '' '' 2024-09-12T13:27:55.347 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:27:55.347 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:27:55.347 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:27:55.347 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:27:55.347 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:27:55.347 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:27:55.347 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:27:55.349 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:27:55.349 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary1 2024-09-12T13:27:55.539 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary1: 2024-09-12T13:27:55.539 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 8bc720a8-a8a7-45cb-ab9a-5646a2cc4cc4 2024-09-12T13:27:55.539 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:27:55.539 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:27:55.539 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:27:55.539 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:27:47' 2024-09-12T13:27:55.540 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:27:55.553 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:27:55.553 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:27:55.553 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 2 2024-09-12T13:27:55.980 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ demote_md5=848c7a0b784bb075592ada3d3d2004b4 2024-09-12T13:27:55.980 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo rbd --cluster cluster1 device unmap -t krbd mirror/image-primary6 2024-09-12T13:27:56.170 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ demote_md5=6ede1b368bc41826fdc31adefbefd08f 2024-09-12T13:27:56.170 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo rbd --cluster cluster1 device unmap -t krbd mirror/image-primary4 2024-09-12T13:27:56.220 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ demote_md5=1a0482490c1e719730ddfb773845098b 2024-09-12T13:27:56.220 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo rbd --cluster cluster1 device unmap -t krbd mirror/image-primary5 2024-09-12T13:27:56.234 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ demote_image cluster1 mirror image-primary6 2024-09-12T13:27:56.234 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:27:56.234 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:27:56.234 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:27:56.234 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 mirror image demote mirror/image-primary6 2024-09-12T13:27:56.380 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ demote_image cluster1 mirror image-primary4 2024-09-12T13:27:56.380 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:27:56.380 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:27:56.380 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:27:56.380 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 mirror image demote mirror/image-primary4 2024-09-12T13:27:56.444 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ demote_image cluster1 mirror image-primary5 2024-09-12T13:27:56.445 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:27:56.445 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:27:56.445 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:27:56.445 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 mirror image demote mirror/image-primary5 2024-09-12T13:27:57.237 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Image demoted to non-primary 2024-09-12T13:27:57.255 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary4 up+unknown 2024-09-12T13:27:57.255 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:27:57.255 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:27:57.255 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:27:57.255 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:27:57.255 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:27:57.255 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:27:57.255 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:27:57.255 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:27:57.351 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ demote_md5=00861b00e10fad231d7844009fbeabd8 2024-09-12T13:27:57.351 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo rbd --cluster cluster1 device unmap -t krbd mirror/image-primary10 2024-09-12T13:27:57.568 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary1 up+unknown '' '' 2024-09-12T13:27:57.569 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:27:57.569 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:27:57.569 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:27:57.569 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:27:57.569 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:27:57.569 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:27:57.569 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:27:57.570 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:27:57.570 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary1 2024-09-12T13:27:57.600 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ demote_image cluster1 mirror image-primary10 2024-09-12T13:27:57.600 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:27:57.600 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:27:57.600 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:27:57.600 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 mirror image demote mirror/image-primary10 2024-09-12T13:27:57.671 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary1: 2024-09-12T13:27:57.671 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 8bc720a8-a8a7-45cb-ab9a-5646a2cc4cc4 2024-09-12T13:27:57.671 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:27:57.672 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:27:57.672 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:27:57.672 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:27:47' 2024-09-12T13:27:57.672 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:27:57.675 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:27:57.675 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:27:57.675 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 4 2024-09-12T13:27:57.788 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ demote_md5=767f6012e29e50caa1e86b4e499a2480 2024-09-12T13:27:57.789 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo rbd --cluster cluster1 device unmap -t krbd mirror/image-primary3 2024-09-12T13:27:57.961 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ demote_image cluster1 mirror image-primary3 2024-09-12T13:27:57.961 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:27:57.961 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:27:57.961 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:27:57.961 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 mirror image demote mirror/image-primary3 2024-09-12T13:27:58.008 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Image demoted to non-primary 2024-09-12T13:27:58.011 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Image demoted to non-primary 2024-09-12T13:27:58.022 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ demote_md5=b4bf42224f4523cb9a457ba142738ff5 2024-09-12T13:27:58.022 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo rbd --cluster cluster1 device unmap -t krbd mirror/image-primary7 2024-09-12T13:27:58.026 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary6 up+unknown 2024-09-12T13:27:58.026 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:27:58.026 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:27:58.026 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:27:58.026 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:27:58.026 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:27:58.026 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:27:58.027 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:27:58.027 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:27:58.027 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary5 up+unknown 2024-09-12T13:27:58.027 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:27:58.027 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:27:58.027 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:27:58.027 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:27:58.027 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:27:58.027 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:27:58.027 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:27:58.027 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:27:58.164 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ demote_md5=90feddaf5dcc21646e6bfa8fc08939f0 2024-09-12T13:27:58.165 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo rbd --cluster cluster1 device unmap -t krbd mirror/image-primary9 2024-09-12T13:27:58.175 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ demote_image cluster1 mirror image-primary7 2024-09-12T13:27:58.175 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:27:58.175 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:27:58.175 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:27:58.175 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 mirror image demote mirror/image-primary7 2024-09-12T13:27:58.291 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary4 up+unknown '' '' 2024-09-12T13:27:58.293 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:27:58.294 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:27:58.294 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:27:58.294 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:27:58.294 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:27:58.294 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:27:58.294 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:27:58.295 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:27:58.295 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary4 2024-09-12T13:27:58.394 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ demote_image cluster1 mirror image-primary9 2024-09-12T13:27:58.395 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:27:58.395 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:27:58.395 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:27:58.395 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 mirror image demote mirror/image-primary9 2024-09-12T13:27:58.657 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary4: 2024-09-12T13:27:58.657 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: ce400cdd-ba29-4924-9381-6e78dcb9b581 2024-09-12T13:27:58.657 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:27:58.657 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:27:58.657 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:27:58.658 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:27:47' 2024-09-12T13:27:58.659 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:27:58.666 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:27:58.666 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:27:58.666 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 2 2024-09-12T13:27:58.995 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ demote_md5=9e6bd4a425594b0e48c8ce7d86200219 2024-09-12T13:27:58.995 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo rbd --cluster cluster1 device unmap -t krbd mirror/image-primary8 2024-09-12T13:27:59.017 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Image demoted to non-primary 2024-09-12T13:27:59.027 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary3 up+unknown 2024-09-12T13:27:59.027 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:27:59.027 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:27:59.028 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:27:59.028 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:27:59.028 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:27:59.028 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:27:59.028 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:27:59.028 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:27:59.029 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary5 up+unknown '' '' 2024-09-12T13:27:59.029 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:27:59.029 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:27:59.029 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:27:59.029 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:27:59.029 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:27:59.030 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:27:59.030 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:27:59.031 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary6 up+unknown '' '' 2024-09-12T13:27:59.031 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:27:59.031 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:27:59.031 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:27:59.031 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:27:59.031 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:27:59.031 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:27:59.031 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:27:59.031 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:27:59.031 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary5 2024-09-12T13:27:59.031 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:27:59.031 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary6 2024-09-12T13:27:59.100 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary5: 2024-09-12T13:27:59.100 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 1d3a6d29-f46e-4dfe-ba1d-2e1bda9e7bc4 2024-09-12T13:27:59.100 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:27:59.100 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:27:59.100 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:27:59.101 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:27:47' 2024-09-12T13:27:59.101 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:27:59.103 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:27:59.103 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:27:59.103 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 2 2024-09-12T13:27:59.106 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary6: 2024-09-12T13:27:59.106 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: b580eb06-ffeb-44e0-8354-b02afae3f6a4 2024-09-12T13:27:59.106 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:27:59.106 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:27:59.106 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:27:59.106 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:27:47' 2024-09-12T13:27:59.106 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:27:59.108 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:27:59.108 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:27:59.109 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 2 2024-09-12T13:27:59.171 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ demote_image cluster1 mirror image-primary8 2024-09-12T13:27:59.171 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:27:59.172 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:27:59.172 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:27:59.172 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 mirror image demote mirror/image-primary8 2024-09-12T13:27:59.269 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ demote_md5=605d8e891318ba820dc2f8e0910359fd 2024-09-12T13:27:59.269 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo rbd --cluster cluster1 device unmap -t krbd mirror/image-primary2 2024-09-12T13:27:59.449 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ demote_image cluster1 mirror image-primary2 2024-09-12T13:27:59.449 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:27:59.450 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:27:59.450 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:27:59.450 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 mirror image demote mirror/image-primary2 2024-09-12T13:28:00.008 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Image demoted to non-primary 2024-09-12T13:28:00.025 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary9 up+unknown 2024-09-12T13:28:00.025 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:28:00.025 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:28:00.025 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:28:00.026 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Image demoted to non-primary 2024-09-12T13:28:00.026 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:28:00.026 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:28:00.026 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:28:00.026 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:28:00.026 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:28:00.029 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary3 up+unknown '' '' 2024-09-12T13:28:00.029 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:28:00.030 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:28:00.030 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:28:00.030 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:28:00.030 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:28:00.030 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:28:00.030 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:28:00.030 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:28:00.030 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary3 2024-09-12T13:28:00.045 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary10 up+unknown 2024-09-12T13:28:00.045 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:28:00.045 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:28:00.045 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:28:00.045 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:28:00.045 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:28:00.045 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:28:00.045 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:28:00.045 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:28:00.053 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Image demoted to non-primary 2024-09-12T13:28:00.064 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary2 up+unknown 2024-09-12T13:28:00.064 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:28:00.065 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:28:00.065 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:28:00.065 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:28:00.065 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:28:00.065 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:28:00.065 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:28:00.065 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:28:00.104 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary3: 2024-09-12T13:28:00.104 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 1d6661c9-3d05-4fd7-9ee6-aad6901ddd33 2024-09-12T13:28:00.104 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:28:00.104 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:28:00.106 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:28:00.106 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:27:47' 2024-09-12T13:28:00.106 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:28:00.107 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:28:00.107 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:28:00.107 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 2 2024-09-12T13:28:00.674 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary4 up+unknown '' '' 2024-09-12T13:28:00.674 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:28:00.674 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:28:00.674 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:28:00.674 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:28:00.674 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:28:00.674 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:28:00.674 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:28:00.674 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:28:00.675 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary4 2024-09-12T13:28:00.748 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary4: 2024-09-12T13:28:00.748 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: ce400cdd-ba29-4924-9381-6e78dcb9b581 2024-09-12T13:28:00.748 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:28:00.748 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:28:00.748 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:28:00.748 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:27:47' 2024-09-12T13:28:00.748 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:28:00.751 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:28:00.751 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:28:00.751 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 4 2024-09-12T13:28:01.028 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary9 up+unknown '' '' 2024-09-12T13:28:01.029 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:28:01.029 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:28:01.029 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:28:01.029 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:28:01.029 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:28:01.029 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:28:01.031 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Image demoted to non-primary 2024-09-12T13:28:01.033 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:28:01.036 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:28:01.036 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary9 2024-09-12T13:28:01.040 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Image demoted to non-primary 2024-09-12T13:28:01.053 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary10 up+unknown '' '' 2024-09-12T13:28:01.055 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary7 up+unknown 2024-09-12T13:28:01.055 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:28:01.055 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:28:01.055 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:28:01.055 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:28:01.056 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:28:01.056 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:28:01.056 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:28:01.056 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:28:01.061 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:28:01.061 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:28:01.061 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:28:01.061 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:28:01.061 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:28:01.061 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:28:01.061 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:28:01.062 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:28:01.063 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary10 2024-09-12T13:28:01.075 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary2 up+unknown '' '' 2024-09-12T13:28:01.075 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:28:01.075 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:28:01.075 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:28:01.075 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:28:01.076 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:28:01.076 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:28:01.076 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:28:01.077 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:28:01.077 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary2 2024-09-12T13:28:01.106 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary5 up+unknown '' '' 2024-09-12T13:28:01.106 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:28:01.106 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:28:01.107 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:28:01.107 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:28:01.107 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:28:01.107 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:28:01.107 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:28:01.108 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:28:01.108 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary5 2024-09-12T13:28:01.108 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary8 up+unknown 2024-09-12T13:28:01.108 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:28:01.108 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:28:01.108 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:28:01.108 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:28:01.108 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:28:01.109 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:28:01.109 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:28:01.109 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:28:01.116 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary6 up+unknown '' '' 2024-09-12T13:28:01.116 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:28:01.116 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:28:01.116 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:28:01.116 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:28:01.116 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:28:01.116 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:28:01.116 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:28:01.117 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:28:01.117 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary6 2024-09-12T13:28:01.258 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary9: 2024-09-12T13:28:01.258 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: cabafe0d-591b-466b-8d57-50e60dd6fa48 2024-09-12T13:28:01.258 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:28:01.259 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:28:01.259 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:28:01.259 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:27:47' 2024-09-12T13:28:01.259 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:28:01.263 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:28:01.263 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:28:01.263 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 2 2024-09-12T13:28:01.282 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary5: 2024-09-12T13:28:01.282 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 1d3a6d29-f46e-4dfe-ba1d-2e1bda9e7bc4 2024-09-12T13:28:01.282 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:28:01.282 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:28:01.282 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:28:01.283 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:27:47' 2024-09-12T13:28:01.283 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:28:01.283 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary10: 2024-09-12T13:28:01.283 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: a020c6a3-79f5-4f05-82bf-2803d0febb5c 2024-09-12T13:28:01.283 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:28:01.283 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:28:01.284 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:28:01.284 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:27:47' 2024-09-12T13:28:01.284 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:28:01.287 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:28:01.287 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:28:01.287 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 2 2024-09-12T13:28:01.288 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:28:01.288 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:28:01.288 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 4 2024-09-12T13:28:01.302 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary6: 2024-09-12T13:28:01.302 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: b580eb06-ffeb-44e0-8354-b02afae3f6a4 2024-09-12T13:28:01.303 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:28:01.303 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:28:01.303 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:28:01.303 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:27:47' 2024-09-12T13:28:01.303 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:28:01.309 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:28:01.309 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:28:01.309 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 4 2024-09-12T13:28:01.312 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary2: 2024-09-12T13:28:01.312 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: b9571de3-33f9-41ad-aae9-a4841d1fab32 2024-09-12T13:28:01.312 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:28:01.312 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:28:01.312 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:28:01.312 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:27:47' 2024-09-12T13:28:01.313 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:28:01.316 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:28:01.316 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:28:01.317 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 2 2024-09-12T13:28:01.678 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary1 up+unknown '' '' 2024-09-12T13:28:01.678 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:28:01.678 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:28:01.678 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:28:01.678 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:28:01.678 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:28:01.678 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:28:01.678 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:28:01.679 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:28:01.679 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary1 2024-09-12T13:28:01.755 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary1: 2024-09-12T13:28:01.755 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 8bc720a8-a8a7-45cb-ab9a-5646a2cc4cc4 2024-09-12T13:28:01.755 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:28:01.755 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:28:01.755 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:28:01.756 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:27:47' 2024-09-12T13:28:01.756 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:28:01.758 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:28:01.758 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:28:01.758 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 8 2024-09-12T13:28:02.059 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary7 up+unknown '' '' 2024-09-12T13:28:02.059 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:28:02.059 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:28:02.059 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:28:02.059 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:28:02.059 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:28:02.059 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:28:02.059 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:28:02.060 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:28:02.060 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary7 2024-09-12T13:28:02.109 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary3 up+unknown '' '' 2024-09-12T13:28:02.109 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:28:02.110 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:28:02.110 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:28:02.110 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:28:02.110 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:28:02.110 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:28:02.110 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:28:02.110 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:28:02.110 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary3 2024-09-12T13:28:02.114 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary8 up+unknown '' '' 2024-09-12T13:28:02.114 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:28:02.114 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:28:02.114 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:28:02.114 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:28:02.114 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:28:02.114 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:28:02.114 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:28:02.115 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:28:02.115 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary8 2024-09-12T13:28:02.164 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary7: 2024-09-12T13:28:02.164 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 43088135-2ee1-48a6-8f76-ae96116f16b1 2024-09-12T13:28:02.164 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:28:02.164 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:28:02.164 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:28:02.164 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:27:47' 2024-09-12T13:28:02.164 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:28:02.166 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:28:02.166 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:28:02.166 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 2 2024-09-12T13:28:02.214 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary3: 2024-09-12T13:28:02.214 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 1d6661c9-3d05-4fd7-9ee6-aad6901ddd33 2024-09-12T13:28:02.214 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:28:02.214 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:28:02.214 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:28:02.215 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:27:47' 2024-09-12T13:28:02.215 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:28:02.221 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:28:02.221 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:28:02.221 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 4 2024-09-12T13:28:02.227 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary8: 2024-09-12T13:28:02.227 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: e5469b72-dbc8-4a77-9c96-ab995f3cc6f1 2024-09-12T13:28:02.227 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:28:02.227 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:28:02.227 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:28:02.227 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:27:47' 2024-09-12T13:28:02.227 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:28:02.234 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:28:02.234 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:28:02.234 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 2 2024-09-12T13:28:03.266 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary9 up+unknown '' '' 2024-09-12T13:28:03.267 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:28:03.267 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:28:03.267 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:28:03.267 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:28:03.267 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:28:03.267 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:28:03.267 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:28:03.268 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:28:03.268 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary9 2024-09-12T13:28:03.291 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary10 up+unknown '' '' 2024-09-12T13:28:03.291 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:28:03.291 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:28:03.291 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:28:03.291 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:28:03.291 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:28:03.291 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:28:03.291 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:28:03.292 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:28:03.292 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary10 2024-09-12T13:28:03.319 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary2 up+unknown '' '' 2024-09-12T13:28:03.319 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:28:03.320 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:28:03.320 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:28:03.320 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:28:03.320 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:28:03.320 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:28:03.320 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:28:03.320 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:28:03.320 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary2 2024-09-12T13:28:03.355 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary9: 2024-09-12T13:28:03.355 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: cabafe0d-591b-466b-8d57-50e60dd6fa48 2024-09-12T13:28:03.355 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:28:03.355 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:28:03.355 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:28:03.355 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:27:47' 2024-09-12T13:28:03.355 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:28:03.358 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:28:03.358 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:28:03.358 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 4 2024-09-12T13:28:03.372 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary10: 2024-09-12T13:28:03.372 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: a020c6a3-79f5-4f05-82bf-2803d0febb5c 2024-09-12T13:28:03.373 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:28:03.373 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:28:03.373 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:28:03.373 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:27:47' 2024-09-12T13:28:03.373 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:28:03.375 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:28:03.375 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:28:03.376 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 4 2024-09-12T13:28:03.399 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary2: 2024-09-12T13:28:03.400 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: b9571de3-33f9-41ad-aae9-a4841d1fab32 2024-09-12T13:28:03.400 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:28:03.400 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:28:03.400 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:28:03.400 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:27:47' 2024-09-12T13:28:03.401 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:28:03.402 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:28:03.402 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:28:03.402 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 4 2024-09-12T13:28:04.169 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary7 up+unknown '' '' 2024-09-12T13:28:04.169 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:28:04.169 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:28:04.169 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:28:04.169 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:28:04.170 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:28:04.170 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:28:04.170 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:28:04.170 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:28:04.170 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary7 2024-09-12T13:28:04.237 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary8 up+unknown '' '' 2024-09-12T13:28:04.237 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:28:04.237 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:28:04.237 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:28:04.237 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:28:04.238 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:28:04.238 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:28:04.238 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:28:04.238 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:28:04.238 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary8 2024-09-12T13:28:04.252 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary7: 2024-09-12T13:28:04.252 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 43088135-2ee1-48a6-8f76-ae96116f16b1 2024-09-12T13:28:04.252 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:28:04.252 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:28:04.253 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:28:04.253 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:27:47' 2024-09-12T13:28:04.253 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:28:04.255 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:28:04.255 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:28:04.255 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 4 2024-09-12T13:28:04.335 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary8: 2024-09-12T13:28:04.335 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: e5469b72-dbc8-4a77-9c96-ab995f3cc6f1 2024-09-12T13:28:04.335 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:28:04.335 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:28:04.335 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:28:04.336 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:27:47' 2024-09-12T13:28:04.336 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:28:04.338 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:28:04.338 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:28:04.338 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 4 2024-09-12T13:28:04.753 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary4 up+unknown '' '' 2024-09-12T13:28:04.753 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:28:04.753 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:28:04.754 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:28:04.754 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:28:04.754 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:28:04.754 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:28:04.754 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:28:04.754 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:28:04.754 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary4 2024-09-12T13:28:04.830 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary4: 2024-09-12T13:28:04.830 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: ce400cdd-ba29-4924-9381-6e78dcb9b581 2024-09-12T13:28:04.831 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:28:04.831 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:28:04.831 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:28:04.831 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:27:47' 2024-09-12T13:28:04.831 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:28:04.833 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:28:04.833 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:28:04.833 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 8 2024-09-12T13:28:05.293 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary5 up+unknown '' '' 2024-09-12T13:28:05.293 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:28:05.293 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:28:05.293 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:28:05.293 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:28:05.293 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:28:05.293 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:28:05.293 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:28:05.294 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:28:05.294 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary5 2024-09-12T13:28:05.313 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary6 up+unknown '' '' 2024-09-12T13:28:05.314 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:28:05.314 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:28:05.314 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:28:05.314 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:28:05.314 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:28:05.315 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:28:05.315 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:28:05.315 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:28:05.315 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary6 2024-09-12T13:28:05.377 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary5: 2024-09-12T13:28:05.377 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 1d3a6d29-f46e-4dfe-ba1d-2e1bda9e7bc4 2024-09-12T13:28:05.377 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:28:05.378 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:28:05.378 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:28:05.378 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:27:47' 2024-09-12T13:28:05.378 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:28:05.380 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:28:05.380 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:28:05.380 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 8 2024-09-12T13:28:05.395 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary6: 2024-09-12T13:28:05.395 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: b580eb06-ffeb-44e0-8354-b02afae3f6a4 2024-09-12T13:28:05.395 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:28:05.395 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:28:05.395 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:28:05.395 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:27:47' 2024-09-12T13:28:05.395 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:28:05.397 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:28:05.398 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:28:05.398 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 8 2024-09-12T13:28:06.224 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary3 up+unknown '' '' 2024-09-12T13:28:06.224 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:28:06.224 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:28:06.224 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:28:06.224 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:28:06.225 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:28:06.225 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:28:06.225 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:28:06.225 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:28:06.225 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary3 2024-09-12T13:28:06.301 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary3: 2024-09-12T13:28:06.302 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 1d6661c9-3d05-4fd7-9ee6-aad6901ddd33 2024-09-12T13:28:06.302 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:28:06.302 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:28:06.302 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:28:06.302 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:27:47' 2024-09-12T13:28:06.302 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:28:06.304 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:28:06.305 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:28:06.305 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 8 2024-09-12T13:28:07.361 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary9 up+unknown '' '' 2024-09-12T13:28:07.361 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:28:07.361 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:28:07.361 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:28:07.361 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:28:07.361 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:28:07.361 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:28:07.361 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:28:07.362 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:28:07.362 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary9 2024-09-12T13:28:07.378 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary10 up+unknown '' '' 2024-09-12T13:28:07.378 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:28:07.378 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:28:07.378 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:28:07.378 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:28:07.379 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:28:07.379 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:28:07.379 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:28:07.379 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:28:07.379 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary10 2024-09-12T13:28:07.405 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary2 up+unknown '' '' 2024-09-12T13:28:07.405 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:28:07.405 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:28:07.405 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:28:07.406 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:28:07.406 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:28:07.406 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:28:07.406 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:28:07.406 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:28:07.407 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary2 2024-09-12T13:28:07.446 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary9: 2024-09-12T13:28:07.447 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: cabafe0d-591b-466b-8d57-50e60dd6fa48 2024-09-12T13:28:07.447 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:28:07.447 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:28:07.447 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:28:07.447 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:27:47' 2024-09-12T13:28:07.447 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:28:07.449 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:28:07.449 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:28:07.450 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 8 2024-09-12T13:28:07.457 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary10: 2024-09-12T13:28:07.457 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: a020c6a3-79f5-4f05-82bf-2803d0febb5c 2024-09-12T13:28:07.458 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:28:07.458 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:28:07.458 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:28:07.458 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:27:47' 2024-09-12T13:28:07.458 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:28:07.460 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:28:07.460 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:28:07.460 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 8 2024-09-12T13:28:07.489 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary2: 2024-09-12T13:28:07.489 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: b9571de3-33f9-41ad-aae9-a4841d1fab32 2024-09-12T13:28:07.489 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:28:07.489 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:28:07.489 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:28:07.489 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:27:47' 2024-09-12T13:28:07.489 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:28:07.492 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:28:07.492 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:28:07.492 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 8 2024-09-12T13:28:08.258 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary7 up+unknown '' '' 2024-09-12T13:28:08.258 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:28:08.258 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:28:08.258 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:28:08.258 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:28:08.258 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:28:08.258 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:28:08.259 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:28:08.259 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:28:08.259 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary7 2024-09-12T13:28:08.340 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary8 up+unknown '' '' 2024-09-12T13:28:08.341 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:28:08.341 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:28:08.341 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:28:08.341 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:28:08.341 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:28:08.341 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:28:08.341 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:28:08.341 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:28:08.341 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary8 2024-09-12T13:28:08.352 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary7: 2024-09-12T13:28:08.352 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 43088135-2ee1-48a6-8f76-ae96116f16b1 2024-09-12T13:28:08.352 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:28:08.352 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:28:08.353 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:28:08.353 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:27:47' 2024-09-12T13:28:08.353 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:28:08.355 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:28:08.356 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:28:08.356 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 8 2024-09-12T13:28:08.419 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary8: 2024-09-12T13:28:08.419 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: e5469b72-dbc8-4a77-9c96-ab995f3cc6f1 2024-09-12T13:28:08.419 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:28:08.419 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:28:08.419 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:28:08.419 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:27:47' 2024-09-12T13:28:08.419 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:28:08.422 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:28:08.422 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:28:08.422 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 8 2024-09-12T13:28:09.760 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary1 up+unknown '' '' 2024-09-12T13:28:09.761 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:28:09.761 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:28:09.761 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:28:09.761 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:28:09.761 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:28:09.761 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:28:09.761 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:28:09.761 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:28:09.761 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary1 2024-09-12T13:28:09.847 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary1: 2024-09-12T13:28:09.847 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 8bc720a8-a8a7-45cb-ab9a-5646a2cc4cc4 2024-09-12T13:28:09.847 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:28:09.847 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:28:09.848 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:28:09.848 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:27:47' 2024-09-12T13:28:09.848 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:28:09.850 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:28:09.850 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:28:09.851 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 8 2024-09-12T13:28:12.836 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary4 up+unknown '' '' 2024-09-12T13:28:12.836 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:28:12.836 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:28:12.837 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:28:12.837 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:28:12.837 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:28:12.837 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:28:12.837 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:28:12.837 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:28:12.837 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary4 2024-09-12T13:28:12.927 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary4: 2024-09-12T13:28:12.927 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: ce400cdd-ba29-4924-9381-6e78dcb9b581 2024-09-12T13:28:12.927 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:28:12.927 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:28:12.927 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:28:12.927 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:27:47' 2024-09-12T13:28:12.927 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:28:12.930 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:28:12.930 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:28:12.930 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 8 2024-09-12T13:28:13.383 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary5 up+unknown '' '' 2024-09-12T13:28:13.383 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:28:13.383 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:28:13.383 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:28:13.384 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:28:13.384 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:28:13.384 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:28:13.384 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:28:13.384 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:28:13.384 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary5 2024-09-12T13:28:13.400 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary6 up+unknown '' '' 2024-09-12T13:28:13.400 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:28:13.400 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:28:13.400 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:28:13.400 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:28:13.400 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:28:13.401 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:28:13.401 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:28:13.401 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:28:13.401 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary6 2024-09-12T13:28:13.462 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary5: 2024-09-12T13:28:13.462 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 1d3a6d29-f46e-4dfe-ba1d-2e1bda9e7bc4 2024-09-12T13:28:13.462 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:28:13.462 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:28:13.462 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:28:13.462 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:27:47' 2024-09-12T13:28:13.462 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:28:13.465 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:28:13.465 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:28:13.465 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 8 2024-09-12T13:28:13.480 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary6: 2024-09-12T13:28:13.480 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: b580eb06-ffeb-44e0-8354-b02afae3f6a4 2024-09-12T13:28:13.480 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:28:13.480 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:28:13.481 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:28:13.481 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:27:47' 2024-09-12T13:28:13.481 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:28:13.482 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:28:13.482 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:28:13.482 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 8 2024-09-12T13:28:14.307 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary3 up+unknown '' '' 2024-09-12T13:28:14.307 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:28:14.307 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:28:14.307 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:28:14.307 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:28:14.307 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:28:14.308 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:28:14.308 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:28:14.308 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:28:14.308 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary3 2024-09-12T13:28:14.393 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary3: 2024-09-12T13:28:14.393 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 1d6661c9-3d05-4fd7-9ee6-aad6901ddd33 2024-09-12T13:28:14.393 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:28:14.393 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:28:14.393 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:28:14.393 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:27:47' 2024-09-12T13:28:14.393 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:28:14.395 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:28:14.396 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:28:14.396 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 8 2024-09-12T13:28:15.452 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary9 up+unknown '' '' 2024-09-12T13:28:15.453 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:28:15.453 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:28:15.453 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:28:15.453 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:28:15.453 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:28:15.453 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:28:15.453 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:28:15.454 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:28:15.454 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary9 2024-09-12T13:28:15.463 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary10 up+unknown '' '' 2024-09-12T13:28:15.463 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:28:15.463 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:28:15.463 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:28:15.463 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:28:15.463 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:28:15.463 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:28:15.463 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:28:15.464 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:28:15.464 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary10 2024-09-12T13:28:15.495 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary2 up+unknown '' '' 2024-09-12T13:28:15.495 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:28:15.495 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:28:15.495 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:28:15.496 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:28:15.496 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:28:15.496 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:28:15.496 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:28:15.496 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:28:15.497 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary2 2024-09-12T13:28:15.539 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary9: 2024-09-12T13:28:15.539 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: cabafe0d-591b-466b-8d57-50e60dd6fa48 2024-09-12T13:28:15.540 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:28:15.540 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:28:15.540 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:28:15.540 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:27:47' 2024-09-12T13:28:15.540 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:28:15.543 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:28:15.543 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:28:15.543 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 8 2024-09-12T13:28:15.548 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary10: 2024-09-12T13:28:15.548 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: a020c6a3-79f5-4f05-82bf-2803d0febb5c 2024-09-12T13:28:15.548 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:28:15.548 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:28:15.549 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:28:15.549 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:27:47' 2024-09-12T13:28:15.549 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:28:15.550 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:28:15.550 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:28:15.550 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 8 2024-09-12T13:28:15.574 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary2: 2024-09-12T13:28:15.574 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: b9571de3-33f9-41ad-aae9-a4841d1fab32 2024-09-12T13:28:15.574 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:28:15.574 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:28:15.574 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:28:15.574 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:27:47' 2024-09-12T13:28:15.574 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:28:15.576 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:28:15.577 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:28:15.577 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 8 2024-09-12T13:28:16.358 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary7 up+unknown '' '' 2024-09-12T13:28:16.358 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:28:16.358 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:28:16.359 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:28:16.359 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:28:16.359 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:28:16.359 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:28:16.359 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:28:16.359 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:28:16.359 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary7 2024-09-12T13:28:16.424 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary8 up+unknown '' '' 2024-09-12T13:28:16.424 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:28:16.424 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:28:16.425 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:28:16.425 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:28:16.425 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:28:16.425 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:28:16.425 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:28:16.425 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:28:16.425 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary8 2024-09-12T13:28:16.437 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary7: 2024-09-12T13:28:16.438 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 43088135-2ee1-48a6-8f76-ae96116f16b1 2024-09-12T13:28:16.438 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:28:16.438 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:28:16.438 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:28:16.438 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:27:47' 2024-09-12T13:28:16.438 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:28:16.441 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:28:16.441 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:28:16.441 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 8 2024-09-12T13:28:16.501 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary8: 2024-09-12T13:28:16.501 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: e5469b72-dbc8-4a77-9c96-ab995f3cc6f1 2024-09-12T13:28:16.501 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:28:16.501 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:28:16.501 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:28:16.501 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:27:47' 2024-09-12T13:28:16.501 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:28:16.503 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:28:16.504 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:28:16.504 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 8 2024-09-12T13:28:17.853 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary1 up+unknown '' '' 2024-09-12T13:28:17.853 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:28:17.853 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:28:17.853 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:28:17.854 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:28:17.854 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:28:17.854 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:28:17.854 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:28:17.854 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:28:17.854 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary1 2024-09-12T13:28:17.944 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary1: 2024-09-12T13:28:17.944 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 8bc720a8-a8a7-45cb-ab9a-5646a2cc4cc4 2024-09-12T13:28:17.944 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+unknown 2024-09-12T13:28:17.944 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: remote image is not primary 2024-09-12T13:28:17.944 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:28:17.944 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:28:17' 2024-09-12T13:28:17.944 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:28:17.946 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+unknown 2024-09-12T13:28:17.946 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:28:17.949 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: remote image is not primary 2024-09-12T13:28:17.949 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:28:17.950 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:28:17.950 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:28:17.952 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:28:17.952 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:28:17.955 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:28:17.955 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:28:17.956 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:28:17.960 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:28:17' 2024-09-12T13:28:17.960 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary1 up+unknown '2024-09-12 13:28:17' 2024-09-12T13:28:17.960 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:28:17.961 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:28:17.961 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:28:17.961 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:28:17.961 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:28:17' 2024-09-12T13:28:17.961 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:28:17.961 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:28:17.961 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-09-12T13:28:18.073 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041934232mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary18bc720a8-a8a7-45cb-ab9a-5646a2cc4cc4up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:28:17image-primary10a020c6a3-79f5-4f05-82bf-2803d0febb5cup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:28:17image-primary2b9571de3-33f9-41ad-aae9-a4841d1fab32up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:28:17image-primary31d6661c9-3d05-4fd7-9ee6-aad6901ddd33up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:28:17image-primary4ce400cdd-ba29-4924-9381-6e78dcb9b581up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:28:17image-primary51d3a6d29-f46e-4dfe-ba1d-2e1bda9e7bc4up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:28:17image-primary6b580eb06-ffeb-44e0-8354-b02afae3f6a4up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:28:17image-primary743088135-2ee1-48a6-8f76-ae96116f16b1up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:28:17image-primary8e5469b72-dbc8-4a77-9c96-ab995f3cc6f1up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:28:17image-primary9cabafe0d-591b-466b-8d57-50e60dd6fa48up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:28:17' 2024-09-12T13:28:18.073 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:28:18.074 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary1'\'']/last_update' 2024-09-12T13:28:18.106 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:28:17' 2024-09-12T13:28:18.107 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary1'\'']/state' 2024-09-12T13:28:18.110 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+unknown 2024-09-12T13:28:18.111 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:28:18.111 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+unknown 2024-09-12T13:28:18.113 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:28:18.114 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:28:18.114 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:28:18.114 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster2 mirror image-primary1 up+unknown 2024-09-12T13:28:18.114 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:28:18.114 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:28:18.115 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:28:18.115 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:28:18.115 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:28:18.115 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:28:18.115 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:28:18.115 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:28:19.117 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster2 mirror image-primary1 up+unknown '' '' 2024-09-12T13:28:19.117 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:28:19.117 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:28:19.117 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:28:19.117 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:28:19.117 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:28:19.117 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:28:19.117 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:28:19.117 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:28:19.118 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster2 mirror image status mirror/image-primary1 2024-09-12T13:28:19.186 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary1: 2024-09-12T13:28:19.186 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 8bc720a8-a8a7-45cb-ab9a-5646a2cc4cc4 2024-09-12T13:28:19.186 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+unknown 2024-09-12T13:28:19.186 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: remote image is not primary 2024-09-12T13:28:19.186 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:28:19.186 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:28:17' 2024-09-12T13:28:19.187 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:28:19.189 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+unknown 2024-09-12T13:28:19.189 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:28:19.191 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: remote image is not primary 2024-09-12T13:28:19.192 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:28:19.192 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:28:19.193 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:28:19.195 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:28:19.195 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:28:19.198 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:28:19.198 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:28:19.199 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:28:19.204 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:28:17' 2024-09-12T13:28:19.205 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster2 mirror image-primary1 up+unknown '2024-09-12 13:28:17' 2024-09-12T13:28:19.205 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:28:19.205 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:28:19.205 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:28:19.205 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:28:19.205 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:28:17' 2024-09-12T13:28:19.205 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:28:19.205 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:28:19.205 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2024-09-12T13:28:19.303 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041924231mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary18bc720a8-a8a7-45cb-ab9a-5646a2cc4cc4up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:28:17image-primary10a020c6a3-79f5-4f05-82bf-2803d0febb5cup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:28:17image-primary2b9571de3-33f9-41ad-aae9-a4841d1fab32up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:28:17image-primary31d6661c9-3d05-4fd7-9ee6-aad6901ddd33up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:28:17image-primary4ce400cdd-ba29-4924-9381-6e78dcb9b581up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:28:17image-primary51d3a6d29-f46e-4dfe-ba1d-2e1bda9e7bc4up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:28:17image-primary6b580eb06-ffeb-44e0-8354-b02afae3f6a4up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:28:17image-primary743088135-2ee1-48a6-8f76-ae96116f16b1up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:28:17image-primary8e5469b72-dbc8-4a77-9c96-ab995f3cc6f1up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:28:17image-primary9cabafe0d-591b-466b-8d57-50e60dd6fa48up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:28:17' 2024-09-12T13:28:19.303 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:28:19.304 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary1'\'']/last_update' 2024-09-12T13:28:19.307 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:28:17' 2024-09-12T13:28:19.308 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary1'\'']/state' 2024-09-12T13:28:19.311 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+unknown 2024-09-12T13:28:19.312 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:28:19.312 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+unknown 2024-09-12T13:28:19.314 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:28:19.315 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:28:19.315 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:28:19.315 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ promote_image cluster2 mirror image-primary1 2024-09-12T13:28:19.315 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:28:19.316 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:28:19.316 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:28:19.316 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local force= 2024-09-12T13:28:19.316 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 mirror image promote mirror/image-primary1 2024-09-12T13:28:20.135 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Image promoted to primary 2024-09-12T13:28:20.145 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \n\b\d ]] 2024-09-12T13:28:20.145 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \k\r\b\d ]] 2024-09-12T13:28:20.146 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo rbd --cluster cluster2 device map -t krbd mirror/image-primary1 2024-09-12T13:28:20.273 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ dev=/dev/rbd0 2024-09-12T13:28:20.274 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo md5sum /dev/rbd0 2024-09-12T13:28:20.274 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ awk '{print $1}' 2024-09-12T13:28:20.932 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary4 up+unknown '' '' 2024-09-12T13:28:20.933 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:28:20.933 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:28:20.933 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:28:20.933 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:28:20.933 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:28:20.933 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:28:20.933 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:28:20.933 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:28:20.933 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary4 2024-09-12T13:28:21.005 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary4: 2024-09-12T13:28:21.005 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: ce400cdd-ba29-4924-9381-6e78dcb9b581 2024-09-12T13:28:21.005 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+unknown 2024-09-12T13:28:21.005 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: remote image is not primary 2024-09-12T13:28:21.005 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:28:21.005 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:28:17' 2024-09-12T13:28:21.005 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:28:21.007 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+unknown 2024-09-12T13:28:21.008 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:28:21.010 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: remote image is not primary 2024-09-12T13:28:21.010 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:28:21.011 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:28:21.011 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:28:21.013 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:28:21.013 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:28:21.016 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:28:21.016 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:28:21.017 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:28:21.021 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:28:17' 2024-09-12T13:28:21.021 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary4 up+unknown '2024-09-12 13:28:17' 2024-09-12T13:28:21.021 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:28:21.021 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:28:21.021 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:28:21.021 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:28:21.021 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:28:17' 2024-09-12T13:28:21.021 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:28:21.022 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:28:21.022 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-09-12T13:28:21.131 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041934232mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary18bc720a8-a8a7-45cb-ab9a-5646a2cc4cc4up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:28:17image-primary10a020c6a3-79f5-4f05-82bf-2803d0febb5cup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:28:17image-primary2b9571de3-33f9-41ad-aae9-a4841d1fab32up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:28:17image-primary31d6661c9-3d05-4fd7-9ee6-aad6901ddd33up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:28:17image-primary4ce400cdd-ba29-4924-9381-6e78dcb9b581up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:28:17image-primary51d3a6d29-f46e-4dfe-ba1d-2e1bda9e7bc4up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:28:17image-primary6b580eb06-ffeb-44e0-8354-b02afae3f6a4up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:28:17image-primary743088135-2ee1-48a6-8f76-ae96116f16b1up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:28:17image-primary8e5469b72-dbc8-4a77-9c96-ab995f3cc6f1up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:28:17image-primary9cabafe0d-591b-466b-8d57-50e60dd6fa48up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:28:17' 2024-09-12T13:28:21.131 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:28:21.131 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary4'\'']/last_update' 2024-09-12T13:28:21.135 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:28:17' 2024-09-12T13:28:21.136 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary4'\'']/state' 2024-09-12T13:28:21.140 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+unknown 2024-09-12T13:28:21.141 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:28:21.142 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+unknown 2024-09-12T13:28:21.143 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:28:21.144 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:28:21.144 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:28:21.144 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster2 mirror image-primary4 up+unknown 2024-09-12T13:28:21.144 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:28:21.145 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:28:21.145 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:28:21.145 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:28:21.145 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:28:21.145 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:28:21.145 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:28:21.145 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:28:21.468 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary5 up+unknown '' '' 2024-09-12T13:28:21.468 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:28:21.468 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:28:21.468 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:28:21.468 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:28:21.468 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:28:21.468 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:28:21.468 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:28:21.469 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:28:21.469 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary5 2024-09-12T13:28:21.485 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary6 up+unknown '' '' 2024-09-12T13:28:21.485 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:28:21.485 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:28:21.485 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:28:21.485 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:28:21.485 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:28:21.485 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:28:21.485 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:28:21.486 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:28:21.486 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary6 2024-09-12T13:28:21.543 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary5: 2024-09-12T13:28:21.544 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 1d3a6d29-f46e-4dfe-ba1d-2e1bda9e7bc4 2024-09-12T13:28:21.544 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+unknown 2024-09-12T13:28:21.544 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: remote image is not primary 2024-09-12T13:28:21.544 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:28:21.544 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:28:17' 2024-09-12T13:28:21.544 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:28:21.546 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+unknown 2024-09-12T13:28:21.546 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:28:21.549 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: remote image is not primary 2024-09-12T13:28:21.549 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:28:21.550 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:28:21.550 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:28:21.552 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:28:21.552 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:28:21.555 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:28:21.556 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:28:21.556 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:28:21.559 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary6: 2024-09-12T13:28:21.559 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: b580eb06-ffeb-44e0-8354-b02afae3f6a4 2024-09-12T13:28:21.559 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+unknown 2024-09-12T13:28:21.559 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: remote image is not primary 2024-09-12T13:28:21.559 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:28:21.559 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:28:17' 2024-09-12T13:28:21.559 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:28:21.560 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:28:17' 2024-09-12T13:28:21.560 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary5 up+unknown '2024-09-12 13:28:17' 2024-09-12T13:28:21.560 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:28:21.561 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:28:21.561 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:28:21.561 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:28:21.561 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:28:17' 2024-09-12T13:28:21.561 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:28:21.561 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:28:21.561 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-09-12T13:28:21.562 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+unknown 2024-09-12T13:28:21.562 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:28:21.564 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: remote image is not primary 2024-09-12T13:28:21.564 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:28:21.565 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:28:21.565 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:28:21.567 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:28:21.567 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:28:21.570 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:28:21.570 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:28:21.571 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:28:21.575 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:28:17' 2024-09-12T13:28:21.575 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary6 up+unknown '2024-09-12 13:28:17' 2024-09-12T13:28:21.575 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:28:21.575 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:28:21.575 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:28:21.575 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:28:21.575 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:28:17' 2024-09-12T13:28:21.575 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:28:21.576 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:28:21.576 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-09-12T13:28:21.684 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041934232mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary18bc720a8-a8a7-45cb-ab9a-5646a2cc4cc4up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:28:17image-primary10a020c6a3-79f5-4f05-82bf-2803d0febb5cup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:28:17image-primary2b9571de3-33f9-41ad-aae9-a4841d1fab32up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:28:17image-primary31d6661c9-3d05-4fd7-9ee6-aad6901ddd33up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:28:17image-primary4ce400cdd-ba29-4924-9381-6e78dcb9b581up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:28:17image-primary51d3a6d29-f46e-4dfe-ba1d-2e1bda9e7bc4up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:28:17image-primary6b580eb06-ffeb-44e0-8354-b02afae3f6a4up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:28:17image-primary743088135-2ee1-48a6-8f76-ae96116f16b1up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:28:17image-primary8e5469b72-dbc8-4a77-9c96-ab995f3cc6f1up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:28:17image-primary9cabafe0d-591b-466b-8d57-50e60dd6fa48up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:28:17' 2024-09-12T13:28:21.684 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:28:21.684 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary5'\'']/last_update' 2024-09-12T13:28:21.688 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:28:17' 2024-09-12T13:28:21.689 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary5'\'']/state' 2024-09-12T13:28:21.693 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+unknown 2024-09-12T13:28:21.693 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:28:21.694 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+unknown 2024-09-12T13:28:21.697 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:28:21.697 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:28:21.698 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:28:21.698 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster2 mirror image-primary5 up+unknown 2024-09-12T13:28:21.698 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:28:21.698 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:28:21.698 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:28:21.698 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:28:21.698 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:28:21.698 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:28:21.699 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:28:21.699 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:28:21.701 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041934232mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary18bc720a8-a8a7-45cb-ab9a-5646a2cc4cc4up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:28:17image-primary10a020c6a3-79f5-4f05-82bf-2803d0febb5cup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:28:17image-primary2b9571de3-33f9-41ad-aae9-a4841d1fab32up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:28:17image-primary31d6661c9-3d05-4fd7-9ee6-aad6901ddd33up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:28:17image-primary4ce400cdd-ba29-4924-9381-6e78dcb9b581up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:28:17image-primary51d3a6d29-f46e-4dfe-ba1d-2e1bda9e7bc4up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:28:17image-primary6b580eb06-ffeb-44e0-8354-b02afae3f6a4up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:28:17image-primary743088135-2ee1-48a6-8f76-ae96116f16b1up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:28:17image-primary8e5469b72-dbc8-4a77-9c96-ab995f3cc6f1up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:28:17image-primary9cabafe0d-591b-466b-8d57-50e60dd6fa48up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:28:17' 2024-09-12T13:28:21.702 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:28:21.702 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary6'\'']/last_update' 2024-09-12T13:28:21.706 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:28:17' 2024-09-12T13:28:21.707 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary6'\'']/state' 2024-09-12T13:28:21.710 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+unknown 2024-09-12T13:28:21.711 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:28:21.711 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+unknown 2024-09-12T13:28:21.713 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:28:21.715 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:28:21.715 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:28:21.715 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster2 mirror image-primary6 up+unknown 2024-09-12T13:28:21.715 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:28:21.715 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:28:21.715 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:28:21.716 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:28:21.716 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:28:21.716 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:28:21.716 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:28:21.716 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:28:22.147 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster2 mirror image-primary4 up+unknown '' '' 2024-09-12T13:28:22.147 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:28:22.147 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:28:22.147 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:28:22.147 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:28:22.147 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:28:22.147 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:28:22.147 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:28:22.148 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:28:22.148 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster2 mirror image status mirror/image-primary4 2024-09-12T13:28:22.214 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary4: 2024-09-12T13:28:22.214 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: ce400cdd-ba29-4924-9381-6e78dcb9b581 2024-09-12T13:28:22.214 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+unknown 2024-09-12T13:28:22.214 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: remote image is not primary 2024-09-12T13:28:22.214 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:28:22.214 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:28:17' 2024-09-12T13:28:22.214 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:28:22.216 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+unknown 2024-09-12T13:28:22.217 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:28:22.219 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: remote image is not primary 2024-09-12T13:28:22.219 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:28:22.220 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:28:22.220 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:28:22.222 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:28:22.223 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:28:22.225 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:28:22.226 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:28:22.226 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:28:22.230 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:28:17' 2024-09-12T13:28:22.230 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster2 mirror image-primary4 up+unknown '2024-09-12 13:28:17' 2024-09-12T13:28:22.230 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:28:22.230 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:28:22.231 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:28:22.231 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:28:22.231 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:28:17' 2024-09-12T13:28:22.231 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:28:22.231 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:28:22.231 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2024-09-12T13:28:22.335 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041924231mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary18bc720a8-a8a7-45cb-ab9a-5646a2cc4cc4up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:28:17image-primary10a020c6a3-79f5-4f05-82bf-2803d0febb5cup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:28:17image-primary2b9571de3-33f9-41ad-aae9-a4841d1fab32up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:28:17image-primary31d6661c9-3d05-4fd7-9ee6-aad6901ddd33up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:28:17image-primary4ce400cdd-ba29-4924-9381-6e78dcb9b581up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:28:17image-primary51d3a6d29-f46e-4dfe-ba1d-2e1bda9e7bc4up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:28:17image-primary6b580eb06-ffeb-44e0-8354-b02afae3f6a4up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:28:17image-primary743088135-2ee1-48a6-8f76-ae96116f16b1up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:28:17image-primary8e5469b72-dbc8-4a77-9c96-ab995f3cc6f1up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:28:17image-primary9cabafe0d-591b-466b-8d57-50e60dd6fa48up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:28:17' 2024-09-12T13:28:22.335 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:28:22.336 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary4'\'']/last_update' 2024-09-12T13:28:22.339 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:28:17' 2024-09-12T13:28:22.340 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary4'\'']/state' 2024-09-12T13:28:22.344 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+unknown 2024-09-12T13:28:22.345 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:28:22.345 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+unknown 2024-09-12T13:28:22.347 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:28:22.348 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:28:22.348 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:28:22.348 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ promote_image cluster2 mirror image-primary4 2024-09-12T13:28:22.348 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:28:22.349 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:28:22.349 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:28:22.349 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local force= 2024-09-12T13:28:22.349 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 mirror image promote mirror/image-primary4 2024-09-12T13:28:22.398 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary3 up+unknown '' '' 2024-09-12T13:28:22.399 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:28:22.399 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:28:22.399 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:28:22.399 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:28:22.399 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:28:22.399 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:28:22.399 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:28:22.399 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:28:22.399 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary3 2024-09-12T13:28:22.477 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary3: 2024-09-12T13:28:22.478 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 1d6661c9-3d05-4fd7-9ee6-aad6901ddd33 2024-09-12T13:28:22.478 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+unknown 2024-09-12T13:28:22.478 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: remote image is not primary 2024-09-12T13:28:22.478 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:28:22.478 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:28:17' 2024-09-12T13:28:22.478 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:28:22.479 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+unknown 2024-09-12T13:28:22.480 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:28:22.482 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: remote image is not primary 2024-09-12T13:28:22.483 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:28:22.483 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:28:22.483 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:28:22.485 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:28:22.486 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:28:22.488 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:28:22.488 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:28:22.492 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:28:22.493 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:28:17' 2024-09-12T13:28:22.493 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary3 up+unknown '2024-09-12 13:28:17' 2024-09-12T13:28:22.493 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:28:22.493 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:28:22.493 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:28:22.493 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:28:22.493 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:28:17' 2024-09-12T13:28:22.493 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:28:22.494 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:28:22.494 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-09-12T13:28:22.600 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041934232mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary18bc720a8-a8a7-45cb-ab9a-5646a2cc4cc4up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:28:17image-primary10a020c6a3-79f5-4f05-82bf-2803d0febb5cup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:28:17image-primary2b9571de3-33f9-41ad-aae9-a4841d1fab32up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:28:17image-primary31d6661c9-3d05-4fd7-9ee6-aad6901ddd33up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:28:17image-primary4ce400cdd-ba29-4924-9381-6e78dcb9b581up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:28:17image-primary51d3a6d29-f46e-4dfe-ba1d-2e1bda9e7bc4up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:28:17image-primary6b580eb06-ffeb-44e0-8354-b02afae3f6a4up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:28:17image-primary743088135-2ee1-48a6-8f76-ae96116f16b1up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:28:17image-primary8e5469b72-dbc8-4a77-9c96-ab995f3cc6f1up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:28:17image-primary9cabafe0d-591b-466b-8d57-50e60dd6fa48up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:28:17' 2024-09-12T13:28:22.601 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:28:22.601 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary3'\'']/last_update' 2024-09-12T13:28:22.605 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:28:17' 2024-09-12T13:28:22.605 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary3'\'']/state' 2024-09-12T13:28:22.609 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+unknown 2024-09-12T13:28:22.610 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:28:22.610 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+unknown 2024-09-12T13:28:22.612 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:28:22.613 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:28:22.613 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:28:22.614 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster2 mirror image-primary3 up+unknown 2024-09-12T13:28:22.614 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:28:22.614 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:28:22.614 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:28:22.614 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:28:22.614 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:28:22.614 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:28:22.614 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:28:22.614 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:28:22.700 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster2 mirror image-primary5 up+unknown '' '' 2024-09-12T13:28:22.700 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:28:22.700 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:28:22.700 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:28:22.701 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:28:22.701 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:28:22.701 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:28:22.701 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:28:22.701 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:28:22.701 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster2 mirror image status mirror/image-primary5 2024-09-12T13:28:22.717 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster2 mirror image-primary6 up+unknown '' '' 2024-09-12T13:28:22.717 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:28:22.717 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:28:22.717 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:28:22.717 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:28:22.717 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:28:22.717 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:28:22.717 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:28:22.718 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:28:22.718 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster2 mirror image status mirror/image-primary6 2024-09-12T13:28:22.772 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary5: 2024-09-12T13:28:22.772 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 1d3a6d29-f46e-4dfe-ba1d-2e1bda9e7bc4 2024-09-12T13:28:22.773 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+unknown 2024-09-12T13:28:22.773 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: remote image is not primary 2024-09-12T13:28:22.773 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:28:22.773 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:28:17' 2024-09-12T13:28:22.773 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:28:22.775 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+unknown 2024-09-12T13:28:22.776 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:28:22.778 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: remote image is not primary 2024-09-12T13:28:22.779 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:28:22.780 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:28:22.780 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:28:22.783 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:28:22.783 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:28:22.786 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:28:22.786 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:28:22.787 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary6: 2024-09-12T13:28:22.787 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: b580eb06-ffeb-44e0-8354-b02afae3f6a4 2024-09-12T13:28:22.787 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+unknown 2024-09-12T13:28:22.787 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: remote image is not primary 2024-09-12T13:28:22.787 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:28:22.787 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:28:17' 2024-09-12T13:28:22.787 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:28:22.787 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:28:22.789 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+unknown 2024-09-12T13:28:22.790 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:28:22.791 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:28:17' 2024-09-12T13:28:22.791 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster2 mirror image-primary5 up+unknown '2024-09-12 13:28:17' 2024-09-12T13:28:22.791 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:28:22.791 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:28:22.791 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:28:22.791 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:28:22.792 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:28:17' 2024-09-12T13:28:22.792 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:28:22.792 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:28:22.792 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2024-09-12T13:28:22.793 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: remote image is not primary 2024-09-12T13:28:22.793 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:28:22.794 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:28:22.794 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:28:22.796 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:28:22.796 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:28:22.799 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:28:22.799 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:28:22.800 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:28:22.804 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:28:17' 2024-09-12T13:28:22.804 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster2 mirror image-primary6 up+unknown '2024-09-12 13:28:17' 2024-09-12T13:28:22.804 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:28:22.804 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:28:22.804 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:28:22.805 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:28:22.805 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:28:17' 2024-09-12T13:28:22.805 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:28:22.805 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:28:22.805 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2024-09-12T13:28:22.932 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041924231mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary18bc720a8-a8a7-45cb-ab9a-5646a2cc4cc4up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:28:17image-primary10a020c6a3-79f5-4f05-82bf-2803d0febb5cup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:28:17image-primary2b9571de3-33f9-41ad-aae9-a4841d1fab32up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:28:17image-primary31d6661c9-3d05-4fd7-9ee6-aad6901ddd33up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:28:17image-primary4ce400cdd-ba29-4924-9381-6e78dcb9b581up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:28:17image-primary51d3a6d29-f46e-4dfe-ba1d-2e1bda9e7bc4up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:28:17image-primary6b580eb06-ffeb-44e0-8354-b02afae3f6a4up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:28:17image-primary743088135-2ee1-48a6-8f76-ae96116f16b1up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:28:17image-primary8e5469b72-dbc8-4a77-9c96-ab995f3cc6f1up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:28:17image-primary9cabafe0d-591b-466b-8d57-50e60dd6fa48up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:28:17' 2024-09-12T13:28:22.932 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:28:22.933 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary5'\'']/last_update' 2024-09-12T13:28:22.938 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:28:17' 2024-09-12T13:28:22.939 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary5'\'']/state' 2024-09-12T13:28:22.941 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041924231mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary18bc720a8-a8a7-45cb-ab9a-5646a2cc4cc4up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:28:17image-primary10a020c6a3-79f5-4f05-82bf-2803d0febb5cup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:28:17image-primary2b9571de3-33f9-41ad-aae9-a4841d1fab32up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:28:17image-primary31d6661c9-3d05-4fd7-9ee6-aad6901ddd33up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:28:17image-primary4ce400cdd-ba29-4924-9381-6e78dcb9b581up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:28:17image-primary51d3a6d29-f46e-4dfe-ba1d-2e1bda9e7bc4up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:28:17image-primary6b580eb06-ffeb-44e0-8354-b02afae3f6a4up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:28:17image-primary743088135-2ee1-48a6-8f76-ae96116f16b1up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:28:17image-primary8e5469b72-dbc8-4a77-9c96-ab995f3cc6f1up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:28:17image-primary9cabafe0d-591b-466b-8d57-50e60dd6fa48up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:28:17' 2024-09-12T13:28:22.941 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:28:22.942 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary6'\'']/last_update' 2024-09-12T13:28:22.943 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+unknown 2024-09-12T13:28:22.944 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:28:22.945 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+unknown 2024-09-12T13:28:22.946 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:28:17' 2024-09-12T13:28:22.947 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:28:22.947 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary6'\'']/state' 2024-09-12T13:28:22.948 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:28:22.948 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:28:22.948 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ promote_image cluster2 mirror image-primary5 2024-09-12T13:28:22.948 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:28:22.949 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:28:22.949 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:28:22.949 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local force= 2024-09-12T13:28:22.949 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 mirror image promote mirror/image-primary5 2024-09-12T13:28:22.951 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+unknown 2024-09-12T13:28:22.952 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:28:22.952 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+unknown 2024-09-12T13:28:22.955 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:28:22.955 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:28:22.955 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:28:22.955 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ promote_image cluster2 mirror image-primary6 2024-09-12T13:28:22.956 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:28:22.956 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:28:22.956 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:28:22.956 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local force= 2024-09-12T13:28:22.956 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 mirror image promote mirror/image-primary6 2024-09-12T13:28:23.141 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Image promoted to primary 2024-09-12T13:28:23.144 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Image promoted to primary 2024-09-12T13:28:23.144 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Image promoted to primary 2024-09-12T13:28:23.155 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \n\b\d ]] 2024-09-12T13:28:23.156 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \k\r\b\d ]] 2024-09-12T13:28:23.156 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo rbd --cluster cluster2 device map -t krbd mirror/image-primary5 2024-09-12T13:28:23.160 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \n\b\d ]] 2024-09-12T13:28:23.160 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \k\r\b\d ]] 2024-09-12T13:28:23.160 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \n\b\d ]] 2024-09-12T13:28:23.160 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \k\r\b\d ]] 2024-09-12T13:28:23.161 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo rbd --cluster cluster2 device map -t krbd mirror/image-primary4 2024-09-12T13:28:23.161 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo rbd --cluster cluster2 device map -t krbd mirror/image-primary6 2024-09-12T13:28:23.309 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ dev=/dev/rbd2 2024-09-12T13:28:23.310 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo md5sum /dev/rbd2 2024-09-12T13:28:23.310 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ awk '{print $1}' 2024-09-12T13:28:23.312 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ dev=/dev/rbd1 2024-09-12T13:28:23.313 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ dev=/dev/rbd3 2024-09-12T13:28:23.314 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo md5sum /dev/rbd1 2024-09-12T13:28:23.315 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo md5sum /dev/rbd3 2024-09-12T13:28:23.315 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ awk '{print $1}' 2024-09-12T13:28:23.315 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ awk '{print $1}' 2024-09-12T13:28:23.545 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary9 up+unknown '' '' 2024-09-12T13:28:23.546 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:28:23.546 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:28:23.546 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:28:23.546 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:28:23.546 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:28:23.546 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:28:23.546 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:28:23.546 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:28:23.547 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary9 2024-09-12T13:28:23.553 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary10 up+unknown '' '' 2024-09-12T13:28:23.553 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:28:23.553 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:28:23.553 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:28:23.553 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:28:23.553 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:28:23.553 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:28:23.553 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:28:23.554 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:28:23.554 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary10 2024-09-12T13:28:23.579 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary2 up+unknown '' '' 2024-09-12T13:28:23.579 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:28:23.580 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:28:23.580 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:28:23.580 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:28:23.580 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:28:23.580 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:28:23.580 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:28:23.581 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:28:23.581 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary2 2024-09-12T13:28:23.619 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster2 mirror image-primary3 up+unknown '' '' 2024-09-12T13:28:23.619 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:28:23.620 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:28:23.620 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:28:23.620 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:28:23.620 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:28:23.620 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:28:23.620 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:28:23.621 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:28:23.621 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster2 mirror image status mirror/image-primary3 2024-09-12T13:28:23.669 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary10: 2024-09-12T13:28:23.670 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: a020c6a3-79f5-4f05-82bf-2803d0febb5c 2024-09-12T13:28:23.670 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+unknown 2024-09-12T13:28:23.670 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: remote image is not primary 2024-09-12T13:28:23.670 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:28:23.670 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:28:17' 2024-09-12T13:28:23.670 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:28:23.673 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+unknown 2024-09-12T13:28:23.673 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:28:23.674 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary9: 2024-09-12T13:28:23.674 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: cabafe0d-591b-466b-8d57-50e60dd6fa48 2024-09-12T13:28:23.675 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+unknown 2024-09-12T13:28:23.675 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: remote image is not primary 2024-09-12T13:28:23.675 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:28:23.675 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:28:17' 2024-09-12T13:28:23.675 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:28:23.678 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: remote image is not primary 2024-09-12T13:28:23.678 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+unknown 2024-09-12T13:28:23.679 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:28:23.679 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:28:23.681 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:28:23.681 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:28:23.683 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: remote image is not primary 2024-09-12T13:28:23.684 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:28:23.684 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:28:23.686 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:28:23.686 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:28:23.686 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:28:23.689 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:28:23.689 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:28:23.690 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:28:23.690 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:28:23.692 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:28:23.695 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:28:23.695 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:28:23.697 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:28:23.699 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:28:17' 2024-09-12T13:28:23.699 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary10 up+unknown '2024-09-12 13:28:17' 2024-09-12T13:28:23.699 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:28:23.700 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:28:23.700 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:28:23.700 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:28:23.700 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:28:17' 2024-09-12T13:28:23.700 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:28:23.701 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:28:23.701 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-09-12T13:28:23.705 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:28:17' 2024-09-12T13:28:23.706 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary9 up+unknown '2024-09-12 13:28:17' 2024-09-12T13:28:23.706 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:28:23.706 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:28:23.706 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:28:23.706 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:28:23.706 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:28:17' 2024-09-12T13:28:23.706 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:28:23.707 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:28:23.707 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-09-12T13:28:23.714 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary2: 2024-09-12T13:28:23.714 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: b9571de3-33f9-41ad-aae9-a4841d1fab32 2024-09-12T13:28:23.714 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+unknown 2024-09-12T13:28:23.714 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: remote image is not primary 2024-09-12T13:28:23.715 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:28:23.715 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:28:17' 2024-09-12T13:28:23.715 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:28:23.726 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+unknown 2024-09-12T13:28:23.728 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:28:23.732 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: remote image is not primary 2024-09-12T13:28:23.737 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:28:23.739 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:28:23.739 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:28:23.742 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:28:23.743 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:28:23.748 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:28:23.749 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:28:23.751 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:28:23.758 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:28:17' 2024-09-12T13:28:23.758 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary2 up+unknown '2024-09-12 13:28:17' 2024-09-12T13:28:23.758 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:28:23.758 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:28:23.758 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary3: 2024-09-12T13:28:23.758 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 1d6661c9-3d05-4fd7-9ee6-aad6901ddd33 2024-09-12T13:28:23.758 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+unknown 2024-09-12T13:28:23.758 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: remote image is not primary 2024-09-12T13:28:23.758 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:28:23.759 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:28:17' 2024-09-12T13:28:23.759 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:28:23.759 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:28:23.759 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:28:23.759 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:28:17' 2024-09-12T13:28:23.759 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:28:23.759 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:28:23.760 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-09-12T13:28:23.762 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+unknown 2024-09-12T13:28:23.763 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:28:23.768 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: remote image is not primary 2024-09-12T13:28:23.769 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:28:23.772 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:28:23.772 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:28:23.775 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:28:23.776 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:28:23.782 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:28:23.782 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:28:23.784 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:28:23.792 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:28:17' 2024-09-12T13:28:23.792 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster2 mirror image-primary3 up+unknown '2024-09-12 13:28:17' 2024-09-12T13:28:23.792 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:28:23.792 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:28:23.792 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:28:23.792 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:28:23.792 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:28:17' 2024-09-12T13:28:23.792 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:28:23.793 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:28:23.793 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2024-09-12T13:28:23.910 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041934232mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary18bc720a8-a8a7-45cb-ab9a-5646a2cc4cc4up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:28:17image-primary10a020c6a3-79f5-4f05-82bf-2803d0febb5cup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:28:17image-primary2b9571de3-33f9-41ad-aae9-a4841d1fab32up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:28:17image-primary31d6661c9-3d05-4fd7-9ee6-aad6901ddd33up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:28:17image-primary4ce400cdd-ba29-4924-9381-6e78dcb9b581up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:28:17image-primary51d3a6d29-f46e-4dfe-ba1d-2e1bda9e7bc4up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:28:17image-primary6b580eb06-ffeb-44e0-8354-b02afae3f6a4up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:28:17image-primary743088135-2ee1-48a6-8f76-ae96116f16b1up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:28:17image-primary8e5469b72-dbc8-4a77-9c96-ab995f3cc6f1up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:28:17image-primary9cabafe0d-591b-466b-8d57-50e60dd6fa48up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:28:17' 2024-09-12T13:28:23.910 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:28:23.910 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary10'\'']/last_update' 2024-09-12T13:28:23.917 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:28:17' 2024-09-12T13:28:23.920 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary10'\'']/state' 2024-09-12T13:28:23.931 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041934232mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary18bc720a8-a8a7-45cb-ab9a-5646a2cc4cc4up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:28:17image-primary10a020c6a3-79f5-4f05-82bf-2803d0febb5cup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:28:17image-primary2b9571de3-33f9-41ad-aae9-a4841d1fab32up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:28:17image-primary31d6661c9-3d05-4fd7-9ee6-aad6901ddd33up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:28:17image-primary4ce400cdd-ba29-4924-9381-6e78dcb9b581up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:28:17image-primary51d3a6d29-f46e-4dfe-ba1d-2e1bda9e7bc4up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:28:17image-primary6b580eb06-ffeb-44e0-8354-b02afae3f6a4up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:28:17image-primary743088135-2ee1-48a6-8f76-ae96116f16b1up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:28:17image-primary8e5469b72-dbc8-4a77-9c96-ab995f3cc6f1up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:28:17image-primary9cabafe0d-591b-466b-8d57-50e60dd6fa48up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:28:17' 2024-09-12T13:28:23.931 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:28:23.931 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+unknown 2024-09-12T13:28:23.932 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:28:23.933 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary9'\'']/last_update' 2024-09-12T13:28:23.933 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+unknown 2024-09-12T13:28:23.937 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:28:23.939 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:28:23.939 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:28:23.939 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster2 mirror image-primary10 up+unknown 2024-09-12T13:28:23.939 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:28:23.939 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:28:23.939 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:28:23.939 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:28:23.940 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:28:23.940 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:28:23.940 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:28:23.940 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:28:23.946 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:28:17' 2024-09-12T13:28:23.947 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary9'\'']/state' 2024-09-12T13:28:23.958 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+unknown 2024-09-12T13:28:23.958 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:28:23.959 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+unknown 2024-09-12T13:28:23.963 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:28:23.964 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:28:23.965 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:28:23.965 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster2 mirror image-primary9 up+unknown 2024-09-12T13:28:23.966 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:28:23.966 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:28:23.966 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:28:23.966 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:28:23.966 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:28:23.966 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:28:23.966 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:28:23.967 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:28:23.971 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041934232mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary18bc720a8-a8a7-45cb-ab9a-5646a2cc4cc4up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:28:17image-primary10a020c6a3-79f5-4f05-82bf-2803d0febb5cup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:28:17image-primary2b9571de3-33f9-41ad-aae9-a4841d1fab32up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:28:17image-primary31d6661c9-3d05-4fd7-9ee6-aad6901ddd33up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:28:17image-primary4ce400cdd-ba29-4924-9381-6e78dcb9b581up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:28:17image-primary51d3a6d29-f46e-4dfe-ba1d-2e1bda9e7bc4up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:28:17image-primary6b580eb06-ffeb-44e0-8354-b02afae3f6a4up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:28:17image-primary743088135-2ee1-48a6-8f76-ae96116f16b1up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:28:17image-primary8e5469b72-dbc8-4a77-9c96-ab995f3cc6f1up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:28:17image-primary9cabafe0d-591b-466b-8d57-50e60dd6fa48up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:28:17' 2024-09-12T13:28:23.972 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:28:23.974 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary2'\'']/last_update' 2024-09-12T13:28:23.982 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:28:17' 2024-09-12T13:28:23.984 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary2'\'']/state' 2024-09-12T13:28:23.991 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+unknown 2024-09-12T13:28:23.992 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:28:23.993 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+unknown 2024-09-12T13:28:23.996 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:28:23.998 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:28:23.998 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:28:23.998 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster2 mirror image-primary2 up+unknown 2024-09-12T13:28:23.998 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:28:23.998 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:28:23.999 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:28:23.999 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:28:23.999 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:28:23.999 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:28:23.999 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:28:23.999 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:28:24.009 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041924231mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary18bc720a8-a8a7-45cb-ab9a-5646a2cc4cc4up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:28:17image-primary10a020c6a3-79f5-4f05-82bf-2803d0febb5cup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:28:17image-primary2b9571de3-33f9-41ad-aae9-a4841d1fab32up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:28:17image-primary31d6661c9-3d05-4fd7-9ee6-aad6901ddd33up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:28:17image-primary4ce400cdd-ba29-4924-9381-6e78dcb9b581up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:28:17image-primary51d3a6d29-f46e-4dfe-ba1d-2e1bda9e7bc4up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:28:17image-primary6b580eb06-ffeb-44e0-8354-b02afae3f6a4up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:28:17image-primary743088135-2ee1-48a6-8f76-ae96116f16b1up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:28:17image-primary8e5469b72-dbc8-4a77-9c96-ab995f3cc6f1up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:28:17image-primary9cabafe0d-591b-466b-8d57-50e60dd6fa48up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:28:17' 2024-09-12T13:28:24.009 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:28:24.010 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary3'\'']/last_update' 2024-09-12T13:28:24.017 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:28:17' 2024-09-12T13:28:24.019 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary3'\'']/state' 2024-09-12T13:28:24.025 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+unknown 2024-09-12T13:28:24.027 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:28:24.027 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+unknown 2024-09-12T13:28:24.030 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:28:24.031 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:28:24.032 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:28:24.032 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ promote_image cluster2 mirror image-primary3 2024-09-12T13:28:24.032 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:28:24.032 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:28:24.032 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:28:24.032 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local force= 2024-09-12T13:28:24.032 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 mirror image promote mirror/image-primary3 2024-09-12T13:28:24.443 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary7 up+unknown '' '' 2024-09-12T13:28:24.444 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:28:24.444 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:28:24.444 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:28:24.444 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:28:24.444 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:28:24.444 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:28:24.444 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:28:24.444 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:28:24.445 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary7 2024-09-12T13:28:24.506 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary8 up+unknown '' '' 2024-09-12T13:28:24.506 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:28:24.506 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:28:24.506 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:28:24.506 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:28:24.507 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:28:24.507 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:28:24.507 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:28:24.507 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:28:24.508 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary8 2024-09-12T13:28:24.552 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary7: 2024-09-12T13:28:24.552 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 43088135-2ee1-48a6-8f76-ae96116f16b1 2024-09-12T13:28:24.552 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+unknown 2024-09-12T13:28:24.552 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: remote image is not primary 2024-09-12T13:28:24.552 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:28:24.552 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:28:17' 2024-09-12T13:28:24.552 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:28:24.555 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+unknown 2024-09-12T13:28:24.555 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:28:24.558 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: remote image is not primary 2024-09-12T13:28:24.559 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:28:24.560 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:28:24.560 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:28:24.564 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:28:24.564 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:28:24.567 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:28:24.568 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:28:24.569 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:28:24.574 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:28:17' 2024-09-12T13:28:24.574 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary7 up+unknown '2024-09-12 13:28:17' 2024-09-12T13:28:24.574 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:28:24.574 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:28:24.574 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:28:24.574 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:28:24.574 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:28:17' 2024-09-12T13:28:24.574 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:28:24.575 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:28:24.575 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-09-12T13:28:24.600 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary8: 2024-09-12T13:28:24.600 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: e5469b72-dbc8-4a77-9c96-ab995f3cc6f1 2024-09-12T13:28:24.600 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+unknown 2024-09-12T13:28:24.600 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: remote image is not primary 2024-09-12T13:28:24.600 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:28:24.600 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:28:17' 2024-09-12T13:28:24.600 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:28:24.604 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+unknown 2024-09-12T13:28:24.604 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:28:24.609 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: remote image is not primary 2024-09-12T13:28:24.609 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:28:24.610 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:28:24.611 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:28:24.614 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:28:24.615 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:28:24.619 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:28:24.620 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:28:24.621 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:28:24.627 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:28:17' 2024-09-12T13:28:24.627 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary8 up+unknown '2024-09-12 13:28:17' 2024-09-12T13:28:24.627 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:28:24.627 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:28:24.627 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:28:24.627 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:28:24.627 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:28:17' 2024-09-12T13:28:24.627 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:28:24.628 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:28:24.628 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-09-12T13:28:24.714 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041934232mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary18bc720a8-a8a7-45cb-ab9a-5646a2cc4cc4up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:28:17image-primary10a020c6a3-79f5-4f05-82bf-2803d0febb5cup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:28:17image-primary2b9571de3-33f9-41ad-aae9-a4841d1fab32up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:28:17image-primary31d6661c9-3d05-4fd7-9ee6-aad6901ddd33up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:28:17image-primary4ce400cdd-ba29-4924-9381-6e78dcb9b581up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:28:17image-primary51d3a6d29-f46e-4dfe-ba1d-2e1bda9e7bc4up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:28:17image-primary6b580eb06-ffeb-44e0-8354-b02afae3f6a4up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:28:17image-primary743088135-2ee1-48a6-8f76-ae96116f16b1up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:28:17image-primary8e5469b72-dbc8-4a77-9c96-ab995f3cc6f1up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:28:17image-primary9cabafe0d-591b-466b-8d57-50e60dd6fa48up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:28:17' 2024-09-12T13:28:24.714 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:28:24.716 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary7'\'']/last_update' 2024-09-12T13:28:24.722 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:28:17' 2024-09-12T13:28:24.724 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary7'\'']/state' 2024-09-12T13:28:24.732 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+unknown 2024-09-12T13:28:24.732 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:28:24.732 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+unknown 2024-09-12T13:28:24.736 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:28:24.737 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:28:24.737 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:28:24.737 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster2 mirror image-primary7 up+unknown 2024-09-12T13:28:24.737 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:28:24.737 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:28:24.738 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:28:24.738 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:28:24.738 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:28:24.738 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:28:24.738 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:28:24.738 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:28:24.768 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041934232mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary18bc720a8-a8a7-45cb-ab9a-5646a2cc4cc4up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:28:17image-primary10a020c6a3-79f5-4f05-82bf-2803d0febb5cup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:28:17image-primary2b9571de3-33f9-41ad-aae9-a4841d1fab32up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:28:17image-primary31d6661c9-3d05-4fd7-9ee6-aad6901ddd33up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:28:17image-primary4ce400cdd-ba29-4924-9381-6e78dcb9b581up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:28:17image-primary51d3a6d29-f46e-4dfe-ba1d-2e1bda9e7bc4up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:28:17image-primary6b580eb06-ffeb-44e0-8354-b02afae3f6a4up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:28:17image-primary743088135-2ee1-48a6-8f76-ae96116f16b1up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:28:17image-primary8e5469b72-dbc8-4a77-9c96-ab995f3cc6f1up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:28:17image-primary9cabafe0d-591b-466b-8d57-50e60dd6fa48up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:28:17' 2024-09-12T13:28:24.768 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:28:24.769 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary8'\'']/last_update' 2024-09-12T13:28:24.776 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:28:17' 2024-09-12T13:28:24.778 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary8'\'']/state' 2024-09-12T13:28:24.785 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+unknown 2024-09-12T13:28:24.786 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:28:24.786 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+unknown 2024-09-12T13:28:24.790 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:28:24.791 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:28:24.791 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:28:24.791 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster2 mirror image-primary8 up+unknown 2024-09-12T13:28:24.791 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:28:24.791 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:28:24.791 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:28:24.791 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:28:24.792 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:28:24.792 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:28:24.792 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:28:24.792 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:28:24.946 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster2 mirror image-primary10 up+unknown '' '' 2024-09-12T13:28:24.946 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:28:24.946 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:28:24.946 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:28:24.946 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:28:24.946 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:28:24.947 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:28:24.947 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:28:24.947 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:28:24.947 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster2 mirror image status mirror/image-primary10 2024-09-12T13:28:24.971 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster2 mirror image-primary9 up+unknown '' '' 2024-09-12T13:28:24.971 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:28:24.971 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:28:24.971 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:28:24.972 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:28:24.972 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:28:24.972 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:28:24.972 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:28:24.972 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:28:24.972 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster2 mirror image status mirror/image-primary9 2024-09-12T13:28:25.003 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster2 mirror image-primary2 up+unknown '' '' 2024-09-12T13:28:25.003 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:28:25.003 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:28:25.004 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:28:25.004 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:28:25.004 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:28:25.004 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:28:25.004 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:28:25.005 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:28:25.005 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster2 mirror image status mirror/image-primary2 2024-09-12T13:28:25.064 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary10: 2024-09-12T13:28:25.064 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: a020c6a3-79f5-4f05-82bf-2803d0febb5c 2024-09-12T13:28:25.064 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+unknown 2024-09-12T13:28:25.064 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: remote image is not primary 2024-09-12T13:28:25.064 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:28:25.064 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:28:17' 2024-09-12T13:28:25.065 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:28:25.067 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+unknown 2024-09-12T13:28:25.069 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:28:25.073 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: remote image is not primary 2024-09-12T13:28:25.075 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:28:25.075 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:28:25.075 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:28:25.079 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:28:25.080 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:28:25.080 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary9: 2024-09-12T13:28:25.080 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: cabafe0d-591b-466b-8d57-50e60dd6fa48 2024-09-12T13:28:25.081 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+unknown 2024-09-12T13:28:25.081 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: remote image is not primary 2024-09-12T13:28:25.081 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:28:25.081 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:28:17' 2024-09-12T13:28:25.081 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:28:25.083 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:28:25.085 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:28:25.086 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:28:25.086 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+unknown 2024-09-12T13:28:25.087 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:28:25.091 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:28:17' 2024-09-12T13:28:25.091 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster2 mirror image-primary10 up+unknown '2024-09-12 13:28:17' 2024-09-12T13:28:25.091 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:28:25.091 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:28:25.091 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:28:25.091 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:28:25.092 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:28:17' 2024-09-12T13:28:25.092 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:28:25.092 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:28:25.092 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2024-09-12T13:28:25.093 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: remote image is not primary 2024-09-12T13:28:25.096 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:28:25.098 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:28:25.098 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:28:25.101 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:28:25.102 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:28:25.106 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:28:25.106 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:28:25.108 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:28:25.116 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:28:17' 2024-09-12T13:28:25.116 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster2 mirror image-primary9 up+unknown '2024-09-12 13:28:17' 2024-09-12T13:28:25.116 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:28:25.116 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:28:25.116 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:28:25.116 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:28:25.116 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:28:17' 2024-09-12T13:28:25.116 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:28:25.117 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:28:25.117 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2024-09-12T13:28:25.138 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary2: 2024-09-12T13:28:25.138 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: b9571de3-33f9-41ad-aae9-a4841d1fab32 2024-09-12T13:28:25.138 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+unknown 2024-09-12T13:28:25.138 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: remote image is not primary 2024-09-12T13:28:25.138 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:28:25.138 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:28:17' 2024-09-12T13:28:25.138 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:28:25.143 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+unknown 2024-09-12T13:28:25.144 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:28:25.154 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: remote image is not primary 2024-09-12T13:28:25.155 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:28:25.157 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:28:25.158 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:28:25.161 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:28:25.162 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:28:25.169 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:28:25.169 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:28:25.171 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:28:25.179 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:28:17' 2024-09-12T13:28:25.179 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster2 mirror image-primary2 up+unknown '2024-09-12 13:28:17' 2024-09-12T13:28:25.179 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:28:25.179 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:28:25.179 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:28:25.179 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:28:25.179 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:28:17' 2024-09-12T13:28:25.179 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:28:25.181 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:28:25.181 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2024-09-12T13:28:25.181 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Image promoted to primary 2024-09-12T13:28:25.223 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \n\b\d ]] 2024-09-12T13:28:25.223 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \k\r\b\d ]] 2024-09-12T13:28:25.225 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo rbd --cluster cluster2 device map -t krbd mirror/image-primary3 2024-09-12T13:28:25.349 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041924231mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary18bc720a8-a8a7-45cb-ab9a-5646a2cc4cc4up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:28:17image-primary10a020c6a3-79f5-4f05-82bf-2803d0febb5cup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:28:17image-primary2b9571de3-33f9-41ad-aae9-a4841d1fab32up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:28:17image-primary31d6661c9-3d05-4fd7-9ee6-aad6901ddd33up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:28:17image-primary4ce400cdd-ba29-4924-9381-6e78dcb9b581up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:28:17image-primary51d3a6d29-f46e-4dfe-ba1d-2e1bda9e7bc4up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:28:17image-primary6b580eb06-ffeb-44e0-8354-b02afae3f6a4up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:28:17image-primary743088135-2ee1-48a6-8f76-ae96116f16b1up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:28:17image-primary8e5469b72-dbc8-4a77-9c96-ab995f3cc6f1up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:28:17image-primary9cabafe0d-591b-466b-8d57-50e60dd6fa48up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:28:17' 2024-09-12T13:28:25.349 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:28:25.351 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary10'\'']/last_update' 2024-09-12T13:28:25.360 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:28:17' 2024-09-12T13:28:25.361 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary10'\'']/state' 2024-09-12T13:28:25.371 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+unknown 2024-09-12T13:28:25.372 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:28:25.374 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041924231mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary18bc720a8-a8a7-45cb-ab9a-5646a2cc4cc4up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:28:17image-primary10a020c6a3-79f5-4f05-82bf-2803d0febb5cup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:28:17image-primary2b9571de3-33f9-41ad-aae9-a4841d1fab32up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:28:17image-primary31d6661c9-3d05-4fd7-9ee6-aad6901ddd33up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:28:17image-primary4ce400cdd-ba29-4924-9381-6e78dcb9b581up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:28:17image-primary51d3a6d29-f46e-4dfe-ba1d-2e1bda9e7bc4up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:28:17image-primary6b580eb06-ffeb-44e0-8354-b02afae3f6a4up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:28:17image-primary743088135-2ee1-48a6-8f76-ae96116f16b1up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:28:17image-primary8e5469b72-dbc8-4a77-9c96-ab995f3cc6f1up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:28:17image-primary9cabafe0d-591b-466b-8d57-50e60dd6fa48up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:28:17' 2024-09-12T13:28:25.375 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:28:25.376 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary9'\'']/last_update' 2024-09-12T13:28:25.376 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:28:25.377 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:28:25.378 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:28:25.378 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ promote_image cluster2 mirror image-primary10 2024-09-12T13:28:25.378 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:28:25.378 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:28:25.378 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:28:25.378 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local force= 2024-09-12T13:28:25.378 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 mirror image promote mirror/image-primary10 2024-09-12T13:28:25.389 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:28:17' 2024-09-12T13:28:25.389 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary9'\'']/state' 2024-09-12T13:28:25.401 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+unknown 2024-09-12T13:28:25.406 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:28:25.408 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+unknown 2024-09-12T13:28:25.412 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:28:25.413 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:28:25.414 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:28:25.414 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ promote_image cluster2 mirror image-primary9 2024-09-12T13:28:25.414 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:28:25.414 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:28:25.414 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:28:25.414 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local force= 2024-09-12T13:28:25.414 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 mirror image promote mirror/image-primary9 2024-09-12T13:28:25.500 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041924231mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary18bc720a8-a8a7-45cb-ab9a-5646a2cc4cc4up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:28:17image-primary10a020c6a3-79f5-4f05-82bf-2803d0febb5cup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:28:17image-primary2b9571de3-33f9-41ad-aae9-a4841d1fab32up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:28:17image-primary31d6661c9-3d05-4fd7-9ee6-aad6901ddd33up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:28:17image-primary4ce400cdd-ba29-4924-9381-6e78dcb9b581up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:28:17image-primary51d3a6d29-f46e-4dfe-ba1d-2e1bda9e7bc4up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:28:17image-primary6b580eb06-ffeb-44e0-8354-b02afae3f6a4up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:28:17image-primary743088135-2ee1-48a6-8f76-ae96116f16b1up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:28:17image-primary8e5469b72-dbc8-4a77-9c96-ab995f3cc6f1up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:28:17image-primary9cabafe0d-591b-466b-8d57-50e60dd6fa48up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:28:17' 2024-09-12T13:28:25.500 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:28:25.502 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary2'\'']/last_update' 2024-09-12T13:28:25.507 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:28:17' 2024-09-12T13:28:25.509 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary2'\'']/state' 2024-09-12T13:28:25.522 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+unknown 2024-09-12T13:28:25.522 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+unknown 2024-09-12T13:28:25.524 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:28:25.525 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:28:25.529 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:28:25.529 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:28:25.529 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ promote_image cluster2 mirror image-primary2 2024-09-12T13:28:25.529 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:28:25.529 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:28:25.529 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:28:25.529 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local force= 2024-09-12T13:28:25.529 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 mirror image promote mirror/image-primary2 2024-09-12T13:28:25.570 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ dev=/dev/rbd4 2024-09-12T13:28:25.572 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo md5sum /dev/rbd4 2024-09-12T13:28:25.573 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ awk '{print $1}' 2024-09-12T13:28:25.741 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster2 mirror image-primary7 up+unknown '' '' 2024-09-12T13:28:25.741 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:28:25.742 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:28:25.742 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:28:25.742 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:28:25.742 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:28:25.742 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:28:25.742 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:28:25.742 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:28:25.742 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster2 mirror image status mirror/image-primary7 2024-09-12T13:28:25.795 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster2 mirror image-primary8 up+unknown '' '' 2024-09-12T13:28:25.795 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:28:25.795 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:28:25.795 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:28:25.795 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:28:25.795 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:28:25.795 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:28:25.796 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:28:25.796 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:28:25.796 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster2 mirror image status mirror/image-primary8 2024-09-12T13:28:25.873 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary7: 2024-09-12T13:28:25.874 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 43088135-2ee1-48a6-8f76-ae96116f16b1 2024-09-12T13:28:25.874 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+unknown 2024-09-12T13:28:25.874 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: remote image is not primary 2024-09-12T13:28:25.874 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:28:25.874 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:28:17' 2024-09-12T13:28:25.874 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:28:25.879 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+unknown 2024-09-12T13:28:25.880 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:28:25.884 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: remote image is not primary 2024-09-12T13:28:25.885 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:28:25.886 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:28:25.887 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:28:25.891 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:28:25.892 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:28:25.896 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:28:25.897 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:28:25.898 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:28:25.905 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:28:17' 2024-09-12T13:28:25.905 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster2 mirror image-primary7 up+unknown '2024-09-12 13:28:17' 2024-09-12T13:28:25.905 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:28:25.905 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:28:25.906 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:28:25.907 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:28:25.907 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:28:17' 2024-09-12T13:28:25.907 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:28:25.908 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:28:25.908 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2024-09-12T13:28:25.921 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary8: 2024-09-12T13:28:25.921 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: e5469b72-dbc8-4a77-9c96-ab995f3cc6f1 2024-09-12T13:28:25.922 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+unknown 2024-09-12T13:28:25.922 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: remote image is not primary 2024-09-12T13:28:25.922 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:28:25.922 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:28:17' 2024-09-12T13:28:25.922 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:28:25.926 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+unknown 2024-09-12T13:28:25.927 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:28:25.931 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: remote image is not primary 2024-09-12T13:28:25.933 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:28:25.933 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:28:25.934 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:28:25.937 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:28:25.938 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:28:25.943 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:28:25.944 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:28:25.945 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:28:25.952 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:28:17' 2024-09-12T13:28:25.952 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster2 mirror image-primary8 up+unknown '2024-09-12 13:28:17' 2024-09-12T13:28:25.952 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:28:25.952 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:28:25.952 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:28:25.952 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:28:25.953 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:28:17' 2024-09-12T13:28:25.953 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:28:25.953 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:28:25.953 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2024-09-12T13:28:26.089 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041924231mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary18bc720a8-a8a7-45cb-ab9a-5646a2cc4cc4up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:28:17image-primary10a020c6a3-79f5-4f05-82bf-2803d0febb5cup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:28:17image-primary2b9571de3-33f9-41ad-aae9-a4841d1fab32up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:28:17image-primary31d6661c9-3d05-4fd7-9ee6-aad6901ddd33up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:28:17image-primary4ce400cdd-ba29-4924-9381-6e78dcb9b581up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:28:17image-primary51d3a6d29-f46e-4dfe-ba1d-2e1bda9e7bc4up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:28:17image-primary6b580eb06-ffeb-44e0-8354-b02afae3f6a4up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:28:17image-primary743088135-2ee1-48a6-8f76-ae96116f16b1up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:28:17image-primary8e5469b72-dbc8-4a77-9c96-ab995f3cc6f1up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:28:17image-primary9cabafe0d-591b-466b-8d57-50e60dd6fa48up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:28:17' 2024-09-12T13:28:26.089 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:28:26.094 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary7'\'']/last_update' 2024-09-12T13:28:26.108 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:28:17' 2024-09-12T13:28:26.110 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary7'\'']/state' 2024-09-12T13:28:26.117 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+unknown 2024-09-12T13:28:26.119 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:28:26.120 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+unknown 2024-09-12T13:28:26.125 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:28:26.126 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:28:26.126 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:28:26.126 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ promote_image cluster2 mirror image-primary7 2024-09-12T13:28:26.126 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:28:26.126 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:28:26.126 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:28:26.126 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local force= 2024-09-12T13:28:26.127 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 mirror image promote mirror/image-primary7 2024-09-12T13:28:26.260 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Image promoted to primary 2024-09-12T13:28:26.268 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Image promoted to primary 2024-09-12T13:28:26.280 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Image promoted to primary 2024-09-12T13:28:26.281 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041924231mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary18bc720a8-a8a7-45cb-ab9a-5646a2cc4cc4up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:28:17image-primary10a020c6a3-79f5-4f05-82bf-2803d0febb5cup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:28:17image-primary2b9571de3-33f9-41ad-aae9-a4841d1fab32up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:28:17image-primary31d6661c9-3d05-4fd7-9ee6-aad6901ddd33up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:28:17image-primary4ce400cdd-ba29-4924-9381-6e78dcb9b581up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:28:17image-primary51d3a6d29-f46e-4dfe-ba1d-2e1bda9e7bc4up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:28:17image-primary6b580eb06-ffeb-44e0-8354-b02afae3f6a4up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:28:17image-primary743088135-2ee1-48a6-8f76-ae96116f16b1up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:28:17image-primary8e5469b72-dbc8-4a77-9c96-ab995f3cc6f1up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:28:17image-primary9cabafe0d-591b-466b-8d57-50e60dd6fa48up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:28:17' 2024-09-12T13:28:26.282 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:28:26.284 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary8'\'']/last_update' 2024-09-12T13:28:26.293 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:28:17' 2024-09-12T13:28:26.295 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary8'\'']/state' 2024-09-12T13:28:26.301 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \n\b\d ]] 2024-09-12T13:28:26.301 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \k\r\b\d ]] 2024-09-12T13:28:26.302 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo rbd --cluster cluster2 device map -t krbd mirror/image-primary10 2024-09-12T13:28:26.302 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+unknown 2024-09-12T13:28:26.304 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:28:26.305 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+unknown 2024-09-12T13:28:26.308 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \n\b\d ]] 2024-09-12T13:28:26.308 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \k\r\b\d ]] 2024-09-12T13:28:26.309 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo rbd --cluster cluster2 device map -t krbd mirror/image-primary2 2024-09-12T13:28:26.319 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:28:26.321 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:28:26.321 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:28:26.322 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ promote_image cluster2 mirror image-primary8 2024-09-12T13:28:26.322 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:28:26.322 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:28:26.322 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:28:26.322 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local force= 2024-09-12T13:28:26.322 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 mirror image promote mirror/image-primary8 2024-09-12T13:28:26.346 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \n\b\d ]] 2024-09-12T13:28:26.346 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \k\r\b\d ]] 2024-09-12T13:28:26.346 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo rbd --cluster cluster2 device map -t krbd mirror/image-primary9 2024-09-12T13:28:26.584 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ dev=/dev/rbd5 2024-09-12T13:28:26.589 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo md5sum /dev/rbd5 2024-09-12T13:28:26.591 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ awk '{print $1}' 2024-09-12T13:28:26.637 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ dev=/dev/rbd6 2024-09-12T13:28:26.643 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo md5sum /dev/rbd6 2024-09-12T13:28:26.643 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ awk '{print $1}' 2024-09-12T13:28:26.665 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ dev=/dev/rbd7 2024-09-12T13:28:26.668 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo md5sum /dev/rbd7 2024-09-12T13:28:26.670 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ awk '{print $1}' 2024-09-12T13:28:27.211 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Image promoted to primary 2024-09-12T13:28:27.217 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Image promoted to primary 2024-09-12T13:28:27.256 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \n\b\d ]] 2024-09-12T13:28:27.256 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \k\r\b\d ]] 2024-09-12T13:28:27.256 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \n\b\d ]] 2024-09-12T13:28:27.257 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \k\r\b\d ]] 2024-09-12T13:28:27.261 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo rbd --cluster cluster2 device map -t krbd mirror/image-primary7 2024-09-12T13:28:27.261 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo rbd --cluster cluster2 device map -t krbd mirror/image-primary8 2024-09-12T13:28:27.640 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ dev=/dev/rbd8 2024-09-12T13:28:27.645 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo md5sum /dev/rbd8 2024-09-12T13:28:27.646 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ awk '{print $1}' 2024-09-12T13:28:27.684 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ dev=/dev/rbd9 2024-09-12T13:28:27.692 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ awk '{print $1}' 2024-09-12T13:28:27.692 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo md5sum /dev/rbd9 2024-09-12T13:29:12.685 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ promote_md5=e88d359832a0849246f40e77fa6e5870 2024-09-12T13:29:12.685 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo rbd --cluster cluster2 device unmap -t krbd /dev/rbd0 2024-09-12T13:29:13.378 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ e88d359832a0849246f40e77fa6e5870 != \e\8\8\d\3\5\9\8\3\2\a\0\8\4\9\2\4\6\f\4\0\e\7\7\f\a\6\e\5\8\7\0 ]] 2024-09-12T13:29:13.381 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${COMPARE_PIDS[@]} 2024-09-12T13:29:13.382 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 78171 2024-09-12T13:29:16.220 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ promote_md5=6ede1b368bc41826fdc31adefbefd08f 2024-09-12T13:29:16.220 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo rbd --cluster cluster2 device unmap -t krbd /dev/rbd1 2024-09-12T13:29:16.225 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ promote_md5=1a0482490c1e719730ddfb773845098b 2024-09-12T13:29:16.225 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo rbd --cluster cluster2 device unmap -t krbd /dev/rbd3 2024-09-12T13:29:16.398 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ 6ede1b368bc41826fdc31adefbefd08f != \6\e\d\e\1\b\3\6\8\b\c\4\1\8\2\6\f\d\c\3\1\a\d\e\f\b\e\f\d\0\8\f ]] 2024-09-12T13:29:16.466 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ 1a0482490c1e719730ddfb773845098b != \1\a\0\4\8\2\4\9\0\c\1\e\7\1\9\7\3\0\d\d\f\b\7\7\3\8\4\5\0\9\8\b ]] 2024-09-12T13:29:17.859 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ promote_md5=848c7a0b784bb075592ada3d3d2004b4 2024-09-12T13:29:17.859 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo rbd --cluster cluster2 device unmap -t krbd /dev/rbd2 2024-09-12T13:29:18.027 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ 848c7a0b784bb075592ada3d3d2004b4 != \8\4\8\c\7\a\0\b\7\8\4\b\b\0\7\5\5\9\2\a\d\a\3\d\3\d\2\0\0\4\b\4 ]] 2024-09-12T13:29:19.148 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ promote_md5=90feddaf5dcc21646e6bfa8fc08939f0 2024-09-12T13:29:19.148 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo rbd --cluster cluster2 device unmap -t krbd /dev/rbd7 2024-09-12T13:29:19.300 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ promote_md5=767f6012e29e50caa1e86b4e499a2480 2024-09-12T13:29:19.301 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo rbd --cluster cluster2 device unmap -t krbd /dev/rbd4 2024-09-12T13:29:19.346 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ 90feddaf5dcc21646e6bfa8fc08939f0 != \9\0\f\e\d\d\a\f\5\d\c\c\2\1\6\4\6\e\6\b\f\a\8\f\c\0\8\9\3\9\f\0 ]] 2024-09-12T13:29:19.364 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ promote_md5=00861b00e10fad231d7844009fbeabd8 2024-09-12T13:29:19.364 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo rbd --cluster cluster2 device unmap -t krbd /dev/rbd6 2024-09-12T13:29:19.488 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ 767f6012e29e50caa1e86b4e499a2480 != \7\6\7\f\6\0\1\2\e\2\9\e\5\0\c\a\a\1\e\8\6\b\4\e\4\9\9\a\2\4\8\0 ]] 2024-09-12T13:29:19.511 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ 00861b00e10fad231d7844009fbeabd8 != \0\0\8\6\1\b\0\0\e\1\0\f\a\d\2\3\1\d\7\8\4\4\0\0\9\f\b\e\a\b\d\8 ]] 2024-09-12T13:29:19.881 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ promote_md5=605d8e891318ba820dc2f8e0910359fd 2024-09-12T13:29:19.881 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo rbd --cluster cluster2 device unmap -t krbd /dev/rbd5 2024-09-12T13:29:20.001 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ 605d8e891318ba820dc2f8e0910359fd != \6\0\5\d\8\e\8\9\1\3\1\8\b\a\8\2\0\d\c\2\f\8\e\0\9\1\0\3\5\9\f\d ]] 2024-09-12T13:29:20.002 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${COMPARE_PIDS[@]} 2024-09-12T13:29:20.002 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 78173 2024-09-12T13:29:20.002 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${COMPARE_PIDS[@]} 2024-09-12T13:29:20.002 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 78175 2024-09-12T13:29:20.002 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${COMPARE_PIDS[@]} 2024-09-12T13:29:20.002 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 78177 2024-09-12T13:29:20.002 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${COMPARE_PIDS[@]} 2024-09-12T13:29:20.002 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 78179 2024-09-12T13:29:20.002 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${COMPARE_PIDS[@]} 2024-09-12T13:29:20.003 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 78181 2024-09-12T13:29:20.484 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ promote_md5=b4bf42224f4523cb9a457ba142738ff5 2024-09-12T13:29:20.484 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo rbd --cluster cluster2 device unmap -t krbd /dev/rbd9 2024-09-12T13:29:20.580 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ b4bf42224f4523cb9a457ba142738ff5 != \b\4\b\f\4\2\2\2\4\f\4\5\2\3\c\b\9\a\4\5\7\b\a\1\4\2\7\3\8\f\f\5 ]] 2024-09-12T13:29:20.580 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${COMPARE_PIDS[@]} 2024-09-12T13:29:20.581 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 78183 2024-09-12T13:29:21.243 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ promote_md5=9e6bd4a425594b0e48c8ce7d86200219 2024-09-12T13:29:21.243 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo rbd --cluster cluster2 device unmap -t krbd /dev/rbd8 2024-09-12T13:29:21.335 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ 9e6bd4a425594b0e48c8ce7d86200219 != \9\e\6\b\d\4\a\4\2\5\5\9\4\b\0\e\4\8\c\8\c\e\7\d\8\6\2\0\0\2\1\9 ]] 2024-09-12T13:29:21.335 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${COMPARE_PIDS[@]} 2024-09-12T13:29:21.335 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 78185 2024-09-12T13:29:21.335 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${COMPARE_PIDS[@]} 2024-09-12T13:29:21.336 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 78186 2024-09-12T13:29:21.336 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ (( RET != 0 )) 2024-09-12T13:29:21.336 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:29:21.336 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ IMG=image-primary1 2024-09-12T13:29:21.336 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_replaying_status_in_pool_dir cluster1 mirror image-primary1 2024-09-12T13:29:21.336 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:29:21.336 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:29:21.336 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:29:21.336 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' snapshot = journal ']' 2024-09-12T13:29:21.336 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary1 up+replaying 2024-09-12T13:29:21.336 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:29:21.336 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:29:21.336 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:29:21.337 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:29:21.337 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:29:21.337 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:29:21.337 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:29:21.337 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:29:22.346 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary1 up+replaying '' '' 2024-09-12T13:29:22.346 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:29:22.346 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:29:22.346 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:29:22.346 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:29:22.347 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:29:22.347 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:29:22.347 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:29:22.347 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:29:22.347 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary1 2024-09-12T13:29:22.426 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary1: 2024-09-12T13:29:22.426 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 8bc720a8-a8a7-45cb-ab9a-5646a2cc4cc4 2024-09-12T13:29:22.426 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+replaying 2024-09-12T13:29:22.426 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: replaying, {"bytes_per_second":3096985.6,"bytes_per_snapshot":61939712.0,"last_snapshot_bytes":61939712,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726147700,"remote_snapshot_timestamp":1726147700,"replay_state":"idle"} 2024-09-12T13:29:22.426 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:29:22.426 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:29:17' 2024-09-12T13:29:22.426 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+replaying' 2024-09-12T13:29:22.428 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+replaying 2024-09-12T13:29:22.429 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:29:22.431 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: replaying, {"bytes_per_second":3096985.6,"bytes_per_snapshot":61939712.0,"last_snapshot_bytes":61939712,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726147700,"remote_snapshot_timestamp":1726147700,"replay_state":"idle"} 2024-09-12T13:29:22.432 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:29:22.432 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+replaying 2024-09-12T13:29:22.432 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:29:22.434 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+replaying 2024-09-12T13:29:22.435 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:29:22.437 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:29:22.438 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:29:22.438 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:29:22.442 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:29:17' 2024-09-12T13:29:22.442 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary1 up+replaying '2024-09-12 13:29:17' 2024-09-12T13:29:22.442 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:29:22.442 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:29:22.442 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:29:22.442 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:29:22.442 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:29:17' 2024-09-12T13:29:22.442 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:29:22.443 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:29:22.443 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-09-12T13:29:22.551 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041934232mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary18bc720a8-a8a7-45cb-ab9a-5646a2cc4cc4up+replayingreplaying, {"bytes_per_second":3096985.6,"bytes_per_snapshot":61939712.0,"last_snapshot_bytes":61939712,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726147700,"remote_snapshot_timestamp":1726147700,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:29:17image-primary10a020c6a3-79f5-4f05-82bf-2803d0febb5cup+replayingreplaying, {"bytes_per_second":3131392.0,"bytes_per_snapshot":62627840.0,"last_snapshot_bytes":62627840,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726147706,"remote_snapshot_timestamp":1726147706,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:29:17image-primary2b9571de3-33f9-41ad-aae9-a4841d1fab32up+replayingreplaying, {"bytes_per_second":3334758.4,"bytes_per_snapshot":66695168.0,"last_snapshot_bytes":66695168,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726147706,"remote_snapshot_timestamp":1726147706,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:29:17image-primary31d6661c9-3d05-4fd7-9ee6-aad6901ddd33up+replayingreplaying, {"bytes_per_second":3421184.0,"bytes_per_snapshot":68423680.0,"last_snapshot_bytes":68423680,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726147705,"remote_snapshot_timestamp":1726147705,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:29:17image-primary4ce400cdd-ba29-4924-9381-6e78dcb9b581up+replayingreplaying, {"bytes_per_second":3337216.0,"bytes_per_snapshot":66744320.0,"last_snapshot_bytes":66744320,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726147703,"remote_snapshot_timestamp":1726147703,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:29:17image-primary51d3a6d29-f46e-4dfe-ba1d-2e1bda9e7bc4up+replayingreplaying, {"bytes_per_second":3345817.6,"bytes_per_snapshot":66916352.0,"last_snapshot_bytes":66916352,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726147703,"remote_snapshot_timestamp":1726147703,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:29:17image-primary6b580eb06-ffeb-44e0-8354-b02afae3f6a4up+replayingreplaying, {"bytes_per_second":3234816.0,"bytes_per_snapshot":64696320.0,"last_snapshot_bytes":64696320,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726147703,"remote_snapshot_timestamp":1726147703,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:29:17image-primary743088135-2ee1-48a6-8f76-ae96116f16b1up+replayingreplaying, {"bytes_per_second":3380633.6,"bytes_per_snapshot":67612672.0,"last_snapshot_bytes":67612672,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726147707,"remote_snapshot_timestamp":1726147707,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:29:17image-primary8e5469b72-dbc8-4a77-9c96-ab995f3cc6f1up+replayingreplaying, {"bytes_per_second":3200204.8,"bytes_per_snapshot":64004096.0,"last_snapshot_bytes":64004096,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726147707,"remote_snapshot_timestamp":1726147707,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:29:17image-primary9cabafe0d-591b-466b-8d57-50e60dd6fa48up+replayingreplaying, {"bytes_per_second":3492249.6,"bytes_per_snapshot":69844992.0,"last_snapshot_bytes":69844992,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726147706,"remote_snapshot_timestamp":1726147706,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:29:17' 2024-09-12T13:29:22.551 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:29:22.552 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary1'\'']/last_update' 2024-09-12T13:29:22.587 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:29:17' 2024-09-12T13:29:22.588 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary1'\'']/state' 2024-09-12T13:29:22.592 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+replaying 2024-09-12T13:29:22.592 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+replaying 2024-09-12T13:29:22.593 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+replaying 2024-09-12T13:29:22.594 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+replaying 2024-09-12T13:29:22.595 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:29:22.595 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:29:22.595 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ remove_image cluster2 mirror image-primary1 2024-09-12T13:29:22.595 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:29:22.595 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:29:22.595 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:29:22.596 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 snap purge mirror/image-primary1 2024-09-12T13:29:22.656 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 rm mirror/image-primary1 2024-09-12T13:29:22.790 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:29:22.790+0000 7f67e4800640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x56055be68d00 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-09-12T13:29:22.793 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:29:22.790+0000 7f67ed400640 -1 rbd::mirror::ImageReplayer: 0x560556cef180 [3/8bc720a8-a8a7-45cb-ab9a-5646a2cc4cc4] operator(): start failed: (85) Interrupted system call should be restarted 2024-09-12T13:29:24.467 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-09-12T13:29:24.473 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_image_removal cluster1 mirror image-primary1 2024-09-12T13:29:24.473 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:29:24.473 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:29:24.473 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:29:24.474 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:29:24.474 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 ls mirror 2024-09-12T13:29:24.474 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep -wq image-primary1 2024-09-12T13:29:24.523 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:29:24.523 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rm -rf test-primary1 2024-09-12T13:29:24.535 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:29:24.535 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ IMG=image-primary2 2024-09-12T13:29:24.535 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_replaying_status_in_pool_dir cluster1 mirror image-primary2 2024-09-12T13:29:24.535 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:29:24.535 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:29:24.535 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:29:24.535 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' snapshot = journal ']' 2024-09-12T13:29:24.535 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary2 up+replaying 2024-09-12T13:29:24.535 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:29:24.535 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:29:24.535 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:29:24.535 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:29:24.536 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:29:24.536 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:29:24.536 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:29:24.536 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:29:25.537 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary2 up+replaying '' '' 2024-09-12T13:29:25.537 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:29:25.537 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:29:25.537 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:29:25.537 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:29:25.538 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:29:25.538 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:29:25.538 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:29:25.538 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:29:25.538 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary2 2024-09-12T13:29:25.619 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary2: 2024-09-12T13:29:25.619 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: b9571de3-33f9-41ad-aae9-a4841d1fab32 2024-09-12T13:29:25.619 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+replaying 2024-09-12T13:29:25.619 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: replaying, {"bytes_per_second":3334758.4,"bytes_per_snapshot":66695168.0,"last_snapshot_bytes":66695168,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726147706,"remote_snapshot_timestamp":1726147706,"replay_state":"idle"} 2024-09-12T13:29:25.619 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:29:25.619 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:29:17' 2024-09-12T13:29:25.619 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+replaying' 2024-09-12T13:29:25.622 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+replaying 2024-09-12T13:29:25.622 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:29:25.624 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: replaying, {"bytes_per_second":3334758.4,"bytes_per_snapshot":66695168.0,"last_snapshot_bytes":66695168,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726147706,"remote_snapshot_timestamp":1726147706,"replay_state":"idle"} 2024-09-12T13:29:25.625 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:29:25.625 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+replaying 2024-09-12T13:29:25.626 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:29:25.628 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+replaying 2024-09-12T13:29:25.628 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:29:25.631 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:29:25.631 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:29:25.632 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:29:25.636 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:29:17' 2024-09-12T13:29:25.636 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary2 up+replaying '2024-09-12 13:29:17' 2024-09-12T13:29:25.636 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:29:25.636 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:29:25.636 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:29:25.636 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:29:25.636 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:29:17' 2024-09-12T13:29:25.636 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:29:25.637 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:29:25.637 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-09-12T13:29:25.744 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING941934232mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary10a020c6a3-79f5-4f05-82bf-2803d0febb5cup+replayingreplaying, {"bytes_per_second":3131392.0,"bytes_per_snapshot":62627840.0,"last_snapshot_bytes":62627840,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726147706,"remote_snapshot_timestamp":1726147706,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:29:17image-primary2b9571de3-33f9-41ad-aae9-a4841d1fab32up+replayingreplaying, {"bytes_per_second":3334758.4,"bytes_per_snapshot":66695168.0,"last_snapshot_bytes":66695168,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726147706,"remote_snapshot_timestamp":1726147706,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:29:17image-primary31d6661c9-3d05-4fd7-9ee6-aad6901ddd33up+replayingreplaying, {"bytes_per_second":3421184.0,"bytes_per_snapshot":68423680.0,"last_snapshot_bytes":68423680,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726147705,"remote_snapshot_timestamp":1726147705,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:29:17image-primary4ce400cdd-ba29-4924-9381-6e78dcb9b581up+replayingreplaying, {"bytes_per_second":3337216.0,"bytes_per_snapshot":66744320.0,"last_snapshot_bytes":66744320,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726147703,"remote_snapshot_timestamp":1726147703,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:29:17image-primary51d3a6d29-f46e-4dfe-ba1d-2e1bda9e7bc4up+replayingreplaying, {"bytes_per_second":3345817.6,"bytes_per_snapshot":66916352.0,"last_snapshot_bytes":66916352,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726147703,"remote_snapshot_timestamp":1726147703,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:29:17image-primary6b580eb06-ffeb-44e0-8354-b02afae3f6a4up+replayingreplaying, {"bytes_per_second":3234816.0,"bytes_per_snapshot":64696320.0,"last_snapshot_bytes":64696320,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726147703,"remote_snapshot_timestamp":1726147703,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:29:17image-primary743088135-2ee1-48a6-8f76-ae96116f16b1up+replayingreplaying, {"bytes_per_second":3380633.6,"bytes_per_snapshot":67612672.0,"last_snapshot_bytes":67612672,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726147707,"remote_snapshot_timestamp":1726147707,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:29:17image-primary8e5469b72-dbc8-4a77-9c96-ab995f3cc6f1up+replayingreplaying, {"bytes_per_second":3200204.8,"bytes_per_snapshot":64004096.0,"last_snapshot_bytes":64004096,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726147707,"remote_snapshot_timestamp":1726147707,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:29:17image-primary9cabafe0d-591b-466b-8d57-50e60dd6fa48up+replayingreplaying, {"bytes_per_second":3492249.6,"bytes_per_snapshot":69844992.0,"last_snapshot_bytes":69844992,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726147706,"remote_snapshot_timestamp":1726147706,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:29:17' 2024-09-12T13:29:25.745 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:29:25.745 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary2'\'']/last_update' 2024-09-12T13:29:25.749 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:29:17' 2024-09-12T13:29:25.750 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary2'\'']/state' 2024-09-12T13:29:25.753 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+replaying 2024-09-12T13:29:25.754 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+replaying 2024-09-12T13:29:25.754 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+replaying 2024-09-12T13:29:25.756 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+replaying 2024-09-12T13:29:25.757 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:29:25.757 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:29:25.757 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ remove_image cluster2 mirror image-primary2 2024-09-12T13:29:25.757 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:29:25.757 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:29:25.757 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:29:25.757 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 snap purge mirror/image-primary2 2024-09-12T13:29:25.817 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 rm mirror/image-primary2 2024-09-12T13:29:25.955 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:29:25.955+0000 7f67e5200640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x5605565829c0 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-09-12T13:29:25.955 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:29:25.955+0000 7f67eca00640 -1 rbd::mirror::ImageReplayer: 0x5605573ca000 [3/b9571de3-33f9-41ad-aae9-a4841d1fab32] operator(): start failed: (85) Interrupted system call should be restarted 2024-09-12T13:29:27.486 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-09-12T13:29:27.492 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_image_removal cluster1 mirror image-primary2 2024-09-12T13:29:27.492 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:29:27.492 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:29:27.492 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:29:27.493 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:29:27.493 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 ls mirror 2024-09-12T13:29:27.493 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep -wq image-primary2 2024-09-12T13:29:27.543 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:29:27.543 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rm -rf test-primary2 2024-09-12T13:29:27.545 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:29:27.546 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ IMG=image-primary3 2024-09-12T13:29:27.546 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_replaying_status_in_pool_dir cluster1 mirror image-primary3 2024-09-12T13:29:27.546 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:29:27.546 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:29:27.546 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:29:27.546 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' snapshot = journal ']' 2024-09-12T13:29:27.546 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary3 up+replaying 2024-09-12T13:29:27.546 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:29:27.546 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:29:27.546 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:29:27.546 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:29:27.546 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:29:27.546 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:29:27.547 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:29:27.547 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:29:28.548 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary3 up+replaying '' '' 2024-09-12T13:29:28.548 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:29:28.548 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:29:28.548 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:29:28.548 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:29:28.548 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:29:28.549 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:29:28.549 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:29:28.549 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:29:28.549 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary3 2024-09-12T13:29:28.628 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary3: 2024-09-12T13:29:28.628 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 1d6661c9-3d05-4fd7-9ee6-aad6901ddd33 2024-09-12T13:29:28.628 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+replaying 2024-09-12T13:29:28.629 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: replaying, {"bytes_per_second":3421184.0,"bytes_per_snapshot":68423680.0,"last_snapshot_bytes":68423680,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726147705,"remote_snapshot_timestamp":1726147705,"replay_state":"idle"} 2024-09-12T13:29:28.629 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:29:28.629 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:29:17' 2024-09-12T13:29:28.629 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+replaying' 2024-09-12T13:29:28.631 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+replaying 2024-09-12T13:29:28.631 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:29:28.634 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: replaying, {"bytes_per_second":3421184.0,"bytes_per_snapshot":68423680.0,"last_snapshot_bytes":68423680,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726147705,"remote_snapshot_timestamp":1726147705,"replay_state":"idle"} 2024-09-12T13:29:28.634 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:29:28.635 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+replaying 2024-09-12T13:29:28.635 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:29:28.637 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+replaying 2024-09-12T13:29:28.637 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:29:28.640 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:29:28.640 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:29:28.641 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:29:28.645 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:29:17' 2024-09-12T13:29:28.645 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary3 up+replaying '2024-09-12 13:29:17' 2024-09-12T13:29:28.646 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:29:28.646 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:29:28.646 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:29:28.646 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:29:28.646 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:29:17' 2024-09-12T13:29:28.646 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:29:28.646 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:29:28.646 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-09-12T13:29:28.753 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING841934232mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary10a020c6a3-79f5-4f05-82bf-2803d0febb5cup+replayingreplaying, {"bytes_per_second":3131392.0,"bytes_per_snapshot":62627840.0,"last_snapshot_bytes":62627840,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726147706,"remote_snapshot_timestamp":1726147706,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:29:17image-primary31d6661c9-3d05-4fd7-9ee6-aad6901ddd33up+replayingreplaying, {"bytes_per_second":3421184.0,"bytes_per_snapshot":68423680.0,"last_snapshot_bytes":68423680,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726147705,"remote_snapshot_timestamp":1726147705,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:29:17image-primary4ce400cdd-ba29-4924-9381-6e78dcb9b581up+replayingreplaying, {"bytes_per_second":3337216.0,"bytes_per_snapshot":66744320.0,"last_snapshot_bytes":66744320,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726147703,"remote_snapshot_timestamp":1726147703,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:29:17image-primary51d3a6d29-f46e-4dfe-ba1d-2e1bda9e7bc4up+replayingreplaying, {"bytes_per_second":3345817.6,"bytes_per_snapshot":66916352.0,"last_snapshot_bytes":66916352,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726147703,"remote_snapshot_timestamp":1726147703,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:29:17image-primary6b580eb06-ffeb-44e0-8354-b02afae3f6a4up+replayingreplaying, {"bytes_per_second":3234816.0,"bytes_per_snapshot":64696320.0,"last_snapshot_bytes":64696320,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726147703,"remote_snapshot_timestamp":1726147703,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:29:17image-primary743088135-2ee1-48a6-8f76-ae96116f16b1up+replayingreplaying, {"bytes_per_second":3380633.6,"bytes_per_snapshot":67612672.0,"last_snapshot_bytes":67612672,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726147707,"remote_snapshot_timestamp":1726147707,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:29:17image-primary8e5469b72-dbc8-4a77-9c96-ab995f3cc6f1up+replayingreplaying, {"bytes_per_second":3200204.8,"bytes_per_snapshot":64004096.0,"last_snapshot_bytes":64004096,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726147707,"remote_snapshot_timestamp":1726147707,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:29:17image-primary9cabafe0d-591b-466b-8d57-50e60dd6fa48up+replayingreplaying, {"bytes_per_second":3492249.6,"bytes_per_snapshot":69844992.0,"last_snapshot_bytes":69844992,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726147706,"remote_snapshot_timestamp":1726147706,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:29:17' 2024-09-12T13:29:28.753 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:29:28.753 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary3'\'']/last_update' 2024-09-12T13:29:28.757 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:29:17' 2024-09-12T13:29:28.758 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary3'\'']/state' 2024-09-12T13:29:28.761 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+replaying 2024-09-12T13:29:28.762 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+replaying 2024-09-12T13:29:28.762 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+replaying 2024-09-12T13:29:28.764 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+replaying 2024-09-12T13:29:28.765 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:29:28.765 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:29:28.765 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ remove_image cluster2 mirror image-primary3 2024-09-12T13:29:28.765 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:29:28.765 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:29:28.765 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:29:28.766 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 snap purge mirror/image-primary3 2024-09-12T13:29:28.826 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 rm mirror/image-primary3 2024-09-12T13:29:28.966 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:29:28.966+0000 7f67e4800640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x5605557d5380 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-09-12T13:29:28.967 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:29:28.966+0000 7f67eca00640 -1 rbd::mirror::ImageReplayer: 0x56055c199180 [3/1d6661c9-3d05-4fd7-9ee6-aad6901ddd33] operator(): start failed: (85) Interrupted system call should be restarted 2024-09-12T13:29:30.086 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-09-12T13:29:30.091 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_image_removal cluster1 mirror image-primary3 2024-09-12T13:29:30.092 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:29:30.092 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:29:30.092 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:29:30.092 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:29:30.092 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 ls mirror 2024-09-12T13:29:30.092 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep -wq image-primary3 2024-09-12T13:29:30.152 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:29:30.152 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rm -rf test-primary3 2024-09-12T13:29:30.155 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:29:30.155 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ IMG=image-primary4 2024-09-12T13:29:30.156 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_replaying_status_in_pool_dir cluster1 mirror image-primary4 2024-09-12T13:29:30.156 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:29:30.156 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:29:30.156 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:29:30.156 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' snapshot = journal ']' 2024-09-12T13:29:30.157 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary4 up+replaying 2024-09-12T13:29:30.157 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:29:30.157 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:29:30.157 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:29:30.157 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:29:30.157 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:29:30.158 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:29:30.158 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:29:30.158 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:29:31.156 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary4 up+replaying '' '' 2024-09-12T13:29:31.157 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:29:31.157 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:29:31.157 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:29:31.157 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:29:31.157 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:29:31.157 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:29:31.157 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:29:31.157 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:29:31.158 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary4 2024-09-12T13:29:31.229 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary4: 2024-09-12T13:29:31.229 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: ce400cdd-ba29-4924-9381-6e78dcb9b581 2024-09-12T13:29:31.229 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+replaying 2024-09-12T13:29:31.229 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: replaying, {"bytes_per_second":3337216.0,"bytes_per_snapshot":66744320.0,"last_snapshot_bytes":66744320,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726147703,"remote_snapshot_timestamp":1726147703,"replay_state":"idle"} 2024-09-12T13:29:31.229 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:29:31.229 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:29:17' 2024-09-12T13:29:31.229 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+replaying' 2024-09-12T13:29:31.231 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+replaying 2024-09-12T13:29:31.231 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:29:31.234 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: replaying, {"bytes_per_second":3337216.0,"bytes_per_snapshot":66744320.0,"last_snapshot_bytes":66744320,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726147703,"remote_snapshot_timestamp":1726147703,"replay_state":"idle"} 2024-09-12T13:29:31.234 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:29:31.235 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+replaying 2024-09-12T13:29:31.235 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:29:31.237 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+replaying 2024-09-12T13:29:31.237 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:29:31.240 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:29:31.240 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:29:31.241 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:29:31.245 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:29:17' 2024-09-12T13:29:31.245 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary4 up+replaying '2024-09-12 13:29:17' 2024-09-12T13:29:31.245 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:29:31.245 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:29:31.245 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:29:31.245 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:29:31.245 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:29:17' 2024-09-12T13:29:31.245 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:29:31.245 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:29:31.246 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-09-12T13:29:31.341 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING741934232mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary10a020c6a3-79f5-4f05-82bf-2803d0febb5cup+replayingreplaying, {"bytes_per_second":3131392.0,"bytes_per_snapshot":62627840.0,"last_snapshot_bytes":62627840,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726147706,"remote_snapshot_timestamp":1726147706,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:29:17image-primary4ce400cdd-ba29-4924-9381-6e78dcb9b581up+replayingreplaying, {"bytes_per_second":3337216.0,"bytes_per_snapshot":66744320.0,"last_snapshot_bytes":66744320,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726147703,"remote_snapshot_timestamp":1726147703,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:29:17image-primary51d3a6d29-f46e-4dfe-ba1d-2e1bda9e7bc4up+replayingreplaying, {"bytes_per_second":3345817.6,"bytes_per_snapshot":66916352.0,"last_snapshot_bytes":66916352,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726147703,"remote_snapshot_timestamp":1726147703,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:29:17image-primary6b580eb06-ffeb-44e0-8354-b02afae3f6a4up+replayingreplaying, {"bytes_per_second":3234816.0,"bytes_per_snapshot":64696320.0,"last_snapshot_bytes":64696320,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726147703,"remote_snapshot_timestamp":1726147703,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:29:17image-primary743088135-2ee1-48a6-8f76-ae96116f16b1up+replayingreplaying, {"bytes_per_second":3380633.6,"bytes_per_snapshot":67612672.0,"last_snapshot_bytes":67612672,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726147707,"remote_snapshot_timestamp":1726147707,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:29:17image-primary8e5469b72-dbc8-4a77-9c96-ab995f3cc6f1up+replayingreplaying, {"bytes_per_second":3200204.8,"bytes_per_snapshot":64004096.0,"last_snapshot_bytes":64004096,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726147707,"remote_snapshot_timestamp":1726147707,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:29:17image-primary9cabafe0d-591b-466b-8d57-50e60dd6fa48up+replayingreplaying, {"bytes_per_second":3492249.6,"bytes_per_snapshot":69844992.0,"last_snapshot_bytes":69844992,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726147706,"remote_snapshot_timestamp":1726147706,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:29:17' 2024-09-12T13:29:31.341 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:29:31.341 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary4'\'']/last_update' 2024-09-12T13:29:31.345 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:29:17' 2024-09-12T13:29:31.346 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary4'\'']/state' 2024-09-12T13:29:31.350 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+replaying 2024-09-12T13:29:31.350 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+replaying 2024-09-12T13:29:31.351 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+replaying 2024-09-12T13:29:31.353 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+replaying 2024-09-12T13:29:31.353 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:29:31.353 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:29:31.353 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ remove_image cluster2 mirror image-primary4 2024-09-12T13:29:31.354 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:29:31.354 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:29:31.354 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:29:31.354 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 snap purge mirror/image-primary4 2024-09-12T13:29:31.415 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 rm mirror/image-primary4 2024-09-12T13:29:31.548 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:29:31.548+0000 7f67e5200640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x5605557d5a00 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-09-12T13:29:31.548 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:29:31.548+0000 7f67eca00640 -1 rbd::mirror::ImageReplayer: 0x5605573cbb80 [3/ce400cdd-ba29-4924-9381-6e78dcb9b581] operator(): start failed: (85) Interrupted system call should be restarted 2024-09-12T13:29:33.102 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-09-12T13:29:33.108 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_image_removal cluster1 mirror image-primary4 2024-09-12T13:29:33.108 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:29:33.108 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:29:33.108 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:29:33.108 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:29:33.109 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 ls mirror 2024-09-12T13:29:33.109 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep -wq image-primary4 2024-09-12T13:29:33.160 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:29:33.160 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rm -rf test-primary4 2024-09-12T13:29:33.162 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:29:33.162 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ IMG=image-primary5 2024-09-12T13:29:33.162 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_replaying_status_in_pool_dir cluster1 mirror image-primary5 2024-09-12T13:29:33.162 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:29:33.162 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:29:33.162 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:29:33.162 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' snapshot = journal ']' 2024-09-12T13:29:33.162 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary5 up+replaying 2024-09-12T13:29:33.162 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:29:33.162 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:29:33.162 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:29:33.163 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:29:33.163 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:29:33.163 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:29:33.163 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:29:33.163 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:29:34.163 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary5 up+replaying '' '' 2024-09-12T13:29:34.164 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:29:34.164 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:29:34.164 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:29:34.164 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:29:34.164 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:29:34.164 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:29:34.164 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:29:34.164 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:29:34.165 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary5 2024-09-12T13:29:34.247 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary5: 2024-09-12T13:29:34.247 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 1d3a6d29-f46e-4dfe-ba1d-2e1bda9e7bc4 2024-09-12T13:29:34.247 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+replaying 2024-09-12T13:29:34.247 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: replaying, {"bytes_per_second":3345817.6,"bytes_per_snapshot":66916352.0,"last_snapshot_bytes":66916352,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726147703,"remote_snapshot_timestamp":1726147703,"replay_state":"idle"} 2024-09-12T13:29:34.247 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:29:34.247 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:29:17' 2024-09-12T13:29:34.247 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+replaying' 2024-09-12T13:29:34.249 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+replaying 2024-09-12T13:29:34.250 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:29:34.253 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: replaying, {"bytes_per_second":3345817.6,"bytes_per_snapshot":66916352.0,"last_snapshot_bytes":66916352,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726147703,"remote_snapshot_timestamp":1726147703,"replay_state":"idle"} 2024-09-12T13:29:34.253 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:29:34.253 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+replaying 2024-09-12T13:29:34.254 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:29:34.256 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+replaying 2024-09-12T13:29:34.256 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:29:34.259 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:29:34.259 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:29:34.260 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:29:34.264 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:29:17' 2024-09-12T13:29:34.264 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary5 up+replaying '2024-09-12 13:29:17' 2024-09-12T13:29:34.264 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:29:34.264 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:29:34.264 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:29:34.264 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:29:34.264 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:29:17' 2024-09-12T13:29:34.264 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:29:34.265 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:29:34.265 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-09-12T13:29:34.363 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING641934232mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary10a020c6a3-79f5-4f05-82bf-2803d0febb5cup+replayingreplaying, {"bytes_per_second":3131392.0,"bytes_per_snapshot":62627840.0,"last_snapshot_bytes":62627840,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726147706,"remote_snapshot_timestamp":1726147706,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:29:17image-primary51d3a6d29-f46e-4dfe-ba1d-2e1bda9e7bc4up+replayingreplaying, {"bytes_per_second":3345817.6,"bytes_per_snapshot":66916352.0,"last_snapshot_bytes":66916352,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726147703,"remote_snapshot_timestamp":1726147703,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:29:17image-primary6b580eb06-ffeb-44e0-8354-b02afae3f6a4up+replayingreplaying, {"bytes_per_second":3234816.0,"bytes_per_snapshot":64696320.0,"last_snapshot_bytes":64696320,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726147703,"remote_snapshot_timestamp":1726147703,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:29:17image-primary743088135-2ee1-48a6-8f76-ae96116f16b1up+replayingreplaying, {"bytes_per_second":3380633.6,"bytes_per_snapshot":67612672.0,"last_snapshot_bytes":67612672,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726147707,"remote_snapshot_timestamp":1726147707,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:29:17image-primary8e5469b72-dbc8-4a77-9c96-ab995f3cc6f1up+replayingreplaying, {"bytes_per_second":3200204.8,"bytes_per_snapshot":64004096.0,"last_snapshot_bytes":64004096,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726147707,"remote_snapshot_timestamp":1726147707,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:29:17image-primary9cabafe0d-591b-466b-8d57-50e60dd6fa48up+replayingreplaying, {"bytes_per_second":3492249.6,"bytes_per_snapshot":69844992.0,"last_snapshot_bytes":69844992,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726147706,"remote_snapshot_timestamp":1726147706,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:29:17' 2024-09-12T13:29:34.363 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:29:34.364 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary5'\'']/last_update' 2024-09-12T13:29:34.367 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:29:17' 2024-09-12T13:29:34.368 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary5'\'']/state' 2024-09-12T13:29:34.372 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+replaying 2024-09-12T13:29:34.373 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+replaying 2024-09-12T13:29:34.373 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+replaying 2024-09-12T13:29:34.375 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+replaying 2024-09-12T13:29:34.376 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:29:34.376 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:29:34.376 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ remove_image cluster2 mirror image-primary5 2024-09-12T13:29:34.376 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:29:34.376 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:29:34.376 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:29:34.376 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 snap purge mirror/image-primary5 2024-09-12T13:29:34.438 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 rm mirror/image-primary5 2024-09-12T13:29:34.575 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:29:34.575+0000 7f67e5200640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x56055ba3f1e0 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-09-12T13:29:34.575 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:29:34.575+0000 7f67ed400640 -1 rbd::mirror::ImageReplayer: 0x5605573cb900 [3/1d3a6d29-f46e-4dfe-ba1d-2e1bda9e7bc4] operator(): start failed: (85) Interrupted system call should be restarted 2024-09-12T13:29:36.107 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-09-12T13:29:36.113 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_image_removal cluster1 mirror image-primary5 2024-09-12T13:29:36.113 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:29:36.113 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:29:36.113 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:29:36.113 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:29:36.114 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 ls mirror 2024-09-12T13:29:36.114 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep -wq image-primary5 2024-09-12T13:29:36.162 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:29:36.162 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rm -rf test-primary5 2024-09-12T13:29:36.165 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:29:36.165 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ IMG=image-primary6 2024-09-12T13:29:36.165 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_replaying_status_in_pool_dir cluster1 mirror image-primary6 2024-09-12T13:29:36.165 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:29:36.165 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:29:36.165 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:29:36.165 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' snapshot = journal ']' 2024-09-12T13:29:36.165 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary6 up+replaying 2024-09-12T13:29:36.165 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:29:36.165 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:29:36.165 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:29:36.166 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:29:36.166 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:29:36.166 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:29:36.166 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:29:36.166 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:29:37.167 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary6 up+replaying '' '' 2024-09-12T13:29:37.167 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:29:37.167 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:29:37.167 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:29:37.168 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:29:37.168 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:29:37.168 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:29:37.168 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:29:37.168 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:29:37.168 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary6 2024-09-12T13:29:37.279 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary6: 2024-09-12T13:29:37.279 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: b580eb06-ffeb-44e0-8354-b02afae3f6a4 2024-09-12T13:29:37.279 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+replaying 2024-09-12T13:29:37.279 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: replaying, {"bytes_per_second":3234816.0,"bytes_per_snapshot":64696320.0,"last_snapshot_bytes":64696320,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726147703,"remote_snapshot_timestamp":1726147703,"replay_state":"idle"} 2024-09-12T13:29:37.279 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:29:37.279 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:29:17' 2024-09-12T13:29:37.279 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+replaying' 2024-09-12T13:29:37.281 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+replaying 2024-09-12T13:29:37.282 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:29:37.284 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: replaying, {"bytes_per_second":3234816.0,"bytes_per_snapshot":64696320.0,"last_snapshot_bytes":64696320,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726147703,"remote_snapshot_timestamp":1726147703,"replay_state":"idle"} 2024-09-12T13:29:37.284 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:29:37.285 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+replaying 2024-09-12T13:29:37.285 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:29:37.287 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+replaying 2024-09-12T13:29:37.288 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:29:37.290 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:29:37.291 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:29:37.291 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:29:37.296 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:29:17' 2024-09-12T13:29:37.296 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary6 up+replaying '2024-09-12 13:29:17' 2024-09-12T13:29:37.296 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:29:37.296 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:29:37.296 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:29:37.296 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:29:37.296 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:29:17' 2024-09-12T13:29:37.296 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:29:37.297 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:29:37.297 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-09-12T13:29:37.427 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING541934232mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary10a020c6a3-79f5-4f05-82bf-2803d0febb5cup+replayingreplaying, {"bytes_per_second":3131392.0,"bytes_per_snapshot":62627840.0,"last_snapshot_bytes":62627840,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726147706,"remote_snapshot_timestamp":1726147706,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:29:17image-primary6b580eb06-ffeb-44e0-8354-b02afae3f6a4up+replayingreplaying, {"bytes_per_second":3234816.0,"bytes_per_snapshot":64696320.0,"last_snapshot_bytes":64696320,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726147703,"remote_snapshot_timestamp":1726147703,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:29:17image-primary743088135-2ee1-48a6-8f76-ae96116f16b1up+replayingreplaying, {"bytes_per_second":3380633.6,"bytes_per_snapshot":67612672.0,"last_snapshot_bytes":67612672,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726147707,"remote_snapshot_timestamp":1726147707,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:29:17image-primary8e5469b72-dbc8-4a77-9c96-ab995f3cc6f1up+replayingreplaying, {"bytes_per_second":3200204.8,"bytes_per_snapshot":64004096.0,"last_snapshot_bytes":64004096,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726147707,"remote_snapshot_timestamp":1726147707,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:29:17image-primary9cabafe0d-591b-466b-8d57-50e60dd6fa48up+replayingreplaying, {"bytes_per_second":3492249.6,"bytes_per_snapshot":69844992.0,"last_snapshot_bytes":69844992,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726147706,"remote_snapshot_timestamp":1726147706,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:29:17' 2024-09-12T13:29:37.427 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:29:37.427 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary6'\'']/last_update' 2024-09-12T13:29:37.431 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:29:17' 2024-09-12T13:29:37.432 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary6'\'']/state' 2024-09-12T13:29:37.436 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+replaying 2024-09-12T13:29:37.437 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+replaying 2024-09-12T13:29:37.437 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+replaying 2024-09-12T13:29:37.439 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+replaying 2024-09-12T13:29:37.440 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:29:37.440 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:29:37.440 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ remove_image cluster2 mirror image-primary6 2024-09-12T13:29:37.440 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:29:37.440 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:29:37.440 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:29:37.440 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 snap purge mirror/image-primary6 2024-09-12T13:29:37.502 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 rm mirror/image-primary6 2024-09-12T13:29:37.702 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:29:37.702+0000 7f67e5200640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x5605557d5ba0 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-09-12T13:29:37.702 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:29:37.702+0000 7f67ed400640 -1 rbd::mirror::ImageReplayer: 0x5605561acf00 [3/b580eb06-ffeb-44e0-8354-b02afae3f6a4] operator(): start failed: (85) Interrupted system call should be restarted 2024-09-12T13:29:39.104 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-09-12T13:29:39.110 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_image_removal cluster1 mirror image-primary6 2024-09-12T13:29:39.110 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:29:39.110 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:29:39.110 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:29:39.110 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:29:39.111 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 ls mirror 2024-09-12T13:29:39.111 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep -wq image-primary6 2024-09-12T13:29:39.159 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:29:39.159 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rm -rf test-primary6 2024-09-12T13:29:39.162 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:29:39.162 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ IMG=image-primary7 2024-09-12T13:29:39.162 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_replaying_status_in_pool_dir cluster1 mirror image-primary7 2024-09-12T13:29:39.162 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:29:39.162 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:29:39.162 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:29:39.162 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' snapshot = journal ']' 2024-09-12T13:29:39.163 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary7 up+replaying 2024-09-12T13:29:39.163 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:29:39.163 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:29:39.163 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:29:39.163 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:29:39.163 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:29:39.163 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:29:39.163 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:29:39.163 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:29:40.164 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary7 up+replaying '' '' 2024-09-12T13:29:40.165 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:29:40.165 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:29:40.165 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:29:40.165 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:29:40.165 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:29:40.165 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:29:40.165 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:29:40.165 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:29:40.165 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary7 2024-09-12T13:29:40.242 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary7: 2024-09-12T13:29:40.243 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 43088135-2ee1-48a6-8f76-ae96116f16b1 2024-09-12T13:29:40.243 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+replaying 2024-09-12T13:29:40.243 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: replaying, {"bytes_per_second":3380633.6,"bytes_per_snapshot":67612672.0,"last_snapshot_bytes":67612672,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726147707,"remote_snapshot_timestamp":1726147707,"replay_state":"idle"} 2024-09-12T13:29:40.243 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:29:40.243 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:29:17' 2024-09-12T13:29:40.243 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+replaying' 2024-09-12T13:29:40.245 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+replaying 2024-09-12T13:29:40.245 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:29:40.248 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: replaying, {"bytes_per_second":3380633.6,"bytes_per_snapshot":67612672.0,"last_snapshot_bytes":67612672,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726147707,"remote_snapshot_timestamp":1726147707,"replay_state":"idle"} 2024-09-12T13:29:40.248 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:29:40.249 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+replaying 2024-09-12T13:29:40.249 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:29:40.251 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+replaying 2024-09-12T13:29:40.252 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:29:40.254 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:29:40.255 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:29:40.255 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:29:40.259 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:29:17' 2024-09-12T13:29:40.260 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary7 up+replaying '2024-09-12 13:29:17' 2024-09-12T13:29:40.260 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:29:40.260 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:29:40.260 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:29:40.260 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:29:40.260 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:29:17' 2024-09-12T13:29:40.260 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:29:40.260 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:29:40.261 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-09-12T13:29:40.349 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING441934232mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary10a020c6a3-79f5-4f05-82bf-2803d0febb5cup+replayingreplaying, {"bytes_per_second":3131392.0,"bytes_per_snapshot":62627840.0,"last_snapshot_bytes":62627840,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726147706,"remote_snapshot_timestamp":1726147706,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:29:17image-primary743088135-2ee1-48a6-8f76-ae96116f16b1up+replayingreplaying, {"bytes_per_second":3380633.6,"bytes_per_snapshot":67612672.0,"last_snapshot_bytes":67612672,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726147707,"remote_snapshot_timestamp":1726147707,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:29:17image-primary8e5469b72-dbc8-4a77-9c96-ab995f3cc6f1up+replayingreplaying, {"bytes_per_second":3200204.8,"bytes_per_snapshot":64004096.0,"last_snapshot_bytes":64004096,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726147707,"remote_snapshot_timestamp":1726147707,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:29:17image-primary9cabafe0d-591b-466b-8d57-50e60dd6fa48up+replayingreplaying, {"bytes_per_second":3492249.6,"bytes_per_snapshot":69844992.0,"last_snapshot_bytes":69844992,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726147706,"remote_snapshot_timestamp":1726147706,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:29:17' 2024-09-12T13:29:40.349 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:29:40.350 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary7'\'']/last_update' 2024-09-12T13:29:40.354 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:29:17' 2024-09-12T13:29:40.354 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary7'\'']/state' 2024-09-12T13:29:40.358 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+replaying 2024-09-12T13:29:40.358 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+replaying 2024-09-12T13:29:40.359 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+replaying 2024-09-12T13:29:40.361 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+replaying 2024-09-12T13:29:40.362 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:29:40.362 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:29:40.362 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ remove_image cluster2 mirror image-primary7 2024-09-12T13:29:40.362 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:29:40.362 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:29:40.362 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:29:40.362 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 snap purge mirror/image-primary7 2024-09-12T13:29:40.431 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 rm mirror/image-primary7 2024-09-12T13:29:40.573 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:29:40.574+0000 7f67e4800640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x56055b7eed00 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-09-12T13:29:40.574 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:29:40.574+0000 7f67eca00640 -1 rbd::mirror::ImageReplayer: 0x56055c199680 [3/43088135-2ee1-48a6-8f76-ae96116f16b1] operator(): start failed: (85) Interrupted system call should be restarted 2024-09-12T13:29:42.125 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-09-12T13:29:42.131 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_image_removal cluster1 mirror image-primary7 2024-09-12T13:29:42.131 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:29:42.131 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:29:42.131 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:29:42.131 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:29:42.131 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 ls mirror 2024-09-12T13:29:42.132 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep -wq image-primary7 2024-09-12T13:29:42.181 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:29:42.181 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rm -rf test-primary7 2024-09-12T13:29:42.183 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:29:42.183 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ IMG=image-primary8 2024-09-12T13:29:42.184 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_replaying_status_in_pool_dir cluster1 mirror image-primary8 2024-09-12T13:29:42.184 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:29:42.184 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:29:42.184 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:29:42.184 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' snapshot = journal ']' 2024-09-12T13:29:42.184 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary8 up+replaying 2024-09-12T13:29:42.184 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:29:42.184 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:29:42.184 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:29:42.184 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:29:42.184 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:29:42.184 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:29:42.184 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:29:42.184 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:29:43.186 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary8 up+replaying '' '' 2024-09-12T13:29:43.186 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:29:43.186 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:29:43.186 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:29:43.186 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:29:43.186 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:29:43.186 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:29:43.186 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:29:43.187 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:29:43.187 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary8 2024-09-12T13:29:43.271 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary8: 2024-09-12T13:29:43.271 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: e5469b72-dbc8-4a77-9c96-ab995f3cc6f1 2024-09-12T13:29:43.271 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+replaying 2024-09-12T13:29:43.271 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: replaying, {"bytes_per_second":3200204.8,"bytes_per_snapshot":64004096.0,"last_snapshot_bytes":64004096,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726147707,"remote_snapshot_timestamp":1726147707,"replay_state":"idle"} 2024-09-12T13:29:43.271 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:29:43.271 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:29:17' 2024-09-12T13:29:43.271 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+replaying' 2024-09-12T13:29:43.273 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+replaying 2024-09-12T13:29:43.274 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:29:43.276 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: replaying, {"bytes_per_second":3200204.8,"bytes_per_snapshot":64004096.0,"last_snapshot_bytes":64004096,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726147707,"remote_snapshot_timestamp":1726147707,"replay_state":"idle"} 2024-09-12T13:29:43.277 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:29:43.277 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+replaying 2024-09-12T13:29:43.278 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:29:43.280 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+replaying 2024-09-12T13:29:43.280 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:29:43.283 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:29:43.283 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:29:43.284 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:29:43.288 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:29:17' 2024-09-12T13:29:43.288 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary8 up+replaying '2024-09-12 13:29:17' 2024-09-12T13:29:43.288 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:29:43.288 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:29:43.288 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:29:43.288 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:29:43.288 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:29:17' 2024-09-12T13:29:43.288 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:29:43.289 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:29:43.289 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-09-12T13:29:43.374 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING341934232mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary10a020c6a3-79f5-4f05-82bf-2803d0febb5cup+replayingreplaying, {"bytes_per_second":3131392.0,"bytes_per_snapshot":62627840.0,"last_snapshot_bytes":62627840,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726147706,"remote_snapshot_timestamp":1726147706,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:29:17image-primary8e5469b72-dbc8-4a77-9c96-ab995f3cc6f1up+replayingreplaying, {"bytes_per_second":3200204.8,"bytes_per_snapshot":64004096.0,"last_snapshot_bytes":64004096,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726147707,"remote_snapshot_timestamp":1726147707,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:29:17image-primary9cabafe0d-591b-466b-8d57-50e60dd6fa48up+replayingreplaying, {"bytes_per_second":3492249.6,"bytes_per_snapshot":69844992.0,"last_snapshot_bytes":69844992,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726147706,"remote_snapshot_timestamp":1726147706,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:29:17' 2024-09-12T13:29:43.374 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:29:43.374 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary8'\'']/last_update' 2024-09-12T13:29:43.378 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:29:17' 2024-09-12T13:29:43.379 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary8'\'']/state' 2024-09-12T13:29:43.383 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+replaying 2024-09-12T13:29:43.384 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+replaying 2024-09-12T13:29:43.384 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+replaying 2024-09-12T13:29:43.386 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+replaying 2024-09-12T13:29:43.387 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:29:43.387 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:29:43.387 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ remove_image cluster2 mirror image-primary8 2024-09-12T13:29:43.387 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:29:43.387 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:29:43.387 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:29:43.387 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 snap purge mirror/image-primary8 2024-09-12T13:29:43.446 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 rm mirror/image-primary8 2024-09-12T13:29:43.586 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:29:43.587+0000 7f67e4800640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x56055be69040 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-09-12T13:29:43.586 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:29:43.587+0000 7f67eca00640 -1 rbd::mirror::ImageReplayer: 0x56055c198780 [3/e5469b72-dbc8-4a77-9c96-ab995f3cc6f1] operator(): start failed: (85) Interrupted system call should be restarted 2024-09-12T13:29:45.107 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-09-12T13:29:45.113 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_image_removal cluster1 mirror image-primary8 2024-09-12T13:29:45.113 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:29:45.113 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:29:45.113 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:29:45.114 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:29:45.114 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 ls mirror 2024-09-12T13:29:45.114 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep -wq image-primary8 2024-09-12T13:29:45.170 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:29:45.170 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rm -rf test-primary8 2024-09-12T13:29:45.172 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:29:45.172 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ IMG=image-primary9 2024-09-12T13:29:45.173 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_replaying_status_in_pool_dir cluster1 mirror image-primary9 2024-09-12T13:29:45.173 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:29:45.173 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:29:45.173 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:29:45.173 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' snapshot = journal ']' 2024-09-12T13:29:45.173 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary9 up+replaying 2024-09-12T13:29:45.173 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:29:45.173 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:29:45.173 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:29:45.173 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:29:45.173 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:29:45.173 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:29:45.173 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:29:45.173 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:29:46.175 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary9 up+replaying '' '' 2024-09-12T13:29:46.175 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:29:46.175 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:29:46.175 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:29:46.175 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:29:46.175 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:29:46.175 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:29:46.175 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:29:46.176 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:29:46.176 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary9 2024-09-12T13:29:46.270 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary9: 2024-09-12T13:29:46.270 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: cabafe0d-591b-466b-8d57-50e60dd6fa48 2024-09-12T13:29:46.270 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+replaying 2024-09-12T13:29:46.270 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: replaying, {"bytes_per_second":3492249.6,"bytes_per_snapshot":69844992.0,"last_snapshot_bytes":69844992,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726147706,"remote_snapshot_timestamp":1726147706,"replay_state":"idle"} 2024-09-12T13:29:46.270 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:29:46.270 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:29:17' 2024-09-12T13:29:46.270 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+replaying' 2024-09-12T13:29:46.272 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+replaying 2024-09-12T13:29:46.272 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:29:46.275 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: replaying, {"bytes_per_second":3492249.6,"bytes_per_snapshot":69844992.0,"last_snapshot_bytes":69844992,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726147706,"remote_snapshot_timestamp":1726147706,"replay_state":"idle"} 2024-09-12T13:29:46.275 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:29:46.276 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+replaying 2024-09-12T13:29:46.276 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:29:46.278 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+replaying 2024-09-12T13:29:46.278 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:29:46.281 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:29:46.281 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:29:46.282 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:29:46.286 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:29:17' 2024-09-12T13:29:46.286 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary9 up+replaying '2024-09-12 13:29:17' 2024-09-12T13:29:46.286 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:29:46.286 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:29:46.286 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:29:46.286 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:29:46.286 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:29:17' 2024-09-12T13:29:46.286 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:29:46.287 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:29:46.287 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-09-12T13:29:46.411 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING241934232mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary10a020c6a3-79f5-4f05-82bf-2803d0febb5cup+replayingreplaying, {"bytes_per_second":3131392.0,"bytes_per_snapshot":62627840.0,"last_snapshot_bytes":62627840,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726147706,"remote_snapshot_timestamp":1726147706,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:29:17image-primary9cabafe0d-591b-466b-8d57-50e60dd6fa48up+replayingreplaying, {"bytes_per_second":3492249.6,"bytes_per_snapshot":69844992.0,"last_snapshot_bytes":69844992,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726147706,"remote_snapshot_timestamp":1726147706,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:29:17' 2024-09-12T13:29:46.411 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:29:46.411 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary9'\'']/last_update' 2024-09-12T13:29:46.415 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:29:17' 2024-09-12T13:29:46.416 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary9'\'']/state' 2024-09-12T13:29:46.420 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+replaying 2024-09-12T13:29:46.421 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+replaying 2024-09-12T13:29:46.421 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+replaying 2024-09-12T13:29:46.423 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+replaying 2024-09-12T13:29:46.423 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:29:46.424 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:29:46.424 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ remove_image cluster2 mirror image-primary9 2024-09-12T13:29:46.424 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:29:46.424 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:29:46.424 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:29:46.424 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 snap purge mirror/image-primary9 2024-09-12T13:29:46.485 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 rm mirror/image-primary9 2024-09-12T13:29:46.684 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:29:46.685+0000 7f67e4800640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x560556582b60 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-09-12T13:29:46.684 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:29:46.685+0000 7f67ed400640 -1 rbd::mirror::ImageReplayer: 0x56055c198c80 [3/cabafe0d-591b-466b-8d57-50e60dd6fa48] operator(): start failed: (85) Interrupted system call should be restarted 2024-09-12T13:29:48.286 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-09-12T13:29:48.292 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_image_removal cluster1 mirror image-primary9 2024-09-12T13:29:48.292 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:29:48.292 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:29:48.292 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:29:48.292 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:29:48.293 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 ls mirror 2024-09-12T13:29:48.293 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep -wq image-primary9 2024-09-12T13:29:48.342 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:29:48.342 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rm -rf test-primary9 2024-09-12T13:29:48.344 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:29:48.344 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ IMG=image-primary10 2024-09-12T13:29:48.344 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_replaying_status_in_pool_dir cluster1 mirror image-primary10 2024-09-12T13:29:48.345 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:29:48.345 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:29:48.345 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:29:48.345 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' snapshot = journal ']' 2024-09-12T13:29:48.345 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary10 up+replaying 2024-09-12T13:29:48.345 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:29:48.345 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:29:48.345 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:29:48.345 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:29:48.345 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:29:48.345 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:29:48.345 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:29:48.345 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:29:49.347 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary10 up+replaying '' '' 2024-09-12T13:29:49.347 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:29:49.347 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:29:49.347 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:29:49.347 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:29:49.347 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:29:49.347 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:29:49.347 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:29:49.348 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:29:49.348 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary10 2024-09-12T13:29:49.425 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary10: 2024-09-12T13:29:49.426 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: a020c6a3-79f5-4f05-82bf-2803d0febb5c 2024-09-12T13:29:49.426 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+replaying 2024-09-12T13:29:49.426 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: replaying, {"bytes_per_second":0.0,"bytes_per_snapshot":62627840.0,"last_snapshot_bytes":62627840,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726147706,"remote_snapshot_timestamp":1726147706,"replay_state":"idle"} 2024-09-12T13:29:49.426 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:29:49.426 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:29:47' 2024-09-12T13:29:49.426 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+replaying' 2024-09-12T13:29:49.428 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+replaying 2024-09-12T13:29:49.428 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:29:49.431 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: replaying, {"bytes_per_second":0.0,"bytes_per_snapshot":62627840.0,"last_snapshot_bytes":62627840,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726147706,"remote_snapshot_timestamp":1726147706,"replay_state":"idle"} 2024-09-12T13:29:49.431 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:29:49.432 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+replaying 2024-09-12T13:29:49.432 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:29:49.434 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+replaying 2024-09-12T13:29:49.434 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:29:49.437 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:29:49.437 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:29:49.438 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:29:49.442 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:29:47' 2024-09-12T13:29:49.442 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary10 up+replaying '2024-09-12 13:29:47' 2024-09-12T13:29:49.442 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:29:49.442 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:29:49.443 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:29:49.443 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:29:49.443 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:29:47' 2024-09-12T13:29:49.443 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:29:49.443 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:29:49.444 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-09-12T13:29:49.525 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING141934232mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary10a020c6a3-79f5-4f05-82bf-2803d0febb5cup+replayingreplaying, {"bytes_per_second":0.0,"bytes_per_snapshot":62627840.0,"last_snapshot_bytes":62627840,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726147706,"remote_snapshot_timestamp":1726147706,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:29:47' 2024-09-12T13:29:49.525 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:29:49.525 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary10'\'']/last_update' 2024-09-12T13:29:49.529 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:29:47' 2024-09-12T13:29:49.530 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary10'\'']/state' 2024-09-12T13:29:49.533 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+replaying 2024-09-12T13:29:49.534 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+replaying 2024-09-12T13:29:49.534 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+replaying 2024-09-12T13:29:49.536 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+replaying 2024-09-12T13:29:49.537 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:29:49.537 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:29:49.537 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ remove_image cluster2 mirror image-primary10 2024-09-12T13:29:49.537 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:29:49.537 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:29:49.537 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:29:49.538 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 snap purge mirror/image-primary10 2024-09-12T13:29:49.599 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 rm mirror/image-primary10 2024-09-12T13:29:49.742 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:29:49.742+0000 7f67e4800640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x56055ba3e680 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-09-12T13:29:49.742 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:29:49.742+0000 7f67eca00640 -1 rbd::mirror::ImageReplayer: 0x56055c198500 [3/a020c6a3-79f5-4f05-82bf-2803d0febb5c] operator(): start failed: (85) Interrupted system call should be restarted 2024-09-12T13:29:51.129 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-09-12T13:29:51.135 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_image_removal cluster1 mirror image-primary10 2024-09-12T13:29:51.135 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:29:51.135 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:29:51.135 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:29:51.135 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:29:51.136 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 ls mirror 2024-09-12T13:29:51.136 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep -wq image-primary10 2024-09-12T13:29:51.185 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:29:51.185 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rm -rf test-primary10 2024-09-12T13:29:51.188 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..10} 2024-09-12T13:29:51.188 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEVS=() 2024-09-12T13:29:51.189 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ SNAP_PIDS=() 2024-09-12T13:29:51.189 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ COMPARE_PIDS=() 2024-09-12T13:29:51.189 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ WORKLOAD_PIDS=() 2024-09-12T13:29:51.189 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ RET=0 2024-09-12T13:29:51.189 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:29:51.189 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ IMG=image-primary1 2024-09-12T13:29:51.189 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ MNTPT=test-primary1 2024-09-12T13:29:51.189 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ create_image_and_enable_mirror cluster1 mirror image-primary1 snapshot 10G 2024-09-12T13:29:51.189 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:29:51.189 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:29:51.189 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:29:51.189 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:29:51.189 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:29:51.190 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:29:51.190 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mode=snapshot 2024-09-12T13:29:51.190 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n snapshot ']' 2024-09-12T13:29:51.190 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:29:51.190 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ create_image cluster1 mirror image-primary1 10G 2024-09-12T13:29:51.190 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:29:51.190 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:29:51.190 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:29:51.190 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:29:51.190 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:29:51.190 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:29:51.190 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local size=128 2024-09-12T13:29:51.190 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n 10G ']' 2024-09-12T13:29:51.191 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ size=10G 2024-09-12T13:29:51.191 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:29:51.191 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 create --size 10G --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image-primary1 2024-09-12T13:29:51.258 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ is_pool_mirror_mode_image mirror 2024-09-12T13:29:51.258 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:29:51.258 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' image = image ']' 2024-09-12T13:29:51.258 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:29:51.258 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ enable_mirror cluster1 mirror image-primary1 snapshot 2024-09-12T13:29:51.259 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:29:51.259 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:29:51.259 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:29:51.259 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mode=snapshot 2024-09-12T13:29:51.259 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 mirror image enable mirror/image-primary1 snapshot 2024-09-12T13:29:52.056 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Mirroring enabled 2024-09-12T13:29:52.070 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 info mirror/image-primary1 2024-09-12T13:29:52.131 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:rbd image 'image-primary1': 2024-09-12T13:29:52.131 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: size 10 GiB in 2560 objects 2024-09-12T13:29:52.131 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: order 22 (4 MiB objects) 2024-09-12T13:29:52.131 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: snapshot_count: 1 2024-09-12T13:29:52.131 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: id: 2458e8e2e84b 2024-09-12T13:29:52.131 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: block_name_prefix: rbd_data.2458e8e2e84b 2024-09-12T13:29:52.131 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: format: 2 2024-09-12T13:29:52.131 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: features: layering, exclusive-lock, object-map, fast-diff 2024-09-12T13:29:52.132 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: op_features: 2024-09-12T13:29:52.132 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: flags: 2024-09-12T13:29:52.132 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: create_timestamp: Thu Sep 12 13:29:51 2024 2024-09-12T13:29:52.132 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: access_timestamp: Thu Sep 12 13:29:51 2024 2024-09-12T13:29:52.132 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: modify_timestamp: Thu Sep 12 13:29:51 2024 2024-09-12T13:29:52.132 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring state: enabled 2024-09-12T13:29:52.132 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring mode: snapshot 2024-09-12T13:29:52.132 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring global id: 4ee25370-f977-4ecf-acda-dc9a3ae2ee9d 2024-09-12T13:29:52.132 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring primary: true 2024-09-12T13:29:52.137 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \n\b\d ]] 2024-09-12T13:29:52.137 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \k\r\b\d ]] 2024-09-12T13:29:52.137 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo rbd --cluster cluster1 device map -t krbd mirror/image-primary1 2024-09-12T13:29:52.278 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEV=/dev/rbd0 2024-09-12T13:29:52.278 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEVS+=($DEV) 2024-09-12T13:29:52.278 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo mkfs.ext4 /dev/rbd0 2024-09-12T13:29:52.332 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:mke2fs 1.46.5 (30-Dec-2021) 2024-09-12T13:29:52.343 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Discarding device blocks: 0/2621440 done 2024-09-12T13:29:52.343 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Creating filesystem with 2621440 4k blocks and 655360 inodes 2024-09-12T13:29:52.343 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Filesystem UUID: c523a805-1295-4c51-be56-f77f565bd286 2024-09-12T13:29:52.344 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Superblock backups stored on blocks: 2024-09-12T13:29:52.344 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 2024-09-12T13:29:52.344 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 2024-09-12T13:29:52.344 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Allocating group tables: 0/80 done 2024-09-12T13:29:52.344 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Writing inode tables: 0/80 done 2024-09-12T13:29:52.348 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Creating journal (16384 blocks): done 2024-09-12T13:29:52.403 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Writing superblocks and filesystem accounting information: 0/80 done 2024-09-12T13:29:52.403 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 2024-09-12T13:29:52.406 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mkdir test-primary1 2024-09-12T13:29:52.415 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo mount /dev/rbd0 test-primary1 2024-09-12T13:29:52.473 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ whoami 2024-09-12T13:29:52.501 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo chown ubuntu test-primary1 2024-09-12T13:29:52.531 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ SNAP_PIDS+=($!) 2024-09-12T13:29:52.531 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ take_mirror_snapshots cluster1 mirror image-primary1 2024-09-12T13:29:52.531 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:29:52.531 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:29:52.531 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:29:52.532 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:29:52.532 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:29:52.532 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:29:52.532 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:29:52.532 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:29:52.532 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ WORKLOAD_PIDS+=($!) 2024-09-12T13:29:52.532 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:29:52.532 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:29:52.534 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ IMG=image-primary2 2024-09-12T13:29:52.534 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ MNTPT=test-primary2 2024-09-12T13:29:52.534 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ create_image_and_enable_mirror cluster1 mirror image-primary2 snapshot 10G 2024-09-12T13:29:52.534 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ slow_untar_workload test-primary1 2024-09-12T13:29:52.534 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:29:52.534 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:29:52.534 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:29:52.534 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:29:52.534 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:29:52.534 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:29:52.534 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mode=snapshot 2024-09-12T13:29:52.534 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n snapshot ']' 2024-09-12T13:29:52.534 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:29:52.535 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ create_image cluster1 mirror image-primary2 10G 2024-09-12T13:29:52.535 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:29:52.535 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mountpt=test-primary1 2024-09-12T13:29:52.535 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:29:52.535 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:29:52.535 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:29:52.535 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:29:52.535 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:29:52.535 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local size=128 2024-09-12T13:29:52.535 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ cp linux-5.4.tar.gz test-primary1 2024-09-12T13:29:52.535 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n 10G ']' 2024-09-12T13:29:52.535 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ size=10G 2024-09-12T13:29:52.535 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:29:52.535 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 create --size 10G --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image-primary2 2024-09-12T13:29:52.604 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ is_pool_mirror_mode_image mirror 2024-09-12T13:29:52.604 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:29:52.604 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' image = image ']' 2024-09-12T13:29:52.604 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:29:52.605 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ enable_mirror cluster1 mirror image-primary2 snapshot 2024-09-12T13:29:52.605 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:29:52.605 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:29:52.605 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:29:52.605 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mode=snapshot 2024-09-12T13:29:52.605 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 mirror image enable mirror/image-primary2 snapshot 2024-09-12T13:29:52.818 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local ret=0 2024-09-12T13:29:52.818 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ timeout 5m bash -c 'zcat test-primary1/linux-5.4.tar.gz | pv -L 256K | tar xf - -C test-primary1' 2024-09-12T13:29:53.077 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1324 2024-09-12T13:29:53.078 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Mirroring enabled 2024-09-12T13:29:53.095 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:29:53.106 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 info mirror/image-primary2 2024-09-12T13:29:53.176 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:rbd image 'image-primary2': 2024-09-12T13:29:53.176 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: size 10 GiB in 2560 objects 2024-09-12T13:29:53.176 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: order 22 (4 MiB objects) 2024-09-12T13:29:53.176 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: snapshot_count: 1 2024-09-12T13:29:53.176 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: id: 2468ea2a84be 2024-09-12T13:29:53.176 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: block_name_prefix: rbd_data.2468ea2a84be 2024-09-12T13:29:53.176 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: format: 2 2024-09-12T13:29:53.176 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: features: layering, exclusive-lock, object-map, fast-diff 2024-09-12T13:29:53.176 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: op_features: 2024-09-12T13:29:53.176 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: flags: 2024-09-12T13:29:53.177 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: create_timestamp: Thu Sep 12 13:29:52 2024 2024-09-12T13:29:53.177 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: access_timestamp: Thu Sep 12 13:29:52 2024 2024-09-12T13:29:53.177 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: modify_timestamp: Thu Sep 12 13:29:52 2024 2024-09-12T13:29:53.177 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring state: enabled 2024-09-12T13:29:53.177 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring mode: snapshot 2024-09-12T13:29:53.177 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring global id: 6c7d67ca-cef8-4a96-a25c-8d89a4671b00 2024-09-12T13:29:53.177 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring primary: true 2024-09-12T13:29:53.181 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \n\b\d ]] 2024-09-12T13:29:53.181 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \k\r\b\d ]] 2024-09-12T13:29:53.182 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo rbd --cluster cluster1 device map -t krbd mirror/image-primary2 2024-09-12T13:29:53.337 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEV=/dev/rbd1 2024-09-12T13:29:53.337 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEVS+=($DEV) 2024-09-12T13:29:53.338 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo mkfs.ext4 /dev/rbd1 2024-09-12T13:29:53.358 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:mke2fs 1.46.5 (30-Dec-2021) 2024-09-12T13:29:53.369 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Discarding device blocks: 0/2621440 done 2024-09-12T13:29:53.369 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Creating filesystem with 2621440 4k blocks and 655360 inodes 2024-09-12T13:29:53.370 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Filesystem UUID: 3a95cf81-1f27-4102-a5ff-7289dcd25a87 2024-09-12T13:29:53.370 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Superblock backups stored on blocks: 2024-09-12T13:29:53.370 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 2024-09-12T13:29:53.370 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 2024-09-12T13:29:53.370 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Allocating group tables: 0/80 done 2024-09-12T13:29:53.370 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Writing inode tables: 0/80 done 2024-09-12T13:29:53.375 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Creating journal (16384 blocks): done 2024-09-12T13:29:53.464 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Writing superblocks and filesystem accounting information: 0/80 done 2024-09-12T13:29:53.464 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 2024-09-12T13:29:53.465 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mkdir test-primary2 2024-09-12T13:29:53.468 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo mount /dev/rbd1 test-primary2 2024-09-12T13:29:53.550 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ whoami 2024-09-12T13:29:53.552 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo chown ubuntu test-primary2 2024-09-12T13:29:53.576 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ SNAP_PIDS+=($!) 2024-09-12T13:29:53.576 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ take_mirror_snapshots cluster1 mirror image-primary2 2024-09-12T13:29:53.576 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:29:53.577 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:29:53.577 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:29:53.577 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:29:53.577 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:29:53.577 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:29:53.577 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:29:53.577 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:29:53.577 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ WORKLOAD_PIDS+=($!) 2024-09-12T13:29:53.577 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:29:53.577 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:29:53.577 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ IMG=image-primary3 2024-09-12T13:29:53.577 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ MNTPT=test-primary3 2024-09-12T13:29:53.577 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ create_image_and_enable_mirror cluster1 mirror image-primary3 snapshot 10G 2024-09-12T13:29:53.578 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:29:53.579 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:29:53.579 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:29:53.579 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:29:53.579 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:29:53.579 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:29:53.579 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mode=snapshot 2024-09-12T13:29:53.579 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n snapshot ']' 2024-09-12T13:29:53.579 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:29:53.579 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ create_image cluster1 mirror image-primary3 10G 2024-09-12T13:29:53.579 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:29:53.579 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:29:53.579 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:29:53.579 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:29:53.580 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:29:53.580 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ slow_untar_workload test-primary2 2024-09-12T13:29:53.580 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:29:53.580 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local size=128 2024-09-12T13:29:53.580 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n 10G ']' 2024-09-12T13:29:53.580 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ size=10G 2024-09-12T13:29:53.580 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:29:53.580 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 create --size 10G --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image-primary3 2024-09-12T13:29:53.580 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mountpt=test-primary2 2024-09-12T13:29:53.580 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ cp linux-5.4.tar.gz test-primary2 2024-09-12T13:29:53.675 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ is_pool_mirror_mode_image mirror 2024-09-12T13:29:53.675 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:29:53.675 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' image = image ']' 2024-09-12T13:29:53.675 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:29:53.675 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ enable_mirror cluster1 mirror image-primary3 snapshot 2024-09-12T13:29:53.675 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:29:53.675 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:29:53.675 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:29:53.676 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mode=snapshot 2024-09-12T13:29:53.676 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 mirror image enable mirror/image-primary3 snapshot 2024-09-12T13:29:53.885 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local ret=0 2024-09-12T13:29:53.885 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ timeout 5m bash -c 'zcat test-primary2/linux-5.4.tar.gz | pv -L 256K | tar xf - -C test-primary2' 2024-09-12T13:29:54.086 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Mirroring enabled 2024-09-12T13:29:54.089 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1326 2024-09-12T13:29:54.104 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:29:54.115 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 info mirror/image-primary3 2024-09-12T13:29:54.195 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:rbd image 'image-primary3': 2024-09-12T13:29:54.195 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: size 10 GiB in 2560 objects 2024-09-12T13:29:54.195 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: order 22 (4 MiB objects) 2024-09-12T13:29:54.195 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: snapshot_count: 1 2024-09-12T13:29:54.195 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: id: 2475139e560b 2024-09-12T13:29:54.195 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: block_name_prefix: rbd_data.2475139e560b 2024-09-12T13:29:54.195 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: format: 2 2024-09-12T13:29:54.195 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: features: layering, exclusive-lock, object-map, fast-diff 2024-09-12T13:29:54.195 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: op_features: 2024-09-12T13:29:54.195 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: flags: 2024-09-12T13:29:54.196 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: create_timestamp: Thu Sep 12 13:29:53 2024 2024-09-12T13:29:54.196 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: access_timestamp: Thu Sep 12 13:29:53 2024 2024-09-12T13:29:54.196 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: modify_timestamp: Thu Sep 12 13:29:53 2024 2024-09-12T13:29:54.196 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring state: enabled 2024-09-12T13:29:54.196 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring mode: snapshot 2024-09-12T13:29:54.196 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring global id: 9cc4db57-5517-41a1-8d14-519d4043d1a5 2024-09-12T13:29:54.196 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring primary: true 2024-09-12T13:29:54.200 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \n\b\d ]] 2024-09-12T13:29:54.200 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \k\r\b\d ]] 2024-09-12T13:29:54.201 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo rbd --cluster cluster1 device map -t krbd mirror/image-primary3 2024-09-12T13:29:54.351 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEV=/dev/rbd2 2024-09-12T13:29:54.351 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEVS+=($DEV) 2024-09-12T13:29:54.351 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo mkfs.ext4 /dev/rbd2 2024-09-12T13:29:54.373 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:mke2fs 1.46.5 (30-Dec-2021) 2024-09-12T13:29:54.383 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Discarding device blocks: 0/2621440 done 2024-09-12T13:29:54.384 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Creating filesystem with 2621440 4k blocks and 655360 inodes 2024-09-12T13:29:54.384 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Filesystem UUID: 223a379a-f4e5-40ad-bb95-e3451c1f28fb 2024-09-12T13:29:54.384 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Superblock backups stored on blocks: 2024-09-12T13:29:54.384 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 2024-09-12T13:29:54.384 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 2024-09-12T13:29:54.384 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Allocating group tables: 0/80 done 2024-09-12T13:29:54.384 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Writing inode tables: 0/80 done 2024-09-12T13:29:54.389 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Creating journal (16384 blocks): done 2024-09-12T13:29:54.469 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Writing superblocks and filesystem accounting information: 0/80 done 2024-09-12T13:29:54.470 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 2024-09-12T13:29:54.473 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mkdir test-primary3 2024-09-12T13:29:54.475 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo mount /dev/rbd2 test-primary3 2024-09-12T13:29:54.556 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ whoami 2024-09-12T13:29:54.557 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo chown ubuntu test-primary3 2024-09-12T13:29:54.582 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ SNAP_PIDS+=($!) 2024-09-12T13:29:54.582 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ take_mirror_snapshots cluster1 mirror image-primary3 2024-09-12T13:29:54.582 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:29:54.582 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ WORKLOAD_PIDS+=($!) 2024-09-12T13:29:54.582 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:29:54.582 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:29:54.583 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:29:54.583 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:29:54.583 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ IMG=image-primary4 2024-09-12T13:29:54.583 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:29:54.583 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ MNTPT=test-primary4 2024-09-12T13:29:54.583 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ slow_untar_workload test-primary3 2024-09-12T13:29:54.583 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ create_image_and_enable_mirror cluster1 mirror image-primary4 snapshot 10G 2024-09-12T13:29:54.583 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:29:54.583 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:29:54.583 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:29:54.583 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:29:54.583 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:29:54.583 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:29:54.583 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:29:54.584 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:29:54.584 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:29:54.584 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:29:54.584 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mode=snapshot 2024-09-12T13:29:54.584 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n snapshot ']' 2024-09-12T13:29:54.584 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:29:54.584 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ create_image cluster1 mirror image-primary4 10G 2024-09-12T13:29:54.584 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mountpt=test-primary3 2024-09-12T13:29:54.584 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:29:54.584 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:29:54.584 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:29:54.584 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:29:54.584 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ cp linux-5.4.tar.gz test-primary3 2024-09-12T13:29:54.585 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:29:54.585 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:29:54.585 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local size=128 2024-09-12T13:29:54.585 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n 10G ']' 2024-09-12T13:29:54.585 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ size=10G 2024-09-12T13:29:54.585 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:29:54.585 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 create --size 10G --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image-primary4 2024-09-12T13:29:54.685 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ is_pool_mirror_mode_image mirror 2024-09-12T13:29:54.685 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:29:54.685 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' image = image ']' 2024-09-12T13:29:54.685 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:29:54.687 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ enable_mirror cluster1 mirror image-primary4 snapshot 2024-09-12T13:29:54.687 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:29:54.688 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:29:54.688 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:29:54.688 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mode=snapshot 2024-09-12T13:29:54.688 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 mirror image enable mirror/image-primary4 snapshot 2024-09-12T13:29:54.888 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local ret=0 2024-09-12T13:29:54.888 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ timeout 5m bash -c 'zcat test-primary3/linux-5.4.tar.gz | pv -L 256K | tar xf - -C test-primary3' 2024-09-12T13:29:55.050 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Mirroring enabled 2024-09-12T13:29:55.054 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1328 2024-09-12T13:29:55.065 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 info mirror/image-primary4 2024-09-12T13:29:55.065 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:29:55.124 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:rbd image 'image-primary4': 2024-09-12T13:29:55.124 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: size 10 GiB in 2560 objects 2024-09-12T13:29:55.124 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: order 22 (4 MiB objects) 2024-09-12T13:29:55.124 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: snapshot_count: 1 2024-09-12T13:29:55.124 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: id: 2482a97b43bd 2024-09-12T13:29:55.124 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: block_name_prefix: rbd_data.2482a97b43bd 2024-09-12T13:29:55.124 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: format: 2 2024-09-12T13:29:55.124 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: features: layering, exclusive-lock, object-map, fast-diff 2024-09-12T13:29:55.124 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: op_features: 2024-09-12T13:29:55.124 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: flags: 2024-09-12T13:29:55.125 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: create_timestamp: Thu Sep 12 13:29:54 2024 2024-09-12T13:29:55.125 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: access_timestamp: Thu Sep 12 13:29:54 2024 2024-09-12T13:29:55.125 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: modify_timestamp: Thu Sep 12 13:29:54 2024 2024-09-12T13:29:55.125 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring state: enabled 2024-09-12T13:29:55.125 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring mode: snapshot 2024-09-12T13:29:55.125 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring global id: 0203765c-0327-4045-b74a-e9bc90c8f4a0 2024-09-12T13:29:55.125 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring primary: true 2024-09-12T13:29:55.129 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \n\b\d ]] 2024-09-12T13:29:55.129 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \k\r\b\d ]] 2024-09-12T13:29:55.130 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo rbd --cluster cluster1 device map -t krbd mirror/image-primary4 2024-09-12T13:29:55.245 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEV=/dev/rbd3 2024-09-12T13:29:55.245 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEVS+=($DEV) 2024-09-12T13:29:55.245 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo mkfs.ext4 /dev/rbd3 2024-09-12T13:29:55.265 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:mke2fs 1.46.5 (30-Dec-2021) 2024-09-12T13:29:55.278 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Discarding device blocks: 0/2621440 done 2024-09-12T13:29:55.278 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Creating filesystem with 2621440 4k blocks and 655360 inodes 2024-09-12T13:29:55.278 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Filesystem UUID: 1a273960-35c0-40d8-9bb9-d3634b67f444 2024-09-12T13:29:55.278 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Superblock backups stored on blocks: 2024-09-12T13:29:55.278 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 2024-09-12T13:29:55.278 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 2024-09-12T13:29:55.279 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Allocating group tables: 0/80 done 2024-09-12T13:29:55.279 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Writing inode tables: 0/80 done 2024-09-12T13:29:55.283 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Creating journal (16384 blocks): done 2024-09-12T13:29:55.333 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Writing superblocks and filesystem accounting information: 0/80 done 2024-09-12T13:29:55.334 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 2024-09-12T13:29:55.336 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mkdir test-primary4 2024-09-12T13:29:55.339 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo mount /dev/rbd3 test-primary4 2024-09-12T13:29:55.395 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ whoami 2024-09-12T13:29:55.397 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo chown ubuntu test-primary4 2024-09-12T13:29:55.422 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ SNAP_PIDS+=($!) 2024-09-12T13:29:55.423 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ take_mirror_snapshots cluster1 mirror image-primary4 2024-09-12T13:29:55.423 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:29:55.423 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:29:55.423 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:29:55.423 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ WORKLOAD_PIDS+=($!) 2024-09-12T13:29:55.423 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:29:55.423 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:29:55.423 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:29:55.423 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:29:55.423 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:29:55.423 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:29:55.423 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:29:55.424 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ IMG=image-primary5 2024-09-12T13:29:55.424 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ MNTPT=test-primary5 2024-09-12T13:29:55.424 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ create_image_and_enable_mirror cluster1 mirror image-primary5 snapshot 10G 2024-09-12T13:29:55.424 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:29:55.424 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:29:55.424 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:29:55.424 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:29:55.424 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:29:55.424 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:29:55.424 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mode=snapshot 2024-09-12T13:29:55.424 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n snapshot ']' 2024-09-12T13:29:55.424 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:29:55.424 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ create_image cluster1 mirror image-primary5 10G 2024-09-12T13:29:55.425 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ slow_untar_workload test-primary4 2024-09-12T13:29:55.425 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:29:55.425 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:29:55.425 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:29:55.425 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:29:55.425 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:29:55.425 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:29:55.425 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local size=128 2024-09-12T13:29:55.425 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n 10G ']' 2024-09-12T13:29:55.425 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ size=10G 2024-09-12T13:29:55.425 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:29:55.425 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 create --size 10G --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image-primary5 2024-09-12T13:29:55.425 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mountpt=test-primary4 2024-09-12T13:29:55.426 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ cp linux-5.4.tar.gz test-primary4 2024-09-12T13:29:55.495 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ is_pool_mirror_mode_image mirror 2024-09-12T13:29:55.495 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:29:55.496 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' image = image ']' 2024-09-12T13:29:55.496 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:29:55.496 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ enable_mirror cluster1 mirror image-primary5 snapshot 2024-09-12T13:29:55.496 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:29:55.496 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:29:55.496 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:29:55.496 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mode=snapshot 2024-09-12T13:29:55.496 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 mirror image enable mirror/image-primary5 snapshot 2024-09-12T13:29:55.713 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local ret=0 2024-09-12T13:29:55.714 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ timeout 5m bash -c 'zcat test-primary4/linux-5.4.tar.gz | pv -L 256K | tar xf - -C test-primary4' 2024-09-12T13:29:56.063 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Mirroring enabled 2024-09-12T13:29:56.066 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1330 2024-09-12T13:29:56.077 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:29:56.079 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 info mirror/image-primary5 2024-09-12T13:29:56.098 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:29:56.098 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:29:56.098 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:29:56.098 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:29:56.098 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:29:56.098 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:29:56.144 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:rbd image 'image-primary5': 2024-09-12T13:29:56.144 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: size 10 GiB in 2560 objects 2024-09-12T13:29:56.144 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: order 22 (4 MiB objects) 2024-09-12T13:29:56.145 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: snapshot_count: 1 2024-09-12T13:29:56.145 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: id: 24911a9a0cfb 2024-09-12T13:29:56.145 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: block_name_prefix: rbd_data.24911a9a0cfb 2024-09-12T13:29:56.145 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: format: 2 2024-09-12T13:29:56.145 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: features: layering, exclusive-lock, object-map, fast-diff 2024-09-12T13:29:56.145 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: op_features: 2024-09-12T13:29:56.145 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: flags: 2024-09-12T13:29:56.145 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: create_timestamp: Thu Sep 12 13:29:55 2024 2024-09-12T13:29:56.145 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: access_timestamp: Thu Sep 12 13:29:55 2024 2024-09-12T13:29:56.145 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: modify_timestamp: Thu Sep 12 13:29:55 2024 2024-09-12T13:29:56.145 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring state: enabled 2024-09-12T13:29:56.145 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring mode: snapshot 2024-09-12T13:29:56.145 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring global id: 95f21a09-f11d-496c-bf07-0203910a05a6 2024-09-12T13:29:56.145 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring primary: true 2024-09-12T13:29:56.150 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \n\b\d ]] 2024-09-12T13:29:56.150 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \k\r\b\d ]] 2024-09-12T13:29:56.150 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo rbd --cluster cluster1 device map -t krbd mirror/image-primary5 2024-09-12T13:29:56.270 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEV=/dev/rbd4 2024-09-12T13:29:56.270 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEVS+=($DEV) 2024-09-12T13:29:56.270 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo mkfs.ext4 /dev/rbd4 2024-09-12T13:29:56.290 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:mke2fs 1.46.5 (30-Dec-2021) 2024-09-12T13:29:56.303 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Discarding device blocks: 0/2621440 done 2024-09-12T13:29:56.303 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Creating filesystem with 2621440 4k blocks and 655360 inodes 2024-09-12T13:29:56.303 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Filesystem UUID: 77344e4e-1fa1-422c-a176-4028b10b725f 2024-09-12T13:29:56.303 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Superblock backups stored on blocks: 2024-09-12T13:29:56.303 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 2024-09-12T13:29:56.303 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 2024-09-12T13:29:56.303 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Allocating group tables: 0/80 done 2024-09-12T13:29:56.303 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Writing inode tables: 0/80 done 2024-09-12T13:29:56.308 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Creating journal (16384 blocks): done 2024-09-12T13:29:56.360 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Writing superblocks and filesystem accounting information: 0/80 done 2024-09-12T13:29:56.360 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 2024-09-12T13:29:56.363 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mkdir test-primary5 2024-09-12T13:29:56.365 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo mount /dev/rbd4 test-primary5 2024-09-12T13:29:56.421 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ whoami 2024-09-12T13:29:56.423 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo chown ubuntu test-primary5 2024-09-12T13:29:56.448 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ SNAP_PIDS+=($!) 2024-09-12T13:29:56.448 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ take_mirror_snapshots cluster1 mirror image-primary5 2024-09-12T13:29:56.448 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:29:56.448 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:29:56.448 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:29:56.448 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:29:56.448 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:29:56.448 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ WORKLOAD_PIDS+=($!) 2024-09-12T13:29:56.449 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:29:56.449 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:29:56.449 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:29:56.449 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:29:56.449 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:29:56.449 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ IMG=image-primary6 2024-09-12T13:29:56.449 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ MNTPT=test-primary6 2024-09-12T13:29:56.449 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ create_image_and_enable_mirror cluster1 mirror image-primary6 snapshot 10G 2024-09-12T13:29:56.449 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:29:56.449 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:29:56.449 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:29:56.449 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:29:56.449 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:29:56.449 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:29:56.450 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mode=snapshot 2024-09-12T13:29:56.450 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n snapshot ']' 2024-09-12T13:29:56.450 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:29:56.450 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ create_image cluster1 mirror image-primary6 10G 2024-09-12T13:29:56.450 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:29:56.450 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:29:56.450 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:29:56.450 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ slow_untar_workload test-primary5 2024-09-12T13:29:56.450 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:29:56.450 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:29:56.450 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:29:56.450 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local size=128 2024-09-12T13:29:56.450 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n 10G ']' 2024-09-12T13:29:56.450 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ size=10G 2024-09-12T13:29:56.451 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:29:56.451 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 create --size 10G --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image-primary6 2024-09-12T13:29:56.451 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mountpt=test-primary5 2024-09-12T13:29:56.451 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ cp linux-5.4.tar.gz test-primary5 2024-09-12T13:29:56.523 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ is_pool_mirror_mode_image mirror 2024-09-12T13:29:56.523 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:29:56.523 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' image = image ']' 2024-09-12T13:29:56.523 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:29:56.523 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ enable_mirror cluster1 mirror image-primary6 snapshot 2024-09-12T13:29:56.523 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:29:56.523 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:29:56.523 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:29:56.523 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mode=snapshot 2024-09-12T13:29:56.523 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 mirror image enable mirror/image-primary6 snapshot 2024-09-12T13:29:56.741 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local ret=0 2024-09-12T13:29:56.742 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ timeout 5m bash -c 'zcat test-primary5/linux-5.4.tar.gz | pv -L 256K | tar xf - -C test-primary5' 2024-09-12T13:29:57.069 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Mirroring enabled 2024-09-12T13:29:57.074 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1333 2024-09-12T13:29:57.079 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1332 2024-09-12T13:29:57.086 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:29:57.087 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 info mirror/image-primary6 2024-09-12T13:29:57.089 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:29:57.107 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:29:57.107 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:29:57.108 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:29:57.108 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:29:57.108 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:29:57.108 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:29:57.147 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:rbd image 'image-primary6': 2024-09-12T13:29:57.147 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: size 10 GiB in 2560 objects 2024-09-12T13:29:57.147 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: order 22 (4 MiB objects) 2024-09-12T13:29:57.147 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: snapshot_count: 1 2024-09-12T13:29:57.147 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: id: 24a313010808 2024-09-12T13:29:57.147 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: block_name_prefix: rbd_data.24a313010808 2024-09-12T13:29:57.147 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: format: 2 2024-09-12T13:29:57.147 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: features: layering, exclusive-lock, object-map, fast-diff 2024-09-12T13:29:57.147 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: op_features: 2024-09-12T13:29:57.147 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: flags: 2024-09-12T13:29:57.147 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: create_timestamp: Thu Sep 12 13:29:56 2024 2024-09-12T13:29:57.147 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: access_timestamp: Thu Sep 12 13:29:56 2024 2024-09-12T13:29:57.148 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: modify_timestamp: Thu Sep 12 13:29:56 2024 2024-09-12T13:29:57.148 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring state: enabled 2024-09-12T13:29:57.148 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring mode: snapshot 2024-09-12T13:29:57.148 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring global id: dcf3f6ae-ecc8-4d77-9308-bede00122fcd 2024-09-12T13:29:57.148 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring primary: true 2024-09-12T13:29:57.152 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \n\b\d ]] 2024-09-12T13:29:57.152 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \k\r\b\d ]] 2024-09-12T13:29:57.153 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo rbd --cluster cluster1 device map -t krbd mirror/image-primary6 2024-09-12T13:29:57.274 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEV=/dev/rbd5 2024-09-12T13:29:57.274 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEVS+=($DEV) 2024-09-12T13:29:57.275 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo mkfs.ext4 /dev/rbd5 2024-09-12T13:29:57.295 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:mke2fs 1.46.5 (30-Dec-2021) 2024-09-12T13:29:57.309 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Discarding device blocks: 0/2621440 done 2024-09-12T13:29:57.309 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Creating filesystem with 2621440 4k blocks and 655360 inodes 2024-09-12T13:29:57.309 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Filesystem UUID: 9bacc1b5-3fee-4554-b496-c386e5824ea0 2024-09-12T13:29:57.309 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Superblock backups stored on blocks: 2024-09-12T13:29:57.309 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 2024-09-12T13:29:57.309 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 2024-09-12T13:29:57.309 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Allocating group tables: 0/80 done 2024-09-12T13:29:57.309 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Writing inode tables: 0/80 done 2024-09-12T13:29:57.314 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Creating journal (16384 blocks): done 2024-09-12T13:29:57.367 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Writing superblocks and filesystem accounting information: 0/80 done 2024-09-12T13:29:57.367 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 2024-09-12T13:29:57.370 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mkdir test-primary6 2024-09-12T13:29:57.372 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo mount /dev/rbd5 test-primary6 2024-09-12T13:29:57.428 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ whoami 2024-09-12T13:29:57.430 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo chown ubuntu test-primary6 2024-09-12T13:29:57.455 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ SNAP_PIDS+=($!) 2024-09-12T13:29:57.455 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ take_mirror_snapshots cluster1 mirror image-primary6 2024-09-12T13:29:57.455 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:29:57.455 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:29:57.456 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:29:57.456 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:29:57.456 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:29:57.456 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:29:57.456 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:29:57.456 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:29:57.456 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ WORKLOAD_PIDS+=($!) 2024-09-12T13:29:57.456 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:29:57.456 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:29:57.456 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ IMG=image-primary7 2024-09-12T13:29:57.456 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ MNTPT=test-primary7 2024-09-12T13:29:57.456 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ create_image_and_enable_mirror cluster1 mirror image-primary7 snapshot 10G 2024-09-12T13:29:57.456 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ slow_untar_workload test-primary6 2024-09-12T13:29:57.456 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:29:57.457 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:29:57.457 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:29:57.457 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:29:57.457 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:29:57.457 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:29:57.457 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mode=snapshot 2024-09-12T13:29:57.457 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n snapshot ']' 2024-09-12T13:29:57.457 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:29:57.457 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ create_image cluster1 mirror image-primary7 10G 2024-09-12T13:29:57.457 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mountpt=test-primary6 2024-09-12T13:29:57.457 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:29:57.457 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:29:57.457 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ cp linux-5.4.tar.gz test-primary6 2024-09-12T13:29:57.457 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:29:57.458 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:29:57.458 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:29:57.458 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:29:57.458 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local size=128 2024-09-12T13:29:57.458 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n 10G ']' 2024-09-12T13:29:57.458 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ size=10G 2024-09-12T13:29:57.458 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:29:57.458 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 create --size 10G --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image-primary7 2024-09-12T13:29:57.528 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ is_pool_mirror_mode_image mirror 2024-09-12T13:29:57.529 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:29:57.529 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' image = image ']' 2024-09-12T13:29:57.529 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:29:57.529 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ enable_mirror cluster1 mirror image-primary7 snapshot 2024-09-12T13:29:57.529 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:29:57.529 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:29:57.529 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:29:57.529 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mode=snapshot 2024-09-12T13:29:57.529 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 mirror image enable mirror/image-primary7 snapshot 2024-09-12T13:29:57.740 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local ret=0 2024-09-12T13:29:57.740 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ timeout 5m bash -c 'zcat test-primary6/linux-5.4.tar.gz | pv -L 256K | tar xf - -C test-primary6' 2024-09-12T13:29:58.041 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Mirroring enabled 2024-09-12T13:29:58.046 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1335 2024-09-12T13:29:58.057 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1336 2024-09-12T13:29:58.065 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 info mirror/image-primary7 2024-09-12T13:29:58.069 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:29:58.069 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:29:58.070 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:29:58.070 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:29:58.070 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:29:58.070 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:29:58.079 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:29:58.098 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:29:58.145 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:rbd image 'image-primary7': 2024-09-12T13:29:58.146 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: size 10 GiB in 2560 objects 2024-09-12T13:29:58.146 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: order 22 (4 MiB objects) 2024-09-12T13:29:58.146 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: snapshot_count: 1 2024-09-12T13:29:58.146 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: id: 24b567735921 2024-09-12T13:29:58.146 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: block_name_prefix: rbd_data.24b567735921 2024-09-12T13:29:58.146 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: format: 2 2024-09-12T13:29:58.146 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: features: layering, exclusive-lock, object-map, fast-diff 2024-09-12T13:29:58.146 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: op_features: 2024-09-12T13:29:58.146 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: flags: 2024-09-12T13:29:58.146 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: create_timestamp: Thu Sep 12 13:29:57 2024 2024-09-12T13:29:58.146 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: access_timestamp: Thu Sep 12 13:29:57 2024 2024-09-12T13:29:58.147 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: modify_timestamp: Thu Sep 12 13:29:57 2024 2024-09-12T13:29:58.147 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring state: enabled 2024-09-12T13:29:58.147 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring mode: snapshot 2024-09-12T13:29:58.147 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring global id: da6437a6-d0c6-4534-a0e0-60fb07f54f52 2024-09-12T13:29:58.147 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring primary: true 2024-09-12T13:29:58.151 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \n\b\d ]] 2024-09-12T13:29:58.151 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \k\r\b\d ]] 2024-09-12T13:29:58.152 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo rbd --cluster cluster1 device map -t krbd mirror/image-primary7 2024-09-12T13:29:58.272 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEV=/dev/rbd6 2024-09-12T13:29:58.273 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEVS+=($DEV) 2024-09-12T13:29:58.273 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo mkfs.ext4 /dev/rbd6 2024-09-12T13:29:58.293 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:mke2fs 1.46.5 (30-Dec-2021) 2024-09-12T13:29:58.305 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Discarding device blocks: 0/2621440 done 2024-09-12T13:29:58.305 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Creating filesystem with 2621440 4k blocks and 655360 inodes 2024-09-12T13:29:58.305 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Filesystem UUID: de0024c6-96a4-4530-bb39-9f1914d4968c 2024-09-12T13:29:58.305 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Superblock backups stored on blocks: 2024-09-12T13:29:58.305 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 2024-09-12T13:29:58.305 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 2024-09-12T13:29:58.305 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Allocating group tables: 0/80 done 2024-09-12T13:29:58.305 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Writing inode tables: 0/80 done 2024-09-12T13:29:58.310 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Creating journal (16384 blocks): done 2024-09-12T13:29:58.357 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Writing superblocks and filesystem accounting information: 0/80 done 2024-09-12T13:29:58.357 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 2024-09-12T13:29:58.359 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mkdir test-primary7 2024-09-12T13:29:58.362 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo mount /dev/rbd6 test-primary7 2024-09-12T13:29:58.417 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ whoami 2024-09-12T13:29:58.419 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo chown ubuntu test-primary7 2024-09-12T13:29:58.444 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ SNAP_PIDS+=($!) 2024-09-12T13:29:58.445 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ take_mirror_snapshots cluster1 mirror image-primary7 2024-09-12T13:29:58.445 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:29:58.445 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:29:58.445 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:29:58.445 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:29:58.445 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:29:58.445 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ WORKLOAD_PIDS+=($!) 2024-09-12T13:29:58.445 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:29:58.445 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:29:58.445 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:29:58.445 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:29:58.445 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:29:58.446 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ IMG=image-primary8 2024-09-12T13:29:58.446 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ MNTPT=test-primary8 2024-09-12T13:29:58.446 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ slow_untar_workload test-primary7 2024-09-12T13:29:58.446 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ create_image_and_enable_mirror cluster1 mirror image-primary8 snapshot 10G 2024-09-12T13:29:58.446 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:29:58.446 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:29:58.446 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:29:58.446 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:29:58.446 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:29:58.446 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:29:58.446 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mode=snapshot 2024-09-12T13:29:58.446 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n snapshot ']' 2024-09-12T13:29:58.446 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:29:58.446 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mountpt=test-primary7 2024-09-12T13:29:58.447 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ create_image cluster1 mirror image-primary8 10G 2024-09-12T13:29:58.447 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ cp linux-5.4.tar.gz test-primary7 2024-09-12T13:29:58.447 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:29:58.447 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:29:58.447 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:29:58.447 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:29:58.447 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:29:58.447 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:29:58.447 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local size=128 2024-09-12T13:29:58.447 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n 10G ']' 2024-09-12T13:29:58.447 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ size=10G 2024-09-12T13:29:58.447 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:29:58.448 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 create --size 10G --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image-primary8 2024-09-12T13:29:58.516 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ is_pool_mirror_mode_image mirror 2024-09-12T13:29:58.516 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:29:58.516 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' image = image ']' 2024-09-12T13:29:58.516 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:29:58.516 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ enable_mirror cluster1 mirror image-primary8 snapshot 2024-09-12T13:29:58.517 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:29:58.517 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:29:58.517 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:29:58.517 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mode=snapshot 2024-09-12T13:29:58.517 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 mirror image enable mirror/image-primary8 snapshot 2024-09-12T13:29:58.732 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local ret=0 2024-09-12T13:29:58.732 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ timeout 5m bash -c 'zcat test-primary7/linux-5.4.tar.gz | pv -L 256K | tar xf - -C test-primary7' 2024-09-12T13:29:59.083 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:29:59.083 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:29:59.083 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:29:59.083 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:29:59.083 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:29:59.084 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:29:59.088 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1339 2024-09-12T13:29:59.089 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Mirroring enabled 2024-09-12T13:29:59.100 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1338 2024-09-12T13:29:59.123 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:29:59.129 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 info mirror/image-primary8 2024-09-12T13:29:59.132 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:29:59.213 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:rbd image 'image-primary8': 2024-09-12T13:29:59.213 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: size 10 GiB in 2560 objects 2024-09-12T13:29:59.213 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: order 22 (4 MiB objects) 2024-09-12T13:29:59.213 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: snapshot_count: 1 2024-09-12T13:29:59.213 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: id: 24c73c9b6e35 2024-09-12T13:29:59.213 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: block_name_prefix: rbd_data.24c73c9b6e35 2024-09-12T13:29:59.213 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: format: 2 2024-09-12T13:29:59.213 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: features: layering, exclusive-lock, object-map, fast-diff 2024-09-12T13:29:59.213 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: op_features: 2024-09-12T13:29:59.213 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: flags: 2024-09-12T13:29:59.214 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: create_timestamp: Thu Sep 12 13:29:58 2024 2024-09-12T13:29:59.214 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: access_timestamp: Thu Sep 12 13:29:58 2024 2024-09-12T13:29:59.214 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: modify_timestamp: Thu Sep 12 13:29:58 2024 2024-09-12T13:29:59.214 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring state: enabled 2024-09-12T13:29:59.214 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring mode: snapshot 2024-09-12T13:29:59.214 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring global id: b415829c-3d1c-41f6-a1aa-e76e40e110d1 2024-09-12T13:29:59.214 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring primary: true 2024-09-12T13:29:59.223 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \n\b\d ]] 2024-09-12T13:29:59.223 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \k\r\b\d ]] 2024-09-12T13:29:59.224 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo rbd --cluster cluster1 device map -t krbd mirror/image-primary8 2024-09-12T13:29:59.343 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEV=/dev/rbd7 2024-09-12T13:29:59.343 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEVS+=($DEV) 2024-09-12T13:29:59.343 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo mkfs.ext4 /dev/rbd7 2024-09-12T13:29:59.364 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:mke2fs 1.46.5 (30-Dec-2021) 2024-09-12T13:29:59.376 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Discarding device blocks: 0/2621440 done 2024-09-12T13:29:59.376 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Creating filesystem with 2621440 4k blocks and 655360 inodes 2024-09-12T13:29:59.376 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Filesystem UUID: 7d350715-00d0-4310-be2c-56c0a5c00299 2024-09-12T13:29:59.376 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Superblock backups stored on blocks: 2024-09-12T13:29:59.377 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 2024-09-12T13:29:59.377 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 2024-09-12T13:29:59.377 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Allocating group tables: 0/80 done 2024-09-12T13:29:59.377 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Writing inode tables: 0/80 done 2024-09-12T13:29:59.381 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Creating journal (16384 blocks): done 2024-09-12T13:29:59.434 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Writing superblocks and filesystem accounting information: 0/80 done 2024-09-12T13:29:59.434 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 2024-09-12T13:29:59.436 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mkdir test-primary8 2024-09-12T13:29:59.439 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo mount /dev/rbd7 test-primary8 2024-09-12T13:29:59.496 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ whoami 2024-09-12T13:29:59.498 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo chown ubuntu test-primary8 2024-09-12T13:29:59.524 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ SNAP_PIDS+=($!) 2024-09-12T13:29:59.524 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ take_mirror_snapshots cluster1 mirror image-primary8 2024-09-12T13:29:59.524 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:29:59.524 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:29:59.524 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:29:59.524 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:29:59.525 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:29:59.525 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:29:59.525 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:29:59.525 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:29:59.525 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:29:59.525 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ WORKLOAD_PIDS+=($!) 2024-09-12T13:29:59.525 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:29:59.525 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ IMG=image-primary9 2024-09-12T13:29:59.525 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ MNTPT=test-primary9 2024-09-12T13:29:59.525 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ create_image_and_enable_mirror cluster1 mirror image-primary9 snapshot 10G 2024-09-12T13:29:59.525 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ slow_untar_workload test-primary8 2024-09-12T13:29:59.525 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:29:59.525 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:29:59.526 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:29:59.526 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:29:59.526 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:29:59.526 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:29:59.526 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mode=snapshot 2024-09-12T13:29:59.526 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mountpt=test-primary8 2024-09-12T13:29:59.526 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n snapshot ']' 2024-09-12T13:29:59.526 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:29:59.526 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ create_image cluster1 mirror image-primary9 10G 2024-09-12T13:29:59.526 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ cp linux-5.4.tar.gz test-primary8 2024-09-12T13:29:59.526 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:29:59.526 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:29:59.526 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:29:59.527 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:29:59.527 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:29:59.527 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:29:59.527 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local size=128 2024-09-12T13:29:59.527 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n 10G ']' 2024-09-12T13:29:59.527 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ size=10G 2024-09-12T13:29:59.527 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:29:59.527 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 create --size 10G --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image-primary9 2024-09-12T13:29:59.596 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ is_pool_mirror_mode_image mirror 2024-09-12T13:29:59.596 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:29:59.596 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' image = image ']' 2024-09-12T13:29:59.596 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:29:59.596 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ enable_mirror cluster1 mirror image-primary9 snapshot 2024-09-12T13:29:59.596 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:29:59.596 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:29:59.596 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:29:59.596 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mode=snapshot 2024-09-12T13:29:59.597 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 mirror image enable mirror/image-primary9 snapshot 2024-09-12T13:29:59.817 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local ret=0 2024-09-12T13:29:59.817 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ timeout 5m bash -c 'zcat test-primary8/linux-5.4.tar.gz | pv -L 256K | tar xf - -C test-primary8' 2024-09-12T13:30:00.081 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Mirroring enabled 2024-09-12T13:30:00.090 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:30:00.090 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:30:00.090 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:30:00.090 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:30:00.090 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:30:00.090 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:30:00.093 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:30:00.093 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:30:00.093 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:30:00.093 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:30:00.093 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:30:00.094 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:30:00.096 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1341 2024-09-12T13:30:00.096 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1342 2024-09-12T13:30:00.099 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 info mirror/image-primary9 2024-09-12T13:30:00.106 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:30:00.107 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:30:00.162 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:rbd image 'image-primary9': 2024-09-12T13:30:00.162 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: size 10 GiB in 2560 objects 2024-09-12T13:30:00.162 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: order 22 (4 MiB objects) 2024-09-12T13:30:00.162 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: snapshot_count: 1 2024-09-12T13:30:00.162 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: id: 24d989160293 2024-09-12T13:30:00.163 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: block_name_prefix: rbd_data.24d989160293 2024-09-12T13:30:00.163 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: format: 2 2024-09-12T13:30:00.163 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: features: layering, exclusive-lock, object-map, fast-diff 2024-09-12T13:30:00.163 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: op_features: 2024-09-12T13:30:00.163 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: flags: 2024-09-12T13:30:00.163 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: create_timestamp: Thu Sep 12 13:29:59 2024 2024-09-12T13:30:00.164 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: access_timestamp: Thu Sep 12 13:29:59 2024 2024-09-12T13:30:00.164 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: modify_timestamp: Thu Sep 12 13:29:59 2024 2024-09-12T13:30:00.164 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring state: enabled 2024-09-12T13:30:00.164 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring mode: snapshot 2024-09-12T13:30:00.164 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring global id: 1a8d4f4e-16a8-41c0-8b42-954a67b2e30d 2024-09-12T13:30:00.164 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring primary: true 2024-09-12T13:30:00.167 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \n\b\d ]] 2024-09-12T13:30:00.167 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \k\r\b\d ]] 2024-09-12T13:30:00.168 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo rbd --cluster cluster1 device map -t krbd mirror/image-primary9 2024-09-12T13:30:00.289 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEV=/dev/rbd8 2024-09-12T13:30:00.289 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEVS+=($DEV) 2024-09-12T13:30:00.289 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo mkfs.ext4 /dev/rbd8 2024-09-12T13:30:00.310 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:mke2fs 1.46.5 (30-Dec-2021) 2024-09-12T13:30:00.324 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Discarding device blocks: 0/2621440 done 2024-09-12T13:30:00.324 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Creating filesystem with 2621440 4k blocks and 655360 inodes 2024-09-12T13:30:00.324 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Filesystem UUID: 038bd60b-fd2f-48ff-9225-ed296fab8f6c 2024-09-12T13:30:00.324 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Superblock backups stored on blocks: 2024-09-12T13:30:00.325 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 2024-09-12T13:30:00.325 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 2024-09-12T13:30:00.325 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Allocating group tables: 0/80 done 2024-09-12T13:30:00.325 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Writing inode tables: 0/80 done 2024-09-12T13:30:00.329 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Creating journal (16384 blocks): done 2024-09-12T13:30:00.445 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Writing superblocks and filesystem accounting information: 0/80 done 2024-09-12T13:30:00.445 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 2024-09-12T13:30:00.448 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mkdir test-primary9 2024-09-12T13:30:00.450 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo mount /dev/rbd8 test-primary9 2024-09-12T13:30:00.510 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ whoami 2024-09-12T13:30:00.512 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo chown ubuntu test-primary9 2024-09-12T13:30:00.539 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ SNAP_PIDS+=($!) 2024-09-12T13:30:00.539 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ take_mirror_snapshots cluster1 mirror image-primary9 2024-09-12T13:30:00.540 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:30:00.540 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:30:00.540 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:30:00.540 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:30:00.540 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:30:00.540 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:30:00.541 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:30:00.541 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ WORKLOAD_PIDS+=($!) 2024-09-12T13:30:00.541 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:30:00.541 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:30:00.541 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:30:00.541 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ IMG=image-primary10 2024-09-12T13:30:00.542 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ MNTPT=test-primary10 2024-09-12T13:30:00.542 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ create_image_and_enable_mirror cluster1 mirror image-primary10 snapshot 10G 2024-09-12T13:30:00.542 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ slow_untar_workload test-primary9 2024-09-12T13:30:00.542 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:30:00.542 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:30:00.542 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:30:00.542 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:30:00.543 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:30:00.543 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:30:00.543 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mode=snapshot 2024-09-12T13:30:00.544 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n snapshot ']' 2024-09-12T13:30:00.544 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:30:00.544 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ create_image cluster1 mirror image-primary10 10G 2024-09-12T13:30:00.544 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:30:00.545 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mountpt=test-primary9 2024-09-12T13:30:00.545 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:30:00.545 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:30:00.545 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:30:00.545 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:30:00.545 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:30:00.546 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local size=128 2024-09-12T13:30:00.546 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ cp linux-5.4.tar.gz test-primary9 2024-09-12T13:30:00.546 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n 10G ']' 2024-09-12T13:30:00.547 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ size=10G 2024-09-12T13:30:00.547 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:30:00.547 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 create --size 10G --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image-primary10 2024-09-12T13:30:00.608 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ is_pool_mirror_mode_image mirror 2024-09-12T13:30:00.608 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:30:00.608 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' image = image ']' 2024-09-12T13:30:00.608 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:30:00.609 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ enable_mirror cluster1 mirror image-primary10 snapshot 2024-09-12T13:30:00.609 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:30:00.609 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:30:00.609 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:30:00.609 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mode=snapshot 2024-09-12T13:30:00.609 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 mirror image enable mirror/image-primary10 snapshot 2024-09-12T13:30:00.829 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local ret=0 2024-09-12T13:30:00.829 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ timeout 5m bash -c 'zcat test-primary9/linux-5.4.tar.gz | pv -L 256K | tar xf - -C test-primary9' 2024-09-12T13:30:01.085 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:30:01.085 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:30:01.085 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:30:01.085 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:30:01.085 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:30:01.085 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:30:01.102 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:30:01.102 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:30:01.103 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:30:01.103 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:30:01.103 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:30:01.103 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:30:01.117 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1345 2024-09-12T13:30:01.118 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Mirroring enabled 2024-09-12T13:30:01.120 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1346 2024-09-12T13:30:01.126 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1344 2024-09-12T13:30:01.140 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:30:01.140 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:30:01.146 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:30:01.151 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 info mirror/image-primary10 2024-09-12T13:30:01.222 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:rbd image 'image-primary10': 2024-09-12T13:30:01.222 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: size 10 GiB in 2560 objects 2024-09-12T13:30:01.222 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: order 22 (4 MiB objects) 2024-09-12T13:30:01.222 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: snapshot_count: 1 2024-09-12T13:30:01.222 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: id: 24ee9621fa30 2024-09-12T13:30:01.222 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: block_name_prefix: rbd_data.24ee9621fa30 2024-09-12T13:30:01.222 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: format: 2 2024-09-12T13:30:01.222 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: features: layering, exclusive-lock, object-map, fast-diff 2024-09-12T13:30:01.222 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: op_features: 2024-09-12T13:30:01.222 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: flags: 2024-09-12T13:30:01.222 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: create_timestamp: Thu Sep 12 13:30:00 2024 2024-09-12T13:30:01.223 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: access_timestamp: Thu Sep 12 13:30:00 2024 2024-09-12T13:30:01.223 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: modify_timestamp: Thu Sep 12 13:30:00 2024 2024-09-12T13:30:01.223 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring state: enabled 2024-09-12T13:30:01.223 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring mode: snapshot 2024-09-12T13:30:01.223 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring global id: 7450998e-7967-4447-af66-572bd4fdded4 2024-09-12T13:30:01.223 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring primary: true 2024-09-12T13:30:01.227 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \n\b\d ]] 2024-09-12T13:30:01.227 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \k\r\b\d ]] 2024-09-12T13:30:01.228 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo rbd --cluster cluster1 device map -t krbd mirror/image-primary10 2024-09-12T13:30:01.351 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEV=/dev/rbd9 2024-09-12T13:30:01.351 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEVS+=($DEV) 2024-09-12T13:30:01.351 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo mkfs.ext4 /dev/rbd9 2024-09-12T13:30:01.376 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:mke2fs 1.46.5 (30-Dec-2021) 2024-09-12T13:30:01.388 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Discarding device blocks: 0/2621440 done 2024-09-12T13:30:01.388 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Creating filesystem with 2621440 4k blocks and 655360 inodes 2024-09-12T13:30:01.388 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Filesystem UUID: 388c2977-b643-47f1-a6e2-44446bebdf82 2024-09-12T13:30:01.388 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Superblock backups stored on blocks: 2024-09-12T13:30:01.388 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 2024-09-12T13:30:01.388 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 2024-09-12T13:30:01.388 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Allocating group tables: 0/80 done 2024-09-12T13:30:01.389 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Writing inode tables: 0/80 done 2024-09-12T13:30:01.394 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Creating journal (16384 blocks): done 2024-09-12T13:30:01.494 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Writing superblocks and filesystem accounting information: 0/80 done 2024-09-12T13:30:01.494 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 2024-09-12T13:30:01.497 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mkdir test-primary10 2024-09-12T13:30:01.499 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo mount /dev/rbd9 test-primary10 2024-09-12T13:30:01.556 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ whoami 2024-09-12T13:30:01.558 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo chown ubuntu test-primary10 2024-09-12T13:30:01.582 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ SNAP_PIDS+=($!) 2024-09-12T13:30:01.582 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ take_mirror_snapshots cluster1 mirror image-primary10 2024-09-12T13:30:01.583 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:30:01.583 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:30:01.583 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:30:01.583 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:30:01.583 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:30:01.583 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:30:01.583 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:30:01.583 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:30:01.583 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:30:01.583 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ WORKLOAD_PIDS+=($!) 2024-09-12T13:30:01.583 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ slow_untar_workload test-primary10 2024-09-12T13:30:01.583 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${SNAP_PIDS[@]} 2024-09-12T13:30:01.583 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 82736 2024-09-12T13:30:01.584 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mountpt=test-primary10 2024-09-12T13:30:01.584 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ cp linux-5.4.tar.gz test-primary10 2024-09-12T13:30:01.867 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local ret=0 2024-09-12T13:30:01.868 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ timeout 5m bash -c 'zcat test-primary10/linux-5.4.tar.gz | pv -L 256K | tar xf - -C test-primary10' 2024-09-12T13:30:02.128 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:30:02.128 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:30:02.129 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:30:02.129 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:30:02.129 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:30:02.129 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:30:02.136 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:30:02.136 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:30:02.136 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:30:02.136 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:30:02.136 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:30:02.136 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:30:02.152 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1350 2024-09-12T13:30:02.154 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1348 2024-09-12T13:30:02.197 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:30:02.199 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1349 2024-09-12T13:30:02.205 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:30:02.234 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:30:03.110 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:30:03.110 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:30:03.110 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:30:03.110 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:30:03.110 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:30:03.111 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:30:03.113 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:30:03.113 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:30:03.113 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:30:03.113 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:30:03.113 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:30:03.114 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:30:03.163 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1351 2024-09-12T13:30:03.200 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1352 2024-09-12T13:30:03.204 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:30:03.285 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:30:04.145 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:30:04.145 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:30:04.145 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:30:04.146 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:30:04.146 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:30:04.146 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:30:04.151 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:30:04.151 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:30:04.151 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:30:04.151 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:30:04.151 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:30:04.151 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:30:04.153 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:30:04.153 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:30:04.153 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:30:04.153 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:30:04.153 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:30:04.153 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:30:04.215 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1353 2024-09-12T13:30:04.218 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1354 2024-09-12T13:30:04.276 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:30:04.285 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:30:05.130 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1357 2024-09-12T13:30:05.134 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1356 2024-09-12T13:30:05.154 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1355 2024-09-12T13:30:05.157 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:30:05.167 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:30:05.177 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:30:05.203 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:30:05.204 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:30:05.204 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:30:05.204 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:30:05.204 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:30:05.204 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:30:05.211 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:30:05.212 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:30:05.212 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:30:05.212 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:30:05.212 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:30:05.212 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:30:05.239 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:30:05.239 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:30:05.239 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:30:05.239 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:30:05.239 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:30:05.239 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:30:06.152 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1360 2024-09-12T13:30:06.160 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1358 2024-09-12T13:30:06.184 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1359 2024-09-12T13:30:06.198 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:30:06.204 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:30:06.211 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:30:06.213 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:30:06.213 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:30:06.213 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:30:06.213 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:30:06.213 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:30:06.217 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:30:06.289 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:30:06.290 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:30:06.290 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:30:06.290 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:30:06.290 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:30:06.290 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:30:07.161 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1362 2024-09-12T13:30:07.177 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1361 2024-09-12T13:30:07.196 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:30:07.216 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:30:07.281 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:30:07.281 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:30:07.281 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:30:07.282 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:30:07.282 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:30:07.282 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:30:07.291 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:30:07.291 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:30:07.291 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:30:07.291 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:30:07.291 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:30:07.291 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:30:08.160 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:30:08.160 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:30:08.160 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:30:08.162 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:30:08.162 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:30:08.162 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:30:08.172 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:30:08.179 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:30:08.179 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:30:08.179 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:30:08.179 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:30:08.179 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:30:08.182 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1363 2024-09-12T13:30:08.182 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:30:08.182 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:30:08.183 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1364 2024-09-12T13:30:08.183 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:30:08.183 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:30:08.183 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:30:08.183 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:30:08.228 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:30:08.240 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:30:09.188 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1365 2024-09-12T13:30:09.194 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1366 2024-09-12T13:30:09.210 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:30:09.210 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:30:09.210 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:30:09.210 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:30:09.211 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:30:09.211 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:30:09.213 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:30:09.213 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:30:09.213 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:30:09.213 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:30:09.213 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:30:09.214 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:30:09.229 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1367 2024-09-12T13:30:09.229 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:30:09.232 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:30:09.232 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:30:09.232 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:30:09.232 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:30:09.232 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:30:09.233 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:30:09.233 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:30:09.284 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:30:10.201 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:30:10.201 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:30:10.202 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:30:10.202 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:30:10.202 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:30:10.202 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:30:10.220 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:30:10.221 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:30:10.221 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:30:10.222 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:30:10.222 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:30:10.222 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:30:10.246 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1370 2024-09-12T13:30:10.253 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1368 2024-09-12T13:30:10.264 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1369 2024-09-12T13:30:10.280 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:30:10.282 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:30:10.310 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:30:11.235 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:30:11.235 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:30:11.237 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:30:11.237 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:30:11.237 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:30:11.237 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:30:11.247 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:30:11.247 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:30:11.247 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:30:11.247 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:30:11.247 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:30:11.247 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:30:11.306 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1371 2024-09-12T13:30:11.332 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1372 2024-09-12T13:30:11.335 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:30:11.389 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:30:12.241 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:30:12.241 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:30:12.241 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:30:12.241 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:30:12.241 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:30:12.241 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:30:12.242 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:30:12.242 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:30:12.242 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:30:12.242 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:30:12.242 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:30:12.242 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:30:12.290 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1373 2024-09-12T13:30:12.291 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:30:12.291 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:30:12.291 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:30:12.291 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:30:12.291 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:30:12.291 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:30:12.329 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1374 2024-09-12T13:30:12.332 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:30:12.345 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:30:13.057 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1376 2024-09-12T13:30:13.096 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:30:13.150 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1375 2024-09-12T13:30:13.175 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:30:13.286 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:30:13.286 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:30:13.286 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:30:13.286 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:30:13.286 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:30:13.286 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:30:13.290 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:30:13.290 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:30:13.290 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:30:13.291 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:30:13.291 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:30:13.291 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:30:13.312 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:30:13.313 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:30:13.313 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:30:13.313 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:30:13.313 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:30:13.313 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:30:14.207 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1378 2024-09-12T13:30:14.225 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:30:14.229 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1379 2024-09-12T13:30:14.248 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:30:14.338 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:30:14.338 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:30:14.338 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:30:14.338 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:30:14.338 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:30:14.338 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:30:14.393 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:30:14.393 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:30:14.393 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:30:14.393 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:30:14.393 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:30:14.393 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:30:15.171 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1377 2024-09-12T13:30:15.208 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:30:15.213 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:30:15.213+0000 7f67dd000640 -1 rbd::mirror::image_replayer::snapshot::Replayer: 0x56055c195000 handle_unlink_peer: failed to unlink local peer from remote image: (22) Invalid argument 2024-09-12T13:30:15.243 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1381 2024-09-12T13:30:15.252 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:30:15.335 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:30:15.335 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:30:15.336 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:30:15.336 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:30:15.336 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:30:15.336 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:30:15.347 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:30:15.348 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:30:15.348 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:30:15.348 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:30:15.348 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:30:15.348 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:30:16.099 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:30:16.100 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:30:16.100 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:30:16.100 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:30:16.100 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:30:16.100 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:30:16.175 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1380 2024-09-12T13:30:16.179 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:30:16.179 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:30:16.180 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:30:16.180 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:30:16.180 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:30:16.180 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:30:16.217 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:30:16.243 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1383 2024-09-12T13:30:16.260 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:30:17.163 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1382 2024-09-12T13:30:17.197 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:30:17.230 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:30:17.230 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:30:17.230 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:30:17.230 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:30:17.230 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:30:17.230 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:30:17.244 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1385 2024-09-12T13:30:17.251 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:30:17.251 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:30:17.252 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:30:17.252 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:30:17.252 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:30:17.252 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:30:17.254 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:30:18.024 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1384 2024-09-12T13:30:18.066 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1386 2024-09-12T13:30:18.082 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:30:18.099 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:30:18.110 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1387 2024-09-12T13:30:18.142 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:30:18.211 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:30:18.211 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:30:18.211 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:30:18.211 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:30:18.211 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:30:18.211 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:30:18.255 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:30:18.255 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:30:18.256 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:30:18.256 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:30:18.256 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:30:18.256 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:30:19.074 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1388 2024-09-12T13:30:19.093 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:30:19.095 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1390 2024-09-12T13:30:19.114 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:30:19.220 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:30:19.220 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:30:19.220 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:30:19.221 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:30:19.221 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:30:19.221 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:30:19.263 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:30:19.263 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:30:19.263 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:30:19.263 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:30:19.263 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:30:19.263 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:30:20.187 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1392 2024-09-12T13:30:20.199 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:30:20.200 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:30:20.200 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:30:20.200 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:30:20.200 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:30:20.200 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:30:20.231 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:30:20.257 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:30:20.257 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:30:20.257 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:30:20.257 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:30:20.257 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:30:20.257 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:30:21.043 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1389 2024-09-12T13:30:21.086 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:30:21.091 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:30:21.091 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:30:21.091 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:30:21.091 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:30:21.091 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:30:21.091 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:30:21.104 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:30:21.104 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:30:21.105 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:30:21.105 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:30:21.105 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:30:21.105 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:30:21.148 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:30:21.148 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:30:21.148 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:30:21.148 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:30:21.149 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:30:21.149 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:30:21.213 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1391 2024-09-12T13:30:21.243 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1394 2024-09-12T13:30:21.267 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:30:21.273 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:30:22.098 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:30:22.099 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:30:22.099 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:30:22.099 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:30:22.099 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:30:22.099 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:30:22.113 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1393 2024-09-12T13:30:22.118 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:30:22.118 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:30:22.118 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:30:22.118 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:30:22.118 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:30:22.118 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:30:22.145 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:30:22.219 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1395 2024-09-12T13:30:22.293 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:30:23.064 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1397 2024-09-12T13:30:23.095 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1396 2024-09-12T13:30:23.101 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:30:23.117 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:30:23.233 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:30:23.234 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:30:23.234 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:30:23.234 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:30:23.234 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:30:23.234 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:30:24.091 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:30:24.091 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:30:24.091 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:30:24.091 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:30:24.091 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:30:24.091 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:30:24.095 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1398 2024-09-12T13:30:24.103 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1399 2024-09-12T13:30:24.133 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:30:24.133 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:30:24.278 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:30:24.278 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:30:24.278 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:30:24.278 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:30:24.278 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:30:24.278 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:30:24.280 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:30:24.280 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:30:24.280 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:30:24.281 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:30:24.281 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:30:24.281 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:30:25.150 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:30:25.151 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:30:25.151 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:30:25.151 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:30:25.151 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:30:25.151 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:30:25.168 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1401 2024-09-12T13:30:25.177 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:30:25.295 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:30:25.296 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:30:25.296 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:30:25.296 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:30:25.296 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:30:25.296 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:30:26.052 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1400 2024-09-12T13:30:26.104 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:30:26.106 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:30:26.106 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:30:26.106 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:30:26.106 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:30:26.106 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:30:26.106 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:30:26.110 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1402 2024-09-12T13:30:26.122 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:30:26.122 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:30:26.122 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:30:26.122 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:30:26.122 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:30:26.122 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:30:26.125 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1403 2024-09-12T13:30:26.189 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:30:26.198 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:30:27.080 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1404 2024-09-12T13:30:27.110 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1405 2024-09-12T13:30:27.119 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:30:27.135 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:30:27.135 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:30:27.136 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:30:27.136 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:30:27.136 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:30:27.136 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:30:27.138 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:30:27.138 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:30:27.138 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:30:27.138 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:30:27.138 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:30:27.139 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:30:27.149 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:30:28.092 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1406 2024-09-12T13:30:28.103 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1407 2024-09-12T13:30:28.122 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:30:28.134 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:30:28.181 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:30:28.181 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:30:28.181 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:30:28.181 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:30:28.182 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:30:28.182 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:30:29.098 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1408 2024-09-12T13:30:29.109 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:30:29.110 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:30:29.110 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:30:29.110 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:30:29.111 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:30:29.111 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:30:29.135 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1409 2024-09-12T13:30:29.153 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:30:29.196 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:30:29.196 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:30:29.196 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:30:29.196 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:30:29.196 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:30:29.196 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:30:29.206 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:30:29.207 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:30:29.207 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:30:29.207 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:30:29.207 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:30:29.207 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:30:29.238 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:30:30.124 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:30:30.124 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:30:30.124 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:30:30.124 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:30:30.124 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:30:30.124 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:30:30.157 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:30:30.157 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:30:30.157 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:30:30.157 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:30:30.157 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:30:30.157 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:30:30.187 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1411 2024-09-12T13:30:30.206 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:30:31.103 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1410 2024-09-12T13:30:31.126 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:30:31.126 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:30:31.127 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:30:31.127 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:30:31.127 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:30:31.127 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:30:31.138 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:30:31.138 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:30:31.138 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:30:31.138 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:30:31.138 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:30:31.138 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:30:31.142 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:30:32.070 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1412 2024-09-12T13:30:32.083 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1413 2024-09-12T13:30:32.119 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1415 2024-09-12T13:30:32.120 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:30:32.158 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:30:32.318 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:30:32.318 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:30:32.319 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:30:32.319 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:30:32.319 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:30:32.320 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1414 2024-09-12T13:30:32.321 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:30:32.321 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:30:32.322 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:30:32.322 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:30:32.322 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:30:32.322 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:30:32.322 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:30:32.322 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:30:32.323 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:30:33.066 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1416 2024-09-12T13:30:33.082 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1417 2024-09-12T13:30:33.101 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:30:33.110 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:30:33.212 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:30:33.212 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:30:33.212 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:30:33.213 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:30:33.213 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:30:33.213 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:30:34.116 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1419 2024-09-12T13:30:34.139 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1418 2024-09-12T13:30:34.146 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:30:34.146 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:30:34.146 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:30:34.146 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:30:34.146 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:30:34.146 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:30:34.147 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:30:34.158 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:30:35.124 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:30:35.124 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:30:35.124 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:30:35.125 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:30:35.125 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:30:35.125 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:30:35.145 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1421 2024-09-12T13:30:35.162 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:30:35.164 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:30:35.165 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:30:35.165 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:30:35.165 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:30:35.165 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:30:35.165 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:30:35.165 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:30:35.166 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:30:35.166 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:30:35.166 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:30:35.166 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:30:35.166 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:30:35.200 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:30:35.200 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:30:35.200 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:30:35.201 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:30:35.201 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:30:35.201 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:30:36.098 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1420 2024-09-12T13:30:36.104 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:30:36.104 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:30:36.104 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:30:36.104 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:30:36.104 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:30:36.104 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:30:36.114 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:30:36.114 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:30:36.114 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:30:36.114 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:30:36.114 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:30:36.114 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:30:36.142 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:30:36.168 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1423 2024-09-12T13:30:36.169 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1422 2024-09-12T13:30:36.192 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:30:36.194 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:30:37.151 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:30:37.151 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:30:37.151 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:30:37.151 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:30:37.151 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:30:37.151 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:30:37.163 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:30:37.163 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:30:37.163 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:30:37.163 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:30:37.163 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:30:37.163 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:30:37.227 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:30:37.228+0000 7f67dc600640 -1 rbd::mirror::image_replayer::snapshot::Replayer: 0x56055ba72c00 handle_unlink_peer: failed to unlink local peer from remote image: (22) Invalid argument 2024-09-12T13:30:38.022 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1425 2024-09-12T13:30:38.033 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1424 2024-09-12T13:30:38.073 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:30:38.090 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:30:38.101 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1427 2024-09-12T13:30:38.165 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:30:38.165 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:30:38.165 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:30:38.165 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:30:38.165 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:30:38.165 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:30:38.166 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:30:38.168 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:30:38.169+0000 7f67dc600640 -1 rbd::mirror::image_replayer::snapshot::Replayer: 0x56055b84f400 handle_unlink_peer: failed to unlink local peer from remote image: (22) Invalid argument 2024-09-12T13:30:39.109 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1426 2024-09-12T13:30:39.146 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:30:39.146 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:30:39.146 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:30:39.146 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:30:39.146 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:30:39.146 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:30:39.165 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:30:39.195 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:30:39.195 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:30:39.195 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:30:39.195 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:30:39.195 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:30:39.195 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:30:39.196 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:30:39.196 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:30:39.196 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:30:39.196 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:30:39.196 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:30:39.196 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:30:40.112 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1429 2024-09-12T13:30:40.117 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1428 2024-09-12T13:30:40.176 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:30:40.177 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:30:40.234 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1431 2024-09-12T13:30:40.272 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:30:41.076 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:30:41.076 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:30:41.076 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:30:41.077 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:30:41.077 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:30:41.077 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:30:41.092 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:30:41.093 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:30:41.093 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:30:41.093 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:30:41.093 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:30:41.093 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:30:41.099 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1430 2024-09-12T13:30:41.147 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:30:41.169 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:30:41.169 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:30:41.169 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:30:41.169 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:30:41.169 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:30:41.170 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:30:42.107 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1433 2024-09-12T13:30:42.118 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1432 2024-09-12T13:30:42.157 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:30:42.168 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:30:42.168 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:30:42.168 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:30:42.168 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:30:42.168 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:30:42.168 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:30:42.172 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:30:42.203 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1434 2024-09-12T13:30:42.615 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1435 2024-09-12T13:30:42.615 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:30:42.633 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:30:43.091 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1437 2024-09-12T13:30:43.109 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:30:43.181 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:30:43.181 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:30:43.181 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:30:43.181 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:30:43.181 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:30:43.182 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:30:43.182 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:30:43.182 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:30:43.182 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:30:43.182 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:30:43.182 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:30:43.183 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:30:43.275 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:30:43.275 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:30:43.275 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:30:43.276 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:30:43.276 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:30:43.276 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:30:44.010 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1436 2024-09-12T13:30:44.061 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:30:44.097 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1438 2024-09-12T13:30:44.106 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1439 2024-09-12T13:30:44.112 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:30:44.115 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:30:44.150 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:30:44.150 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:30:44.150 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:30:44.150 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:30:44.150 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:30:44.150 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:30:45.091 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1441 2024-09-12T13:30:45.107 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:30:45.160 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:30:45.160 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:30:45.160 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:30:45.160 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:30:45.160 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:30:45.160 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:30:45.175 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:30:45.175 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:30:45.175 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:30:45.175 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:30:45.175 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:30:45.175 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:30:45.618 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:30:45.619 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:30:45.619 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:30:45.619 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:30:45.619 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:30:45.619 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:30:45.636 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:30:45.636 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:30:45.636 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:30:45.636 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:30:45.636 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:30:45.636 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:30:46.037 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1440 2024-09-12T13:30:46.089 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:30:46.111 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:30:46.111 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:30:46.111 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:30:46.111 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:30:46.111 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:30:46.111 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:30:46.128 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1443 2024-09-12T13:30:46.138 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1442 2024-09-12T13:30:46.139 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1444 2024-09-12T13:30:46.151 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:30:46.151 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:30:46.158 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:30:47.063 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:30:47.063 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:30:47.063 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:30:47.064 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:30:47.064 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:30:47.064 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:30:47.115 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:30:47.115 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:30:47.115 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:30:47.115 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:30:47.115 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:30:47.115 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:30:47.117 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:30:47.118 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:30:47.118 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:30:47.118 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:30:47.118 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:30:47.118 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:30:47.163 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1446 2024-09-12T13:30:47.178 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:30:48.036 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1445 2024-09-12T13:30:48.099 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:30:48.111 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:30:48.111 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:30:48.111 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:30:48.112 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:30:48.112 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:30:48.112 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:30:48.145 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1449 2024-09-12T13:30:48.183 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:30:49.094 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:30:49.094 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:30:49.094 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:30:49.094 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:30:49.094 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:30:49.094 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:30:49.154 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:30:49.155 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:30:49.155 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:30:49.155 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:30:49.155 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:30:49.155 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:30:49.155 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:30:49.155 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:30:49.155 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:30:49.155 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:30:49.155 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:30:49.155 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:30:49.161 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:30:49.161 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:30:49.161 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:30:49.161 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:30:49.161 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:30:49.161 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:30:50.045 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1447 2024-09-12T13:30:50.049 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1448 2024-09-12T13:30:50.106 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:30:50.115 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:30:50.154 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1451 2024-09-12T13:30:50.173 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:30:50.180 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:30:50.180 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:30:50.181 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:30:50.181 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:30:50.181 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:30:50.181 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:30:51.058 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1450 2024-09-12T13:30:51.105 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:30:51.105 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:30:51.105 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:30:51.105 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:30:51.105 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:30:51.105 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:30:51.105 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:30:51.116 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1454 2024-09-12T13:30:51.117 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1452 2024-09-12T13:30:51.147 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:30:51.150 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:30:51.186 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:30:51.186 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:30:51.186 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:30:51.186 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:30:51.186 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:30:51.187 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:30:52.056 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1453 2024-09-12T13:30:52.097 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:30:52.145 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1456 2024-09-12T13:30:52.163 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:30:53.058 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1455 2024-09-12T13:30:53.080 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1457 2024-09-12T13:30:53.087 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:30:53.095 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:30:53.109 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:30:53.109 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:30:53.109 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:30:53.109 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:30:53.109 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:30:53.109 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:30:53.119 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:30:53.119 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:30:53.119 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:30:53.119 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:30:53.119 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:30:53.119 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:30:53.176 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:30:53.176 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:30:53.176 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:30:53.176 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:30:53.176 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:30:53.176 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:30:54.105 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1459 2024-09-12T13:30:54.108 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:30:54.108 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:30:54.108 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:30:54.108 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:30:54.108 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:30:54.108 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:30:54.115 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1458 2024-09-12T13:30:54.123 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:30:54.142 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:30:54.150 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:30:54.150 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:30:54.150 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:30:54.150 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:30:54.150 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:30:54.151 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:30:54.153 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:30:54.153 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:30:54.153 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:30:54.153 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:30:54.153 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:30:54.153 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:30:55.104 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:30:55.104 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:30:55.104 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:30:55.105 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:30:55.105 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:30:55.105 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:30:55.132 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1461 2024-09-12T13:30:55.151 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:30:55.165 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:30:55.165 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:30:55.166 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:30:55.166 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:30:55.166 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:30:55.166 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:30:56.073 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1460 2024-09-12T13:30:56.091 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:30:56.091 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:30:56.091 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:30:56.091 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:30:56.091 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:30:56.091 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:30:56.105 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:30:56.105 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:30:56.105 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:30:56.105 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:30:56.105 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:30:56.105 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:30:56.119 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1463 2024-09-12T13:30:56.129 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1462 2024-09-12T13:30:56.136 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:30:56.182 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:30:56.196 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:30:57.124 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1465 2024-09-12T13:30:57.129 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:30:57.129 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:30:57.129 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:30:57.129 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:30:57.129 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:30:57.129 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:30:57.145 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:30:57.145 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:30:57.146 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:30:57.146 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:30:57.146 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:30:57.146 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:30:57.149 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1464 2024-09-12T13:30:57.161 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:30:57.187 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:30:58.094 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1466 2024-09-12T13:30:58.102 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1467 2024-09-12T13:30:58.117 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:30:58.121 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:30:58.156 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:30:58.156 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:30:58.156 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:30:58.156 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:30:58.156 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:30:58.157 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:30:59.138 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1469 2024-09-12T13:30:59.141 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1468 2024-09-12T13:30:59.148 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:30:59.148 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:30:59.149 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:30:59.149 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:30:59.149 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:30:59.149 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:30:59.154 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:30:59.157 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:30:59.187 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:30:59.187 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:30:59.187 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:30:59.187 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:30:59.187 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:30:59.187 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:30:59.199 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:30:59.199 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:30:59.199 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:30:59.199 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:30:59.199 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:30:59.199 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:31:00.122 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1471 2024-09-12T13:31:00.139 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:31:00.165 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:31:00.165 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:31:00.165 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:31:00.165 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:31:00.165 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:31:00.165 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:31:00.194 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:31:00.194 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:31:00.195 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:31:00.195 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:31:00.195 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:31:00.195 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:31:01.090 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1470 2024-09-12T13:31:01.123 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:31:01.123 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:31:01.123 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:31:01.123 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:31:01.123 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:31:01.123 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:31:01.125 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:31:01.125 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:31:01.125 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:31:01.125 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:31:01.125 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:31:01.125 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:31:01.131 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1472 2024-09-12T13:31:01.134 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:31:01.160 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:31:02.102 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1473 2024-09-12T13:31:02.160 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:31:02.160 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:31:02.160 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:31:02.160 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:31:02.160 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:31:02.160 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:31:02.161 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:31:02.161 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:31:02.161 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:31:02.161 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:31:02.161 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:31:02.161 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:31:02.162 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1474 2024-09-12T13:31:02.164 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:31:02.173 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1475 2024-09-12T13:31:02.184 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:31:02.190 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:31:03.077 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1477 2024-09-12T13:31:03.089 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1476 2024-09-12T13:31:03.109 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:31:03.112 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:31:03.142 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:31:03.142 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:31:03.142 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:31:03.142 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:31:03.142 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:31:03.142 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:31:04.137 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:31:04.137 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:31:04.137 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:31:04.137 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:31:04.137 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:31:04.137 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:31:04.164 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:31:04.164 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:31:04.164 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:31:04.164 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:31:04.165 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:31:04.165 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:31:04.165 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1479 2024-09-12T13:31:04.165 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1478 2024-09-12T13:31:04.186 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:31:04.190 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:31:05.143 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1481 2024-09-12T13:31:05.157 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:31:05.168 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:31:05.168 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:31:05.168 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:31:05.168 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:31:05.168 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:31:05.169 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:31:05.187 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:31:05.187 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:31:05.187 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:31:05.187 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:31:05.188 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:31:05.188 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:31:05.193 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:31:05.193 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:31:05.193 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:31:05.193 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:31:05.193 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:31:05.193 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:31:06.103 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1480 2024-09-12T13:31:06.113 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:31:06.114 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:31:06.114 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:31:06.114 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:31:06.114 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:31:06.114 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:31:06.115 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:31:06.116 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:31:06.116 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:31:06.116 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:31:06.116 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:31:06.116 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:31:06.137 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:31:06.165 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1483 2024-09-12T13:31:06.182 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:31:07.118 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1482 2024-09-12T13:31:07.155 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:31:07.189 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:31:07.189 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:31:07.189 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:31:07.189 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:31:07.190 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:31:07.190 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:31:07.192 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:31:07.192 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:31:07.192 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:31:07.192 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:31:07.192 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:31:07.193 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:31:08.037 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1485 2024-09-12T13:31:08.044 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1484 2024-09-12T13:31:08.079 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:31:08.084 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:31:08.160 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:31:08.160 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:31:08.161 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:31:08.161 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:31:08.161 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:31:08.161 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:31:09.106 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1487 2024-09-12T13:31:09.110 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1486 2024-09-12T13:31:09.140 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:31:09.140 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:31:09.141 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:31:09.141 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:31:09.141 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:31:09.141 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:31:09.154 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:31:09.161 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:31:09.184 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:31:09.184 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:31:09.184 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:31:09.184 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:31:09.185 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:31:09.185 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:31:10.131 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1488 2024-09-12T13:31:10.134 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1489 2024-09-12T13:31:10.157 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:31:10.157 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:31:10.158 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:31:10.158 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:31:10.158 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:31:10.158 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:31:10.167 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:31:10.180 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:31:10.194 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1491 2024-09-12T13:31:10.213 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:31:11.083 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:31:11.083 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:31:11.083 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:31:11.083 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:31:11.083 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:31:11.083 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:31:11.087 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:31:11.088 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:31:11.088 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:31:11.088 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:31:11.088 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:31:11.088 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:31:11.124 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1490 2024-09-12T13:31:11.158 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:31:11.185 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1493 2024-09-12T13:31:11.202 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:31:12.130 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1492 2024-09-12T13:31:12.148 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:31:12.157 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:31:12.157 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:31:12.157 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:31:12.157 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:31:12.157 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:31:12.157 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:31:12.164 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:31:12.164 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:31:12.164 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:31:12.164 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:31:12.164 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:31:12.164 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:31:12.171 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1494 2024-09-12T13:31:12.174 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1495 2024-09-12T13:31:12.181 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:31:12.185 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:31:13.077 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1497 2024-09-12T13:31:13.082 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1496 2024-09-12T13:31:13.093 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:31:13.096 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:31:13.169 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:31:13.170 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:31:13.170 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:31:13.170 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:31:13.170 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:31:13.170 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:31:13.183 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:31:13.183 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:31:13.183 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:31:13.183 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:31:13.183 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:31:13.183 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:31:13.216 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:31:13.216 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:31:13.216 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:31:13.216 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:31:13.216 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:31:13.216 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:31:14.161 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:31:14.161 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:31:14.161 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:31:14.161 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:31:14.162 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:31:14.162 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:31:14.181 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1499 2024-09-12T13:31:14.185 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1498 2024-09-12T13:31:14.192 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:31:14.194 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:31:14.205 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:31:14.205 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:31:14.205 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:31:14.205 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:31:14.205 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:31:14.205 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:31:15.151 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:31:15.152 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:31:15.152 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:31:15.152 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:31:15.152 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:31:15.152 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:31:15.184 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:31:15.184 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:31:15.184 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:31:15.184 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:31:15.184 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:31:15.184 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:31:15.188 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:31:15.188 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:31:15.188 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:31:15.189 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:31:15.189 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:31:15.189 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:31:15.221 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1501 2024-09-12T13:31:15.231 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:31:16.097 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:31:16.097 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:31:16.097 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:31:16.097 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:31:16.097 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:31:16.097 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:31:16.100 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:31:16.101 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:31:16.101 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:31:16.101 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:31:16.101 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:31:16.101 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:31:16.149 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1500 2024-09-12T13:31:16.186 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:31:16.202 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:31:16.202+0000 7f67dc600640 -1 rbd::mirror::image_replayer::snapshot::Replayer: 0x56055c2db400 handle_unlink_peer: failed to unlink local peer from remote image: (22) Invalid argument 2024-09-12T13:31:16.234 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1503 2024-09-12T13:31:16.243 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:31:17.141 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1502 2024-09-12T13:31:17.179 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:31:17.196 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:31:17.196 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:31:17.196 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:31:17.197 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:31:17.197 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:31:17.197 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:31:17.197 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:31:17.197 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:31:17.197 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:31:17.197 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:31:17.197 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:31:17.197 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:31:18.039 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1505 2024-09-12T13:31:18.041 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1504 2024-09-12T13:31:18.090 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:31:18.102 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:31:18.115 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1507 2024-09-12T13:31:18.120 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1506 2024-09-12T13:31:18.151 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:31:18.161 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:31:18.234 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:31:18.234 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:31:18.234 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:31:18.234 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:31:18.234 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:31:18.234 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:31:19.189 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:31:19.189 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:31:19.189 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:31:19.189 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:31:19.189 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:31:19.189 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:31:19.246 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:31:19.247 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:31:19.247 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:31:19.247 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:31:19.247 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:31:19.247 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:31:20.145 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1509 2024-09-12T13:31:20.159 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1508 2024-09-12T13:31:20.180 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:31:20.182 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:31:20.182 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:31:20.182 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:31:20.182 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:31:20.182 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:31:20.182 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:31:20.193 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:31:20.227 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1511 2024-09-12T13:31:20.247 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:31:21.093 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:31:21.093 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:31:21.093 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:31:21.093 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:31:21.093 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:31:21.094 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:31:21.106 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:31:21.107 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:31:21.107 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:31:21.107 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:31:21.107 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:31:21.107 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:31:21.155 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1510 2024-09-12T13:31:21.156 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:31:21.157 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:31:21.157 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:31:21.157 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:31:21.157 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:31:21.157 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:31:21.164 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:31:21.165 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:31:21.165 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:31:21.165 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:31:21.165 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:31:21.165 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:31:21.184 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:31:21.213 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1513 2024-09-12T13:31:21.235 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:31:22.160 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1512 2024-09-12T13:31:22.188 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:31:22.206 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1515 2024-09-12T13:31:22.223 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1514 2024-09-12T13:31:22.223 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:31:22.232 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:31:23.083 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1517 2024-09-12T13:31:23.095 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1516 2024-09-12T13:31:23.119 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:31:23.125 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:31:23.183 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:31:23.184 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:31:23.184 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:31:23.184 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:31:23.184 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:31:23.184 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:31:23.196 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:31:23.196 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:31:23.196 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:31:23.196 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:31:23.196 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:31:23.196 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:31:23.249 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:31:23.249 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:31:23.249 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:31:23.250 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:31:23.250 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:31:23.250 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:31:24.187 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:31:24.187 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:31:24.187 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:31:24.187 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:31:24.187 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:31:24.188 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:31:24.201 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1518 2024-09-12T13:31:24.205 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1519 2024-09-12T13:31:24.213 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:31:24.216 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:31:24.237 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:31:24.237 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:31:24.237 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:31:24.238 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:31:24.238 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:31:24.238 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:31:25.191 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:31:25.191 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:31:25.191 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:31:25.191 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:31:25.191 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:31:25.191 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:31:25.211 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1521 2024-09-12T13:31:25.223 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:31:25.226 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:31:25.226 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:31:25.226 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:31:25.226 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:31:25.226 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:31:25.226 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:31:25.234 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:31:25.234 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:31:25.235 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:31:25.235 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:31:25.235 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:31:25.235 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:31:26.124 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:31:26.125 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:31:26.125 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:31:26.125 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:31:26.125 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:31:26.125 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:31:26.129 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:31:26.129 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:31:26.129 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:31:26.129 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:31:26.129 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:31:26.129 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:31:26.186 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1520 2024-09-12T13:31:26.217 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:31:26.265 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1523 2024-09-12T13:31:26.274 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:31:27.190 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1522 2024-09-12T13:31:27.216 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:31:27.216 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:31:27.216 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:31:27.216 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:31:27.216 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:31:27.216 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:31:27.219 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:31:27.219 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:31:27.219 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:31:27.219 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:31:27.219 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:31:27.219 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:31:27.221 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:31:28.036 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1524 2024-09-12T13:31:28.046 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1525 2024-09-12T13:31:28.098 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:31:28.098 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:31:28.103 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1526 2024-09-12T13:31:28.112 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1527 2024-09-12T13:31:28.143 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:31:28.147 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:31:28.226 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:31:28.226 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:31:28.226 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:31:28.226 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:31:28.226 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:31:28.226 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:31:29.092 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1528 2024-09-12T13:31:29.093 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1529 2024-09-12T13:31:29.105 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:31:29.117 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:31:29.220 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:31:29.220 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:31:29.220 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:31:29.221 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:31:29.221 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:31:29.221 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:31:29.276 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:31:29.277 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:31:29.277 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:31:29.277 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:31:29.277 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:31:29.277 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:31:30.069 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1530 2024-09-12T13:31:30.082 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:31:30.084 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1531 2024-09-12T13:31:30.100 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:31:30.224 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:31:30.224 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:31:30.225 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:31:30.225 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:31:30.225 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:31:30.225 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:31:31.086 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1532 2024-09-12T13:31:31.091 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1533 2024-09-12T13:31:31.102 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:31:31.103 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:31:31.104 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:31:31.104 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:31:31.104 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:31:31.104 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:31:31.104 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:31:31.104 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:31:31.105 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:31:31.105 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:31:31.105 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:31:31.105 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:31:31.105 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:31:31.113 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:31:31.149 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:31:31.149 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:31:31.149 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:31:31.149 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:31:31.149 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:31:31.149 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:31:31.153 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:31:31.153 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:31:31.153 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:31:31.153 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:31:31.153 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:31:31.153 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:31:32.094 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1535 2024-09-12T13:31:32.096 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1534 2024-09-12T13:31:32.109 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:31:32.110 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:31:32.110 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:31:32.111 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:31:32.111 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:31:32.111 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:31:32.121 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:31:32.122 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:31:32.122 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:31:32.122 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:31:32.123 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:31:32.123 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:31:32.123 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:31:32.126 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:31:33.081 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1536 2024-09-12T13:31:33.084 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:31:33.086 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:31:33.086 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:31:33.086 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:31:33.086 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:31:33.086 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:31:33.094 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1537 2024-09-12T13:31:33.105 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:31:33.105 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:31:33.106 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:31:33.106 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:31:33.106 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:31:33.106 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:31:33.116 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:31:33.123 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:31:34.107 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:31:34.108 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:31:34.108 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:31:34.108 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:31:34.108 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:31:34.108 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:31:34.109 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1539 2024-09-12T13:31:34.110 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1538 2024-09-12T13:31:34.120 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:31:34.120 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:31:34.120 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:31:34.120 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:31:34.120 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:31:34.120 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:31:34.140 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:31:34.161 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:31:35.123 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:31:35.124 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:31:35.124 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:31:35.124 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:31:35.124 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:31:35.124 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:31:35.132 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:31:35.132 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:31:35.132 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:31:35.132 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:31:35.132 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:31:35.132 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:31:36.063 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1540 2024-09-12T13:31:36.075 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1541 2024-09-12T13:31:36.120 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:31:36.124 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:31:36.124 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:31:36.124 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:31:36.125 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:31:36.125 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:31:36.125 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:31:36.128 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:31:36.128 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:31:36.128 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:31:36.128 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:31:36.128 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:31:36.128 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:31:36.148 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1543 2024-09-12T13:31:36.182 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:31:36.185 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1542 2024-09-12T13:31:36.188 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:31:36.200 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:31:37.098 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1545 2024-09-12T13:31:37.132 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1544 2024-09-12T13:31:37.137 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:31:37.146 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:31:37.146 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:31:37.147 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:31:37.147 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:31:37.148 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:31:37.148 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:31:37.158 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:31:37.167 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:31:37.167 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:31:37.167 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:31:37.168 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:31:37.168 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:31:37.168 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:31:38.085 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1547 2024-09-12T13:31:38.105 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1546 2024-09-12T13:31:38.119 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:31:38.128 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:31:39.123 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1549 2024-09-12T13:31:39.124 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1548 2024-09-12T13:31:39.124 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:31:39.125 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:31:39.125 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:31:39.126 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:31:39.126 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:31:39.126 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:31:39.145 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:31:39.148 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:31:39.187 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:31:39.187 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:31:39.187 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:31:39.187 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:31:39.187 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:31:39.187 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:31:39.192 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:31:39.192 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:31:39.193 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:31:39.193 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:31:39.193 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:31:39.193 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:31:39.203 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:31:39.204 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:31:39.204 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:31:39.204 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:31:39.204 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:31:39.204 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:31:40.120 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1550 2024-09-12T13:31:40.129 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1551 2024-09-12T13:31:40.140 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:31:40.141 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:31:40.141 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:31:40.141 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:31:40.141 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:31:40.141 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:31:40.141 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:31:40.141 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:31:40.162 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:31:40.162 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:31:40.162 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:31:40.163 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:31:40.163 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:31:40.163 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:31:41.122 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:31:41.122 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:31:41.122 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:31:41.122 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:31:41.122 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:31:41.122 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:31:41.131 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1552 2024-09-12T13:31:41.132 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:31:41.132 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:31:41.132 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:31:41.132 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:31:41.133 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:31:41.133 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:31:41.139 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1553 2024-09-12T13:31:41.159 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:31:41.163 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:31:42.125 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1554 2024-09-12T13:31:42.133 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1555 2024-09-12T13:31:42.150 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:31:42.150 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:31:42.150 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:31:42.150 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:31:42.150 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:31:42.150 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:31:42.151 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:31:42.152 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:31:42.152 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:31:42.152 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:31:42.152 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:31:42.152 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:31:42.152 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:31:42.156 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:31:43.084 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1556 2024-09-12T13:31:43.102 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1557 2024-09-12T13:31:43.109 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:31:43.115 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:31:43.143 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:31:43.143 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:31:43.143 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:31:43.144 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:31:43.144 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:31:43.144 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:31:43.144 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:31:43.144 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:31:43.144 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:31:43.144 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:31:43.144 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:31:43.145 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:31:44.157 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1559 2024-09-12T13:31:44.165 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:31:44.165 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:31:44.165 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:31:44.166 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:31:44.166 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:31:44.166 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:31:44.167 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:31:44.167 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:31:44.167 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:31:44.167 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:31:44.167 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:31:44.167 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:31:44.180 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:31:45.102 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1558 2024-09-12T13:31:45.152 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:31:45.159 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:31:45.159 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:31:45.159 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:31:45.159 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:31:45.159 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:31:45.159 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:31:45.160 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:31:45.160 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:31:45.160 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:31:45.160 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:31:45.160 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:31:45.161 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:31:46.113 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:31:46.113 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:31:46.114 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:31:46.114 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:31:46.114 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:31:46.114 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:31:46.119 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:31:46.119 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:31:46.119 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:31:46.119 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:31:46.119 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:31:46.119 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:31:46.121 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1560 2024-09-12T13:31:46.131 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1561 2024-09-12T13:31:46.167 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:31:46.171 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:31:47.113 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1562 2024-09-12T13:31:47.129 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1563 2024-09-12T13:31:47.150 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:31:47.177 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:31:47.184 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:31:47.185 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:31:47.185 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:31:47.185 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:31:47.185 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:31:47.185 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:31:48.074 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1564 2024-09-12T13:31:48.112 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1565 2024-09-12T13:31:48.154 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:31:48.155 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:31:48.155 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:31:48.155 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:31:48.155 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:31:48.155 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:31:48.155 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:31:48.162 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:31:48.167 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:31:48.167+0000 7f67dc600640 -1 rbd::mirror::image_replayer::snapshot::Replayer: 0x5605568d3400 handle_unlink_peer: failed to unlink local peer from remote image: (22) Invalid argument 2024-09-12T13:31:49.122 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1567 2024-09-12T13:31:49.128 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1566 2024-09-12T13:31:49.156 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:31:49.162 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:31:49.170 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:31:49.170 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:31:49.170 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:31:49.170 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:31:49.170 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:31:49.170 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:31:49.173 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:31:49.173 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:31:49.173 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:31:49.174 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:31:49.174 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:31:49.174 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:31:49.185 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1569 2024-09-12T13:31:49.197 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:31:50.115 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1568 2024-09-12T13:31:50.139 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:31:50.152 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:31:50.152 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:31:50.153 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:31:50.153 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:31:50.153 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:31:50.153 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:31:50.155 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1571 2024-09-12T13:31:50.165 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1570 2024-09-12T13:31:50.173 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:31:50.174 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:31:50.180 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:31:50.180 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:31:50.180 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:31:50.181 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:31:50.181 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:31:50.181 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:31:51.148 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1572 2024-09-12T13:31:51.155 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1573 2024-09-12T13:31:51.159 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:31:51.160 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:31:51.160 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:31:51.160 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:31:51.160 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:31:51.160 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:31:51.163 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:31:51.164 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:31:51.164 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:31:51.164 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:31:51.164 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:31:51.164 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:31:51.165 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:31:51.167 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:31:52.151 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1575 2024-09-12T13:31:52.159 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:31:52.159 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:31:52.159 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:31:52.159 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:31:52.159 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:31:52.159 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:31:52.163 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1574 2024-09-12T13:31:52.165 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:31:52.165 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:31:52.165 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:31:52.165 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:31:52.165 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:31:52.166 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:31:52.166 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:31:52.180 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:31:52.201 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:31:52.202 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:31:52.202 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:31:52.202 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:31:52.202 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:31:52.202 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:31:53.097 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1577 2024-09-12T13:31:53.098 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1576 2024-09-12T13:31:53.108 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:31:53.115 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:31:53.143 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:31:53.143 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:31:53.144 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:31:53.144 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:31:53.144 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:31:53.144 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:31:53.176 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:31:53.177 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:31:53.177 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:31:53.177 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:31:53.177 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:31:53.177 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:31:53.177 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:31:53.177 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:31:53.177 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:31:53.177 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:31:53.177 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:31:53.177 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:31:54.091 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1578 2024-09-12T13:31:54.105 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:31:54.111 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1579 2024-09-12T13:31:54.132 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:31:54.166 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:31:54.166 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:31:54.166 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:31:54.166 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:31:54.166 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:31:54.166 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:31:54.170 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:31:54.170 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:31:54.170 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:31:54.170 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:31:54.170 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:31:54.170 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:31:55.102 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:31:55.102+0000 7f67dd000640 -1 rbd::mirror::image_replayer::snapshot::Replayer: 0x560556bea400 handle_unlink_peer: failed to unlink local peer from remote image: (22) Invalid argument 2024-09-12T13:31:55.169 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:31:55.169 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:31:55.170 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:31:55.170 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:31:55.170 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:31:55.170 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:31:55.182 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:31:55.182 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:31:55.183 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:31:55.183 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:31:55.183 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:31:55.183 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:31:56.050 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1581 2024-09-12T13:31:56.065 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1580 2024-09-12T13:31:56.101 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:31:56.109 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1582 2024-09-12T13:31:56.110 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:31:56.110 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:31:56.110 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:31:56.110 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:31:56.111 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:31:56.111 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:31:56.111 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:31:56.117 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:31:56.117 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:31:56.118 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:31:56.118 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:31:56.118 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:31:56.118 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:31:56.142 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:31:57.063 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1583 2024-09-12T13:31:57.107 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1584 2024-09-12T13:31:57.108 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:31:57.108 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:31:57.108 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:31:57.108 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:31:57.108 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:31:57.108 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:31:57.111 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:31:57.134 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:31:57.134 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:31:57.134 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:31:57.135 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:31:57.135 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:31:57.135 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:31:57.143 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:31:57.164 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1587 2024-09-12T13:31:57.176 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:31:58.062 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1585 2024-09-12T13:31:58.086 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1586 2024-09-12T13:31:58.093 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:31:58.112 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:31:59.093 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1589 2024-09-12T13:31:59.104 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:31:59.104 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:31:59.104 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:31:59.104 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:31:59.104 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:31:59.104 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:31:59.112 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:31:59.113 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:31:59.113 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:31:59.113 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:31:59.113 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:31:59.113 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:31:59.119 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:31:59.146 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:31:59.146 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:31:59.146 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:31:59.146 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:31:59.146 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:31:59.146 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:32:00.060 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1588 2024-09-12T13:32:00.089 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:32:00.107 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1591 2024-09-12T13:32:00.114 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:32:00.114 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:32:00.114 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:32:00.114 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:32:00.115 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:32:00.115 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:32:00.115 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1590 2024-09-12T13:32:00.125 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:32:00.128 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:32:00.145 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:32:00.145 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:32:00.146 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:32:00.146 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:32:00.146 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:32:00.146 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:32:00.179 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:32:00.180 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:32:00.180 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:32:00.180 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:32:00.180 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:32:00.180 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:32:01.096 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:32:01.096 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:32:01.096 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:32:01.096 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:32:01.096 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:32:01.097 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:32:01.108 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1592 2024-09-12T13:32:01.115 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:32:01.115 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:32:01.115 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:32:01.115 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:32:01.115 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:32:01.115 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:32:01.130 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:32:01.132 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1595 2024-09-12T13:32:01.146 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1593 2024-09-12T13:32:01.152 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:32:01.159 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:32:02.093 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1594 2024-09-12T13:32:02.113 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:32:02.122 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:32:02.122 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:32:02.122 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:32:02.122 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:32:02.122 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:32:02.122 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:32:02.124 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1596 2024-09-12T13:32:02.137 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:32:03.068 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1597 2024-09-12T13:32:03.089 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:32:03.091 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:32:03.091 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:32:03.091 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:32:03.091 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:32:03.092 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:32:03.092 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:32:03.128 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:32:03.128 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:32:03.128 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:32:03.129 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:32:03.129 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:32:03.129 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:32:03.130 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:32:03.131 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:32:03.131 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:32:03.131 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:32:03.131 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:32:03.131 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:32:04.111 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1598 2024-09-12T13:32:04.134 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:32:04.134 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:32:04.134 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:32:04.134 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:32:04.134 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:32:04.134 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:32:04.135 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:32:04.136 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1601 2024-09-12T13:32:04.143 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1599 2024-09-12T13:32:04.154 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:32:04.155 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:32:04.155 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:32:04.155 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:32:04.155 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:32:04.155 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:32:04.155 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:32:04.162 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:32:04.162 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:32:04.162 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:32:04.162 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:32:04.162 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:32:04.163 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:32:04.191 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:32:05.116 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:32:05.116 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:32:05.116 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:32:05.117 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:32:05.117 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:32:05.117 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:32:05.140 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:32:05.140 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:32:05.141 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:32:05.141 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:32:05.141 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:32:05.141 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:32:05.154 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1604 2024-09-12T13:32:05.166 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:32:06.090 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1600 2024-09-12T13:32:06.094 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:32:06.094 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:32:06.094 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:32:06.094 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:32:06.094 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:32:06.094 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:32:06.137 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:32:06.173 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:32:06.173+0000 7f67dd000640 -1 rbd::mirror::image_replayer::snapshot::Replayer: 0x5605562aa800 handle_unlink_peer: failed to unlink local peer from remote image: (22) Invalid argument 2024-09-12T13:32:07.089 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1603 2024-09-12T13:32:07.113 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1602 2024-09-12T13:32:07.136 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:32:07.138 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:32:07.138 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:32:07.138 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:32:07.139 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:32:07.139 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:32:07.139 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:32:07.153 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:32:07.157 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:32:07.157 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:32:07.157 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:32:07.157 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:32:07.158 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:32:07.158 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:32:07.194 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:32:07.194 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:32:07.194 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:32:07.194 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:32:07.194 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:32:07.195 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:32:08.054 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1605 2024-09-12T13:32:08.066 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1606 2024-09-12T13:32:08.100 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:32:08.113 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:32:08.116 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1607 2024-09-12T13:32:08.128 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:32:08.156 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1610 2024-09-12T13:32:08.169 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:32:08.169 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:32:08.169 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:32:08.169 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:32:08.169 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:32:08.170 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:32:08.170 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:32:09.130 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1609 2024-09-12T13:32:09.140 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:32:09.140 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:32:09.141 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:32:09.141 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:32:09.141 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:32:09.141 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:32:09.167 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:32:10.087 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1611 2024-09-12T13:32:10.098 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1608 2024-09-12T13:32:10.123 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:32:10.130 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:32:10.140 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:32:10.140 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:32:10.140 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:32:10.140 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:32:10.140 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:32:10.140 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:32:10.155 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1612 2024-09-12T13:32:10.156 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:32:10.156 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:32:10.156 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:32:10.156 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:32:10.156 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:32:10.156 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:32:10.167 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:32:11.105 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:32:11.105 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:32:11.105 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:32:11.105 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:32:11.105 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:32:11.105 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:32:11.116 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:32:11.116 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:32:11.117 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:32:11.117 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:32:11.117 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:32:11.117 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:32:11.127 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1614 2024-09-12T13:32:11.131 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:32:11.131 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:32:11.131 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:32:11.131 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:32:11.132 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:32:11.132 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:32:11.139 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1613 2024-09-12T13:32:11.148 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:32:11.149 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:32:11.172 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:32:11.173 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:32:11.173 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:32:11.173 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:32:11.173 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:32:11.173 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:32:12.159 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1618 2024-09-12T13:32:12.164 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1616 2024-09-12T13:32:12.165 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1615 2024-09-12T13:32:12.170 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:32:12.170 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:32:12.170 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:32:12.170 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:32:12.170 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:32:12.170 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:32:12.177 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:32:12.177 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:32:12.184 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:32:13.084 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1617 2024-09-12T13:32:13.105 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:32:13.127 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:32:13.127 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:32:13.127 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:32:13.127 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:32:13.127 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:32:13.127 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:32:13.135 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:32:13.135 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:32:13.135 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:32:13.135 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:32:13.135 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:32:13.135 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:32:13.170 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:32:13.170 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:32:13.170 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:32:13.170 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:32:13.170 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:32:13.171 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:32:14.133 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1619 2024-09-12T13:32:14.152 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:32:14.152 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:32:14.152 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:32:14.152 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:32:14.152 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:32:14.152 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:32:14.153 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:32:14.153 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:32:14.154 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:32:14.154 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:32:14.154 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:32:14.154 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:32:14.163 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:32:14.171 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1621 2024-09-12T13:32:14.187 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:32:15.134 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1620 2024-09-12T13:32:15.165 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:32:15.180 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:32:15.180 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:32:15.180 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:32:15.180 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:32:15.180 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:32:15.180 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:32:15.180 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:32:15.180 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:32:15.181 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:32:15.181 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:32:15.181 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:32:15.181 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:32:15.198 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1624 2024-09-12T13:32:15.211 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:32:16.107 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:32:16.108 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:32:16.108 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:32:16.108 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:32:16.108 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:32:16.108 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:32:16.134 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1622 2024-09-12T13:32:16.162 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:32:17.139 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1623 2024-09-12T13:32:17.168 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:32:17.168 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:32:17.168 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:32:17.169 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:32:17.169 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:32:17.169 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:32:17.183 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:32:17.190 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${SNAP_PIDS[@]} 2024-09-12T13:32:17.190 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 82868 2024-09-12T13:32:17.190 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${SNAP_PIDS[@]} 2024-09-12T13:32:17.190 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 83000 2024-09-12T13:32:18.053 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1625 2024-09-12T13:32:18.061 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1626 2024-09-12T13:32:18.105 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:32:18.117 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1627 2024-09-12T13:32:18.146 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:32:18.158 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:32:18.168 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:32:18.168 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:32:18.168 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:32:18.168 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:32:18.168 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:32:18.168 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:32:18.213 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${SNAP_PIDS[@]} 2024-09-12T13:32:18.213 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 83132 2024-09-12T13:32:19.165 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:32:19.165 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:32:19.165 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:32:19.165 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:32:19.165 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:32:19.165 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:32:20.127 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1628 2024-09-12T13:32:20.175 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:32:20.185 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:32:20.185 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:32:20.185 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:32:20.185 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:32:20.186 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:32:20.186 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:32:20.198 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1629 2024-09-12T13:32:20.208 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:32:20.213 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1630 2024-09-12T13:32:20.226 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:32:21.109 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:32:21.110 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:32:21.110 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:32:21.110 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:32:21.110 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:32:21.110 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:32:21.150 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${SNAP_PIDS[@]} 2024-09-12T13:32:21.150 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 83284 2024-09-12T13:32:21.157 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1631 2024-09-12T13:32:21.160 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${SNAP_PIDS[@]} 2024-09-12T13:32:21.160 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 83437 2024-09-12T13:32:21.170 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:32:22.166 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1632 2024-09-12T13:32:22.180 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:32:23.178 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${SNAP_PIDS[@]} 2024-09-12T13:32:23.178 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 83590 2024-09-12T13:32:23.229 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${SNAP_PIDS[@]} 2024-09-12T13:32:23.229 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 83743 2024-09-12T13:32:24.173 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${SNAP_PIDS[@]} 2024-09-12T13:32:24.173 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 83917 2024-09-12T13:32:25.183 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${SNAP_PIDS[@]} 2024-09-12T13:32:25.183 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 84091 2024-09-12T13:32:25.183 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ (( RET != 0 )) 2024-09-12T13:32:25.183 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${WORKLOAD_PIDS[@]} 2024-09-12T13:32:25.183 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 82737 2024-09-12T13:34:52.830 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ ret=124 2024-09-12T13:34:52.831 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ (( ret != 124 )) 2024-09-12T13:34:52.831 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${WORKLOAD_PIDS[@]} 2024-09-12T13:34:52.831 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 82869 2024-09-12T13:34:53.889 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ ret=124 2024-09-12T13:34:53.889 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ (( ret != 124 )) 2024-09-12T13:34:53.889 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${WORKLOAD_PIDS[@]} 2024-09-12T13:34:53.889 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 83001 2024-09-12T13:34:54.892 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ ret=124 2024-09-12T13:34:54.892 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ (( ret != 124 )) 2024-09-12T13:34:54.892 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${WORKLOAD_PIDS[@]} 2024-09-12T13:34:54.892 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 83133 2024-09-12T13:34:55.717 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ ret=124 2024-09-12T13:34:55.717 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ (( ret != 124 )) 2024-09-12T13:34:55.717 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${WORKLOAD_PIDS[@]} 2024-09-12T13:34:55.717 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 83285 2024-09-12T13:34:56.745 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ ret=124 2024-09-12T13:34:56.745 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ (( ret != 124 )) 2024-09-12T13:34:56.745 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${WORKLOAD_PIDS[@]} 2024-09-12T13:34:56.745 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 83438 2024-09-12T13:34:57.744 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ ret=124 2024-09-12T13:34:57.744 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ (( ret != 124 )) 2024-09-12T13:34:57.744 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${WORKLOAD_PIDS[@]} 2024-09-12T13:34:57.744 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 83591 2024-09-12T13:34:58.735 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ ret=124 2024-09-12T13:34:58.735 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ (( ret != 124 )) 2024-09-12T13:34:58.735 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${WORKLOAD_PIDS[@]} 2024-09-12T13:34:58.735 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 83744 2024-09-12T13:34:59.818 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ ret=124 2024-09-12T13:34:59.818 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ (( ret != 124 )) 2024-09-12T13:34:59.819 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${WORKLOAD_PIDS[@]} 2024-09-12T13:34:59.819 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 83918 2024-09-12T13:35:00.832 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ ret=124 2024-09-12T13:35:00.832 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ (( ret != 124 )) 2024-09-12T13:35:00.833 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${WORKLOAD_PIDS[@]} 2024-09-12T13:35:00.833 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 84092 2024-09-12T13:35:01.870 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ ret=124 2024-09-12T13:35:01.871 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ (( ret != 124 )) 2024-09-12T13:35:01.871 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ (( RET != 0 )) 2024-09-12T13:35:01.871 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:35:01.872 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ COMPARE_PIDS+=($!) 2024-09-12T13:35:01.872 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:35:01.872 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ compare_demoted_promoted_image 1 2024-09-12T13:35:01.873 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local dev=/dev/rbd0 2024-09-12T13:35:01.873 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local img=image-primary1 2024-09-12T13:35:01.873 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mntpt=test-primary1 2024-09-12T13:35:01.873 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local demote_md5 promote_md5 2024-09-12T13:35:01.873 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo umount test-primary1 2024-09-12T13:35:01.873 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ COMPARE_PIDS+=($!) 2024-09-12T13:35:01.873 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:35:01.873 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ compare_demoted_promoted_image 2 2024-09-12T13:35:01.875 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local dev=/dev/rbd1 2024-09-12T13:35:01.875 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local img=image-primary2 2024-09-12T13:35:01.875 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mntpt=test-primary2 2024-09-12T13:35:01.875 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local demote_md5 promote_md5 2024-09-12T13:35:01.875 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ COMPARE_PIDS+=($!) 2024-09-12T13:35:01.875 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo umount test-primary2 2024-09-12T13:35:01.876 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:35:01.876 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ compare_demoted_promoted_image 3 2024-09-12T13:35:01.876 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local dev=/dev/rbd2 2024-09-12T13:35:01.876 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local img=image-primary3 2024-09-12T13:35:01.876 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mntpt=test-primary3 2024-09-12T13:35:01.876 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local demote_md5 promote_md5 2024-09-12T13:35:01.876 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo umount test-primary3 2024-09-12T13:35:01.876 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ COMPARE_PIDS+=($!) 2024-09-12T13:35:01.877 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ compare_demoted_promoted_image 4 2024-09-12T13:35:01.877 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:35:01.877 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local dev=/dev/rbd3 2024-09-12T13:35:01.877 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local img=image-primary4 2024-09-12T13:35:01.877 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mntpt=test-primary4 2024-09-12T13:35:01.877 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local demote_md5 promote_md5 2024-09-12T13:35:01.877 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo umount test-primary4 2024-09-12T13:35:01.877 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ COMPARE_PIDS+=($!) 2024-09-12T13:35:01.877 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:35:01.879 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ compare_demoted_promoted_image 5 2024-09-12T13:35:01.879 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local dev=/dev/rbd4 2024-09-12T13:35:01.879 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local img=image-primary5 2024-09-12T13:35:01.880 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mntpt=test-primary5 2024-09-12T13:35:01.880 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local demote_md5 promote_md5 2024-09-12T13:35:01.880 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo umount test-primary5 2024-09-12T13:35:01.880 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ COMPARE_PIDS+=($!) 2024-09-12T13:35:01.880 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:35:01.880 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ compare_demoted_promoted_image 6 2024-09-12T13:35:01.880 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local dev=/dev/rbd5 2024-09-12T13:35:01.880 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local img=image-primary6 2024-09-12T13:35:01.880 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ COMPARE_PIDS+=($!) 2024-09-12T13:35:01.880 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mntpt=test-primary6 2024-09-12T13:35:01.880 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local demote_md5 promote_md5 2024-09-12T13:35:01.880 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo umount test-primary6 2024-09-12T13:35:01.880 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:35:01.881 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ compare_demoted_promoted_image 7 2024-09-12T13:35:01.881 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local dev=/dev/rbd6 2024-09-12T13:35:01.881 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local img=image-primary7 2024-09-12T13:35:01.881 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mntpt=test-primary7 2024-09-12T13:35:01.881 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local demote_md5 promote_md5 2024-09-12T13:35:01.881 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo umount test-primary7 2024-09-12T13:35:01.881 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ COMPARE_PIDS+=($!) 2024-09-12T13:35:01.881 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:35:01.881 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ compare_demoted_promoted_image 8 2024-09-12T13:35:01.881 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local dev=/dev/rbd7 2024-09-12T13:35:01.881 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ COMPARE_PIDS+=($!) 2024-09-12T13:35:01.881 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:35:01.882 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ COMPARE_PIDS+=($!) 2024-09-12T13:35:01.882 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${COMPARE_PIDS[@]} 2024-09-12T13:35:01.883 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 90045 2024-09-12T13:35:01.883 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local img=image-primary8 2024-09-12T13:35:01.883 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mntpt=test-primary8 2024-09-12T13:35:01.883 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local demote_md5 promote_md5 2024-09-12T13:35:01.883 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo umount test-primary8 2024-09-12T13:35:01.883 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ compare_demoted_promoted_image 10 2024-09-12T13:35:01.883 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local dev=/dev/rbd9 2024-09-12T13:35:01.883 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local img=image-primary10 2024-09-12T13:35:01.883 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mntpt=test-primary10 2024-09-12T13:35:01.883 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local demote_md5 promote_md5 2024-09-12T13:35:01.883 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo umount test-primary10 2024-09-12T13:35:01.883 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ compare_demoted_promoted_image 9 2024-09-12T13:35:01.883 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local dev=/dev/rbd8 2024-09-12T13:35:01.883 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local img=image-primary9 2024-09-12T13:35:01.884 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mntpt=test-primary9 2024-09-12T13:35:01.884 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local demote_md5 promote_md5 2024-09-12T13:35:01.884 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo umount test-primary9 2024-09-12T13:35:02.717 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo md5sum /dev/rbd0 2024-09-12T13:35:02.719 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ awk '{print $1}' 2024-09-12T13:35:02.761 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo md5sum /dev/rbd5 2024-09-12T13:35:02.763 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ awk '{print $1}' 2024-09-12T13:35:02.773 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo md5sum /dev/rbd9 2024-09-12T13:35:02.775 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ awk '{print $1}' 2024-09-12T13:35:02.778 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo md5sum /dev/rbd6 2024-09-12T13:35:02.780 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ awk '{print $1}' 2024-09-12T13:35:02.780 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo md5sum /dev/rbd2 2024-09-12T13:35:02.781 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ awk '{print $1}' 2024-09-12T13:35:02.821 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo md5sum /dev/rbd8 2024-09-12T13:35:02.824 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ awk '{print $1}' 2024-09-12T13:35:02.824 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo md5sum /dev/rbd3 2024-09-12T13:35:02.827 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ awk '{print $1}' 2024-09-12T13:35:02.840 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo md5sum /dev/rbd1 2024-09-12T13:35:02.842 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ awk '{print $1}' 2024-09-12T13:35:02.853 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo md5sum /dev/rbd7 2024-09-12T13:35:02.854 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ awk '{print $1}' 2024-09-12T13:35:02.906 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ awk '{print $1}' 2024-09-12T13:35:02.907 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo md5sum /dev/rbd4 2024-09-12T13:35:54.442 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ demote_md5=d6bd25670a3cb91d0ea71320af1306ab 2024-09-12T13:35:54.443 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo rbd --cluster cluster1 device unmap -t krbd mirror/image-primary8 2024-09-12T13:35:55.005 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ demote_image cluster1 mirror image-primary8 2024-09-12T13:35:55.005 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:35:55.006 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:35:55.006 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:35:55.006 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 mirror image demote mirror/image-primary8 2024-09-12T13:35:55.176 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ demote_md5=c158e231ec70974d8764bac597b398f3 2024-09-12T13:35:55.176 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo rbd --cluster cluster1 device unmap -t krbd mirror/image-primary3 2024-09-12T13:35:55.386 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ demote_image cluster1 mirror image-primary3 2024-09-12T13:35:55.386 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:35:55.386 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:35:55.386 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:35:55.386 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 mirror image demote mirror/image-primary3 2024-09-12T13:35:56.422 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ demote_md5=9fcb612174a84a4dc5dc4639dffa95bd 2024-09-12T13:35:56.422 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo rbd --cluster cluster1 device unmap -t krbd mirror/image-primary5 2024-09-12T13:35:56.516 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Image demoted to non-primary 2024-09-12T13:35:56.603 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary8 up+unknown 2024-09-12T13:35:56.603 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:35:56.603 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:35:56.603 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:35:56.603 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:35:56.603 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:35:56.603 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:35:56.603 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:35:56.603 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:35:56.781 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ demote_image cluster1 mirror image-primary5 2024-09-12T13:35:56.781 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:35:56.782 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:35:56.782 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:35:56.782 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 mirror image demote mirror/image-primary5 2024-09-12T13:35:56.922 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ demote_md5=89b6dc0f5d258aa7b152b3b7f3ae9ffe 2024-09-12T13:35:56.922 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo rbd --cluster cluster1 device unmap -t krbd mirror/image-primary1 2024-09-12T13:35:57.198 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ demote_image cluster1 mirror image-primary1 2024-09-12T13:35:57.198 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:35:57.199 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:35:57.199 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:35:57.199 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 mirror image demote mirror/image-primary1 2024-09-12T13:35:57.644 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary8 up+unknown '' '' 2024-09-12T13:35:57.644 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:35:57.644 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:35:57.644 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:35:57.644 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:35:57.644 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:35:57.644 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:35:57.644 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:35:57.647 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:35:57.647 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary8 2024-09-12T13:35:57.795 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ demote_md5=3d029a8d927783cfa30b5ad87d7fd36d 2024-09-12T13:35:57.795 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo rbd --cluster cluster1 device unmap -t krbd mirror/image-primary6 2024-09-12T13:35:57.866 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary8: 2024-09-12T13:35:57.866 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: b415829c-3d1c-41f6-a1aa-e76e40e110d1 2024-09-12T13:35:57.866 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:35:57.866 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:35:57.867 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:35:57.867 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:35:47' 2024-09-12T13:35:57.867 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:35:57.871 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:35:57.871 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:35:57.872 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 2 2024-09-12T13:35:57.980 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ demote_md5=c766e1b9b40887e252f808b7c8765512 2024-09-12T13:35:57.981 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo rbd --cluster cluster1 device unmap -t krbd mirror/image-primary2 2024-09-12T13:35:57.982 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ demote_image cluster1 mirror image-primary6 2024-09-12T13:35:57.982 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:35:57.982 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:35:57.982 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:35:57.982 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 mirror image demote mirror/image-primary6 2024-09-12T13:35:58.287 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ demote_image cluster1 mirror image-primary2 2024-09-12T13:35:58.287 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:35:58.287 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:35:58.287 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:35:58.287 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 mirror image demote mirror/image-primary2 2024-09-12T13:35:58.418 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ demote_md5=bc43079dacea4896e58f625c488ff1a1 2024-09-12T13:35:58.419 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo rbd --cluster cluster1 device unmap -t krbd mirror/image-primary7 2024-09-12T13:35:58.515 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Image demoted to non-primary 2024-09-12T13:35:58.518 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Image demoted to non-primary 2024-09-12T13:35:58.540 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Image demoted to non-primary 2024-09-12T13:35:58.570 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary1 up+unknown 2024-09-12T13:35:58.570 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:35:58.570 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:35:58.570 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:35:58.570 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:35:58.570 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:35:58.570 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:35:58.570 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:35:58.570 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:35:58.571 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary3 up+unknown 2024-09-12T13:35:58.571 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:35:58.571 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:35:58.571 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:35:58.571 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:35:58.571 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:35:58.571 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:35:58.572 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:35:58.572 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:35:58.589 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Image demoted to non-primary 2024-09-12T13:35:58.593 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary5 up+unknown 2024-09-12T13:35:58.593 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:35:58.595 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:35:58.595 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:35:58.595 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:35:58.595 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:35:58.595 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:35:58.596 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:35:58.596 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:35:58.664 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary6 up+unknown 2024-09-12T13:35:58.665 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:35:58.665 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:35:58.665 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:35:58.665 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:35:58.665 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:35:58.665 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:35:58.665 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:35:58.665 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:35:58.665 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ demote_image cluster1 mirror image-primary7 2024-09-12T13:35:58.665 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:35:58.665 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:35:58.665 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:35:58.665 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 mirror image demote mirror/image-primary7 2024-09-12T13:35:58.892 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ demote_md5=c9ae793770c6420b1bb1ca9b01a292ed 2024-09-12T13:35:58.893 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo rbd --cluster cluster1 device unmap -t krbd mirror/image-primary4 2024-09-12T13:35:58.963 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ demote_md5=639b299f6db074096373eb56deafce73 2024-09-12T13:35:58.963 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo rbd --cluster cluster1 device unmap -t krbd mirror/image-primary10 2024-09-12T13:35:59.020 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ demote_image cluster1 mirror image-primary4 2024-09-12T13:35:59.020 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:35:59.020 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:35:59.020 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:35:59.020 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 mirror image demote mirror/image-primary4 2024-09-12T13:35:59.086 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ demote_image cluster1 mirror image-primary10 2024-09-12T13:35:59.087 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:35:59.087 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:35:59.087 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:35:59.087 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 mirror image demote mirror/image-primary10 2024-09-12T13:35:59.522 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Image demoted to non-primary 2024-09-12T13:35:59.545 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary4 up+unknown 2024-09-12T13:35:59.546 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:35:59.546 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:35:59.546 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:35:59.546 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:35:59.546 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:35:59.546 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:35:59.547 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:35:59.547 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:35:59.584 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary1 up+unknown '' '' 2024-09-12T13:35:59.584 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary3 up+unknown '' '' 2024-09-12T13:35:59.584 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:35:59.584 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:35:59.584 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:35:59.585 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:35:59.585 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:35:59.585 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:35:59.585 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:35:59.585 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:35:59.585 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:35:59.585 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:35:59.585 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:35:59.586 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:35:59.586 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:35:59.586 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:35:59.586 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:35:59.586 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary1 2024-09-12T13:35:59.586 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:35:59.587 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary3 2024-09-12T13:35:59.601 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary5 up+unknown '' '' 2024-09-12T13:35:59.601 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:35:59.601 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:35:59.601 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:35:59.601 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:35:59.601 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:35:59.601 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:35:59.601 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:35:59.602 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:35:59.602 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary5 2024-09-12T13:35:59.670 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary6 up+unknown '' '' 2024-09-12T13:35:59.670 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:35:59.670 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:35:59.671 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:35:59.671 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:35:59.671 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:35:59.671 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:35:59.671 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:35:59.671 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:35:59.671 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary6 2024-09-12T13:35:59.673 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary3: 2024-09-12T13:35:59.673 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 9cc4db57-5517-41a1-8d14-519d4043d1a5 2024-09-12T13:35:59.673 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:35:59.673 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:35:59.674 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:35:59.674 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:35:47' 2024-09-12T13:35:59.674 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:35:59.676 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:35:59.677 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:35:59.677 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 2 2024-09-12T13:35:59.683 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary5: 2024-09-12T13:35:59.683 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 95f21a09-f11d-496c-bf07-0203910a05a6 2024-09-12T13:35:59.683 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:35:59.683 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:35:59.684 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:35:59.684 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:35:47' 2024-09-12T13:35:59.684 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:35:59.686 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary1: 2024-09-12T13:35:59.686 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 4ee25370-f977-4ecf-acda-dc9a3ae2ee9d 2024-09-12T13:35:59.686 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:35:59.686 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:35:59.687 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:35:59.687 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:35:47' 2024-09-12T13:35:59.687 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:35:59.687 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:35:59.687 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:35:59.687 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 2 2024-09-12T13:35:59.689 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:35:59.689 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:35:59.689 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 2 2024-09-12T13:35:59.734 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ demote_md5=85577d212ced413d69d145a64948a111 2024-09-12T13:35:59.734 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo rbd --cluster cluster1 device unmap -t krbd mirror/image-primary9 2024-09-12T13:35:59.742 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary6: 2024-09-12T13:35:59.742 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: dcf3f6ae-ecc8-4d77-9308-bede00122fcd 2024-09-12T13:35:59.742 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:35:59.742 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:35:59.742 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:35:59.742 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:35:47' 2024-09-12T13:35:59.742 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:35:59.745 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:35:59.745 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:35:59.745 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 2 2024-09-12T13:35:59.840 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ demote_image cluster1 mirror image-primary9 2024-09-12T13:35:59.840 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:35:59.840 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:35:59.841 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:35:59.841 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 mirror image demote mirror/image-primary9 2024-09-12T13:35:59.875 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary8 up+unknown '' '' 2024-09-12T13:35:59.876 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:35:59.876 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:35:59.876 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:35:59.876 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:35:59.876 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:35:59.876 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:35:59.876 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:35:59.876 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:35:59.876 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary8 2024-09-12T13:35:59.949 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary8: 2024-09-12T13:35:59.949 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: b415829c-3d1c-41f6-a1aa-e76e40e110d1 2024-09-12T13:35:59.949 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:35:59.949 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:35:59.950 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:35:59.950 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:35:47' 2024-09-12T13:35:59.950 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:35:59.952 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:35:59.952 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:35:59.952 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 4 2024-09-12T13:36:00.533 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Image demoted to non-primary 2024-09-12T13:36:00.551 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary4 up+unknown '' '' 2024-09-12T13:36:00.551 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:36:00.551 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:36:00.551 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:36:00.552 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:36:00.552 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:36:00.552 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:36:00.552 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:36:00.555 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:36:00.555 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary4 2024-09-12T13:36:00.592 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary7 up+unknown 2024-09-12T13:36:00.592 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:36:00.592 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:36:00.592 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:36:00.592 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:36:00.592 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:36:00.592 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:36:00.592 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:36:00.592 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:36:00.740 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary4: 2024-09-12T13:36:00.740 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 0203765c-0327-4045-b74a-e9bc90c8f4a0 2024-09-12T13:36:00.740 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:36:00.741 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:36:00.741 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:36:00.741 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:35:47' 2024-09-12T13:36:00.741 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:36:00.749 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:36:00.750 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:36:00.750 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 2 2024-09-12T13:36:01.524 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Image demoted to non-primary 2024-09-12T13:36:01.559 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary9 up+unknown 2024-09-12T13:36:01.559 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:36:01.559 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:36:01.560 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:36:01.560 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:36:01.560 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:36:01.560 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:36:01.560 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:36:01.560 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:36:01.596 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary7 up+unknown '' '' 2024-09-12T13:36:01.596 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:36:01.596 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:36:01.596 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:36:01.596 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:36:01.596 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:36:01.596 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:36:01.596 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:36:01.597 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:36:01.597 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary7 2024-09-12T13:36:01.680 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary3 up+unknown '' '' 2024-09-12T13:36:01.680 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:36:01.680 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:36:01.680 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:36:01.680 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:36:01.680 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:36:01.680 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:36:01.680 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:36:01.681 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:36:01.681 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary3 2024-09-12T13:36:01.689 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary5 up+unknown '' '' 2024-09-12T13:36:01.689 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:36:01.689 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:36:01.689 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:36:01.689 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:36:01.689 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:36:01.689 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:36:01.690 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:36:01.690 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:36:01.690 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary5 2024-09-12T13:36:01.692 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary1 up+unknown '' '' 2024-09-12T13:36:01.692 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:36:01.692 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:36:01.692 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:36:01.693 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:36:01.693 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:36:01.693 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:36:01.693 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:36:01.693 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:36:01.693 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary1 2024-09-12T13:36:01.694 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary7: 2024-09-12T13:36:01.695 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: da6437a6-d0c6-4534-a0e0-60fb07f54f52 2024-09-12T13:36:01.695 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:36:01.695 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:36:01.695 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:36:01.695 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:35:47' 2024-09-12T13:36:01.695 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:36:01.699 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:36:01.699 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:36:01.699 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 2 2024-09-12T13:36:01.747 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary6 up+unknown '' '' 2024-09-12T13:36:01.748 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:36:01.748 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:36:01.748 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:36:01.748 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:36:01.748 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:36:01.748 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:36:01.748 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:36:01.748 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:36:01.749 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary6 2024-09-12T13:36:01.762 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary3: 2024-09-12T13:36:01.762 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 9cc4db57-5517-41a1-8d14-519d4043d1a5 2024-09-12T13:36:01.762 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:36:01.762 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:36:01.763 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:36:01.763 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:35:47' 2024-09-12T13:36:01.763 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:36:01.763 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary5: 2024-09-12T13:36:01.763 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 95f21a09-f11d-496c-bf07-0203910a05a6 2024-09-12T13:36:01.764 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:36:01.764 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:36:01.764 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:36:01.764 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:35:47' 2024-09-12T13:36:01.764 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:36:01.765 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:36:01.765 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:36:01.765 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 4 2024-09-12T13:36:01.766 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:36:01.766 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:36:01.766 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 4 2024-09-12T13:36:01.766 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary1: 2024-09-12T13:36:01.767 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 4ee25370-f977-4ecf-acda-dc9a3ae2ee9d 2024-09-12T13:36:01.767 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:36:01.767 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:36:01.767 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:36:01.767 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:35:47' 2024-09-12T13:36:01.767 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:36:01.769 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:36:01.769 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:36:01.769 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 4 2024-09-12T13:36:01.825 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary6: 2024-09-12T13:36:01.826 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: dcf3f6ae-ecc8-4d77-9308-bede00122fcd 2024-09-12T13:36:01.826 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:36:01.826 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:36:01.828 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:36:01.828 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:35:47' 2024-09-12T13:36:01.828 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:36:01.829 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:36:01.829 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:36:01.829 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 4 2024-09-12T13:36:02.563 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary9 up+unknown '' '' 2024-09-12T13:36:02.564 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:36:02.564 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:36:02.564 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:36:02.564 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:36:02.564 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:36:02.564 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:36:02.564 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:36:02.564 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:36:02.564 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary9 2024-09-12T13:36:02.637 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary9: 2024-09-12T13:36:02.638 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 1a8d4f4e-16a8-41c0-8b42-954a67b2e30d 2024-09-12T13:36:02.638 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:36:02.638 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:36:02.638 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:36:02.638 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:35:47' 2024-09-12T13:36:02.638 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:36:02.640 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:36:02.640 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:36:02.641 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 2 2024-09-12T13:36:02.754 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary4 up+unknown '' '' 2024-09-12T13:36:02.754 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:36:02.754 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:36:02.754 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:36:02.754 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:36:02.754 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:36:02.755 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:36:02.755 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:36:02.755 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:36:02.755 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary4 2024-09-12T13:36:02.828 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary4: 2024-09-12T13:36:02.828 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 0203765c-0327-4045-b74a-e9bc90c8f4a0 2024-09-12T13:36:02.828 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:36:02.828 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:36:02.828 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:36:02.828 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:35:47' 2024-09-12T13:36:02.828 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:36:02.830 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:36:02.830 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:36:02.831 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 4 2024-09-12T13:36:03.093 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Image demoted to non-primary 2024-09-12T13:36:03.110 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Image demoted to non-primary 2024-09-12T13:36:03.112 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary2 up+unknown 2024-09-12T13:36:03.112 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:36:03.112 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:36:03.112 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:36:03.112 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:36:03.112 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:36:03.112 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:36:03.112 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:36:03.112 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:36:03.127 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary10 up+unknown 2024-09-12T13:36:03.127 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:36:03.127 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:36:03.127 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:36:03.127 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:36:03.127 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:36:03.127 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:36:03.127 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:36:03.127 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:36:03.702 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary7 up+unknown '' '' 2024-09-12T13:36:03.702 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:36:03.702 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:36:03.702 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:36:03.702 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:36:03.702 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:36:03.702 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:36:03.702 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:36:03.703 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:36:03.703 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary7 2024-09-12T13:36:03.783 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary7: 2024-09-12T13:36:03.783 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: da6437a6-d0c6-4534-a0e0-60fb07f54f52 2024-09-12T13:36:03.783 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:36:03.783 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:36:03.783 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:36:03.783 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:35:47' 2024-09-12T13:36:03.783 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:36:03.785 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:36:03.785 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:36:03.786 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 4 2024-09-12T13:36:03.954 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary8 up+unknown '' '' 2024-09-12T13:36:03.954 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:36:03.955 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:36:03.955 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:36:03.955 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:36:03.955 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:36:03.955 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:36:03.955 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:36:03.955 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:36:03.955 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary8 2024-09-12T13:36:04.033 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary8: 2024-09-12T13:36:04.033 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: b415829c-3d1c-41f6-a1aa-e76e40e110d1 2024-09-12T13:36:04.033 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:36:04.033 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:36:04.033 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:36:04.033 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:35:47' 2024-09-12T13:36:04.033 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:36:04.035 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:36:04.036 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:36:04.036 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 8 2024-09-12T13:36:04.115 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary2 up+unknown '' '' 2024-09-12T13:36:04.115 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:36:04.115 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:36:04.115 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:36:04.115 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:36:04.115 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:36:04.115 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:36:04.115 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:36:04.115 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:36:04.116 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary2 2024-09-12T13:36:04.129 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary10 up+unknown '' '' 2024-09-12T13:36:04.129 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:36:04.129 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:36:04.129 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:36:04.129 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:36:04.129 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:36:04.130 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:36:04.130 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:36:04.130 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:36:04.130 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary10 2024-09-12T13:36:04.191 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary2: 2024-09-12T13:36:04.191 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 6c7d67ca-cef8-4a96-a25c-8d89a4671b00 2024-09-12T13:36:04.191 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:36:04.191 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:36:04.191 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:36:04.191 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:35:47' 2024-09-12T13:36:04.191 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:36:04.193 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:36:04.193 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:36:04.193 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 2 2024-09-12T13:36:04.206 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary10: 2024-09-12T13:36:04.206 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 7450998e-7967-4447-af66-572bd4fdded4 2024-09-12T13:36:04.206 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:36:04.206 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:36:04.206 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:36:04.206 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:35:47' 2024-09-12T13:36:04.206 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:36:04.209 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:36:04.209 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:36:04.209 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 2 2024-09-12T13:36:04.643 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary9 up+unknown '' '' 2024-09-12T13:36:04.643 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:36:04.643 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:36:04.643 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:36:04.643 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:36:04.644 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:36:04.644 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:36:04.644 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:36:04.644 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:36:04.644 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary9 2024-09-12T13:36:04.727 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary9: 2024-09-12T13:36:04.727 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 1a8d4f4e-16a8-41c0-8b42-954a67b2e30d 2024-09-12T13:36:04.727 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:36:04.727 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:36:04.727 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:36:04.727 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:35:47' 2024-09-12T13:36:04.727 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:36:04.727 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:36:04.727 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:36:04.727 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 4 2024-09-12T13:36:05.767 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary3 up+unknown '' '' 2024-09-12T13:36:05.767 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:36:05.767 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:36:05.768 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:36:05.768 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:36:05.768 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:36:05.768 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:36:05.768 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:36:05.769 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:36:05.769 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary3 2024-09-12T13:36:05.769 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary5 up+unknown '' '' 2024-09-12T13:36:05.769 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:36:05.769 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:36:05.769 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:36:05.769 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:36:05.769 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:36:05.769 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:36:05.769 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:36:05.770 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:36:05.770 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary5 2024-09-12T13:36:05.771 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary1 up+unknown '' '' 2024-09-12T13:36:05.771 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:36:05.771 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:36:05.771 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:36:05.771 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:36:05.772 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:36:05.772 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:36:05.772 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:36:05.772 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:36:05.772 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary1 2024-09-12T13:36:05.831 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary6 up+unknown '' '' 2024-09-12T13:36:05.831 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:36:05.831 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:36:05.831 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:36:05.831 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:36:05.831 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:36:05.832 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:36:05.832 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:36:05.832 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:36:05.832 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary6 2024-09-12T13:36:05.856 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary5: 2024-09-12T13:36:05.856 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 95f21a09-f11d-496c-bf07-0203910a05a6 2024-09-12T13:36:05.856 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:36:05.856 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:36:05.856 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:36:05.856 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:35:47' 2024-09-12T13:36:05.856 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:36:05.857 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary1: 2024-09-12T13:36:05.857 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 4ee25370-f977-4ecf-acda-dc9a3ae2ee9d 2024-09-12T13:36:05.857 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:36:05.857 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:36:05.857 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:36:05.857 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:35:47' 2024-09-12T13:36:05.857 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:36:05.857 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary3: 2024-09-12T13:36:05.857 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 9cc4db57-5517-41a1-8d14-519d4043d1a5 2024-09-12T13:36:05.857 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:36:05.857 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:36:05.857 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:36:05.858 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:35:47' 2024-09-12T13:36:05.858 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:36:05.859 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:36:05.860 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:36:05.860 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:36:05.860 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 8 2024-09-12T13:36:05.860 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:36:05.860 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 8 2024-09-12T13:36:05.860 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:36:05.860 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:36:05.860 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 8 2024-09-12T13:36:05.913 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary6: 2024-09-12T13:36:05.913 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: dcf3f6ae-ecc8-4d77-9308-bede00122fcd 2024-09-12T13:36:05.913 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:36:05.913 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:36:05.913 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:36:05.913 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:35:47' 2024-09-12T13:36:05.914 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:36:05.916 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:36:05.916 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:36:05.916 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 8 2024-09-12T13:36:06.196 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary2 up+unknown '' '' 2024-09-12T13:36:06.196 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:36:06.196 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:36:06.196 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:36:06.196 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:36:06.196 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:36:06.196 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:36:06.197 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:36:06.197 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:36:06.197 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary2 2024-09-12T13:36:06.211 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary10 up+unknown '' '' 2024-09-12T13:36:06.211 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:36:06.211 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:36:06.211 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:36:06.212 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:36:06.212 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:36:06.212 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:36:06.212 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:36:06.212 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:36:06.212 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary10 2024-09-12T13:36:06.273 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary2: 2024-09-12T13:36:06.273 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 6c7d67ca-cef8-4a96-a25c-8d89a4671b00 2024-09-12T13:36:06.273 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:36:06.273 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:36:06.273 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:36:06.273 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:35:47' 2024-09-12T13:36:06.273 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:36:06.276 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:36:06.276 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:36:06.276 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 4 2024-09-12T13:36:06.285 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary10: 2024-09-12T13:36:06.285 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 7450998e-7967-4447-af66-572bd4fdded4 2024-09-12T13:36:06.285 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:36:06.285 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:36:06.285 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:36:06.285 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:35:47' 2024-09-12T13:36:06.286 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:36:06.288 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:36:06.288 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:36:06.288 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 4 2024-09-12T13:36:06.833 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary4 up+unknown '' '' 2024-09-12T13:36:06.833 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:36:06.833 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:36:06.834 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:36:06.834 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:36:06.834 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:36:06.834 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:36:06.834 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:36:06.834 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:36:06.834 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary4 2024-09-12T13:36:06.912 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary4: 2024-09-12T13:36:06.912 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 0203765c-0327-4045-b74a-e9bc90c8f4a0 2024-09-12T13:36:06.912 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:36:06.912 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:36:06.912 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:36:06.912 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:35:47' 2024-09-12T13:36:06.912 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:36:06.914 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:36:06.914 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:36:06.914 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 8 2024-09-12T13:36:07.788 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary7 up+unknown '' '' 2024-09-12T13:36:07.788 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:36:07.788 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:36:07.789 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:36:07.789 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:36:07.789 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:36:07.789 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:36:07.789 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:36:07.789 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:36:07.789 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary7 2024-09-12T13:36:07.873 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary7: 2024-09-12T13:36:07.873 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: da6437a6-d0c6-4534-a0e0-60fb07f54f52 2024-09-12T13:36:07.873 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:36:07.873 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:36:07.873 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:36:07.873 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:35:47' 2024-09-12T13:36:07.873 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:36:07.875 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:36:07.876 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:36:07.876 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 8 2024-09-12T13:36:08.727 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary9 up+unknown '' '' 2024-09-12T13:36:08.727 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:36:08.727 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:36:08.727 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:36:08.727 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:36:08.728 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:36:08.728 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:36:08.728 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:36:08.728 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:36:08.728 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary9 2024-09-12T13:36:08.813 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary9: 2024-09-12T13:36:08.813 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 1a8d4f4e-16a8-41c0-8b42-954a67b2e30d 2024-09-12T13:36:08.813 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:36:08.814 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:36:08.814 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:36:08.814 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:35:47' 2024-09-12T13:36:08.814 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:36:08.816 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:36:08.816 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:36:08.816 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 8 2024-09-12T13:36:10.278 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary2 up+unknown '' '' 2024-09-12T13:36:10.278 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:36:10.279 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:36:10.279 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:36:10.279 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:36:10.279 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:36:10.279 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:36:10.279 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:36:10.279 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:36:10.279 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary2 2024-09-12T13:36:10.290 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary10 up+unknown '' '' 2024-09-12T13:36:10.290 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:36:10.290 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:36:10.290 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:36:10.291 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:36:10.291 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:36:10.291 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:36:10.291 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:36:10.291 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:36:10.291 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary10 2024-09-12T13:36:10.357 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary2: 2024-09-12T13:36:10.357 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 6c7d67ca-cef8-4a96-a25c-8d89a4671b00 2024-09-12T13:36:10.357 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:36:10.357 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:36:10.358 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:36:10.358 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:35:47' 2024-09-12T13:36:10.358 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:36:10.361 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:36:10.361 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:36:10.361 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 8 2024-09-12T13:36:10.369 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary10: 2024-09-12T13:36:10.370 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 7450998e-7967-4447-af66-572bd4fdded4 2024-09-12T13:36:10.370 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:36:10.370 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:36:10.370 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:36:10.370 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:35:47' 2024-09-12T13:36:10.370 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:36:10.372 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:36:10.372 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:36:10.372 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 8 2024-09-12T13:36:12.039 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary8 up+unknown '' '' 2024-09-12T13:36:12.039 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:36:12.039 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:36:12.039 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:36:12.039 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:36:12.039 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:36:12.039 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:36:12.039 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:36:12.040 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:36:12.040 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary8 2024-09-12T13:36:12.120 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary8: 2024-09-12T13:36:12.121 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: b415829c-3d1c-41f6-a1aa-e76e40e110d1 2024-09-12T13:36:12.121 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:36:12.121 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:36:12.121 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:36:12.121 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:35:47' 2024-09-12T13:36:12.121 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:36:12.123 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:36:12.123 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:36:12.123 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 8 2024-09-12T13:36:13.862 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary5 up+unknown '' '' 2024-09-12T13:36:13.863 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary3 up+unknown '' '' 2024-09-12T13:36:13.863 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary1 up+unknown '' '' 2024-09-12T13:36:13.863 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:36:13.863 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:36:13.863 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:36:13.863 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:36:13.863 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:36:13.863 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:36:13.863 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:36:13.863 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:36:13.864 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:36:13.864 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:36:13.864 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:36:13.864 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:36:13.864 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:36:13.864 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:36:13.864 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:36:13.864 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:36:13.864 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:36:13.864 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:36:13.864 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:36:13.864 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:36:13.864 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:36:13.865 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:36:13.865 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:36:13.865 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary5 2024-09-12T13:36:13.865 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary3 2024-09-12T13:36:13.865 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:36:13.865 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary1 2024-09-12T13:36:13.918 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary6 up+unknown '' '' 2024-09-12T13:36:13.919 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:36:13.919 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:36:13.919 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:36:13.919 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:36:13.919 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:36:13.919 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:36:13.919 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:36:13.919 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:36:13.920 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary6 2024-09-12T13:36:13.948 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary3: 2024-09-12T13:36:13.948 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 9cc4db57-5517-41a1-8d14-519d4043d1a5 2024-09-12T13:36:13.948 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:36:13.948 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:36:13.949 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:36:13.949 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:35:47' 2024-09-12T13:36:13.949 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:36:13.949 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary5: 2024-09-12T13:36:13.949 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 95f21a09-f11d-496c-bf07-0203910a05a6 2024-09-12T13:36:13.949 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:36:13.949 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:36:13.949 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:36:13.950 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:35:47' 2024-09-12T13:36:13.950 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:36:13.950 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary1: 2024-09-12T13:36:13.950 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 4ee25370-f977-4ecf-acda-dc9a3ae2ee9d 2024-09-12T13:36:13.950 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:36:13.950 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:36:13.950 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:36:13.950 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:35:47' 2024-09-12T13:36:13.951 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:36:13.951 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:36:13.952 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:36:13.952 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 8 2024-09-12T13:36:13.952 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:36:13.952 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:36:13.952 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 8 2024-09-12T13:36:13.953 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:36:13.953 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:36:13.953 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 8 2024-09-12T13:36:14.000 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary6: 2024-09-12T13:36:14.000 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: dcf3f6ae-ecc8-4d77-9308-bede00122fcd 2024-09-12T13:36:14.001 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:36:14.001 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:36:14.001 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:36:14.001 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:35:47' 2024-09-12T13:36:14.001 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:36:14.003 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:36:14.003 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:36:14.003 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 8 2024-09-12T13:36:14.917 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary4 up+unknown '' '' 2024-09-12T13:36:14.917 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:36:14.917 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:36:14.917 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:36:14.917 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:36:14.917 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:36:14.917 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:36:14.917 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:36:14.918 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:36:14.918 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary4 2024-09-12T13:36:14.999 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary4: 2024-09-12T13:36:14.999 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 0203765c-0327-4045-b74a-e9bc90c8f4a0 2024-09-12T13:36:14.999 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:36:14.999 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:36:14.999 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:36:14.999 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:35:47' 2024-09-12T13:36:14.999 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:36:15.002 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:36:15.002 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:36:15.002 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 8 2024-09-12T13:36:15.878 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary7 up+unknown '' '' 2024-09-12T13:36:15.879 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:36:15.879 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:36:15.879 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:36:15.879 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:36:15.879 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:36:15.879 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:36:15.879 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:36:15.879 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:36:15.880 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary7 2024-09-12T13:36:15.961 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary7: 2024-09-12T13:36:15.961 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: da6437a6-d0c6-4534-a0e0-60fb07f54f52 2024-09-12T13:36:15.961 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:36:15.961 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:36:15.961 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:36:15.961 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:35:47' 2024-09-12T13:36:15.961 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:36:15.964 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:36:15.964 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:36:15.964 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 8 2024-09-12T13:36:16.819 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary9 up+unknown '' '' 2024-09-12T13:36:16.819 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:36:16.819 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:36:16.820 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:36:16.820 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:36:16.820 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:36:16.820 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:36:16.820 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:36:16.820 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:36:16.820 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary9 2024-09-12T13:36:16.902 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary9: 2024-09-12T13:36:16.903 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 1a8d4f4e-16a8-41c0-8b42-954a67b2e30d 2024-09-12T13:36:16.903 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:36:16.903 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:36:16.903 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:36:16.903 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:35:47' 2024-09-12T13:36:16.903 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:36:16.906 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:36:16.906 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:36:16.906 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 8 2024-09-12T13:36:18.363 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary2 up+unknown '' '' 2024-09-12T13:36:18.363 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:36:18.363 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:36:18.363 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:36:18.363 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:36:18.363 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:36:18.364 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:36:18.364 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:36:18.364 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:36:18.364 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary2 2024-09-12T13:36:18.375 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary10 up+unknown '' '' 2024-09-12T13:36:18.375 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:36:18.375 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:36:18.375 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:36:18.375 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:36:18.375 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:36:18.376 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:36:18.376 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:36:18.376 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:36:18.376 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary10 2024-09-12T13:36:18.449 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary2: 2024-09-12T13:36:18.449 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 6c7d67ca-cef8-4a96-a25c-8d89a4671b00 2024-09-12T13:36:18.449 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+unknown 2024-09-12T13:36:18.449 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: remote image is not primary 2024-09-12T13:36:18.449 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:36:18.449 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:36:17' 2024-09-12T13:36:18.449 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:36:18.451 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+unknown 2024-09-12T13:36:18.452 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:36:18.454 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: remote image is not primary 2024-09-12T13:36:18.455 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:36:18.455 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary10: 2024-09-12T13:36:18.455 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 7450998e-7967-4447-af66-572bd4fdded4 2024-09-12T13:36:18.455 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+unknown 2024-09-12T13:36:18.456 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: remote image is not primary 2024-09-12T13:36:18.456 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:36:18.456 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:36:17' 2024-09-12T13:36:18.456 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:36:18.456 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:36:18.456 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:36:18.457 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:36:18.458 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+unknown 2024-09-12T13:36:18.458 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:36:18.458 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:36:18.461 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:36:18.461 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: remote image is not primary 2024-09-12T13:36:18.461 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:36:18.461 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:36:18.462 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:36:18.462 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:36:18.462 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:36:18.464 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:36:18.465 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:36:18.466 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:36:17' 2024-09-12T13:36:18.466 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary2 up+unknown '2024-09-12 13:36:17' 2024-09-12T13:36:18.466 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:36:18.466 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:36:18.466 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:36:18.466 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:36:18.466 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:36:17' 2024-09-12T13:36:18.466 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:36:18.466 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:36:18.467 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-09-12T13:36:18.468 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:36:18.468 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:36:18.469 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:36:18.472 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:36:17' 2024-09-12T13:36:18.472 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary10 up+unknown '2024-09-12 13:36:17' 2024-09-12T13:36:18.472 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:36:18.473 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:36:18.473 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:36:18.473 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:36:18.473 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:36:17' 2024-09-12T13:36:18.473 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:36:18.473 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:36:18.473 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-09-12T13:36:18.595 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041934232mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary14ee25370-f977-4ecf-acda-dc9a3ae2ee9dup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:36:17image-primary107450998e-7967-4447-af66-572bd4fdded4up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:36:17image-primary26c7d67ca-cef8-4a96-a25c-8d89a4671b00up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:36:17image-primary39cc4db57-5517-41a1-8d14-519d4043d1a5up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:36:17image-primary40203765c-0327-4045-b74a-e9bc90c8f4a0up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:36:17image-primary595f21a09-f11d-496c-bf07-0203910a05a6up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:36:17image-primary6dcf3f6ae-ecc8-4d77-9308-bede00122fcdup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:36:17image-primary7da6437a6-d0c6-4534-a0e0-60fb07f54f52up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:36:17image-primary8b415829c-3d1c-41f6-a1aa-e76e40e110d1up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:36:17image-primary91a8d4f4e-16a8-41c0-8b42-954a67b2e30dup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:36:17' 2024-09-12T13:36:18.595 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:36:18.595 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary2'\'']/last_update' 2024-09-12T13:36:18.605 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041934232mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary14ee25370-f977-4ecf-acda-dc9a3ae2ee9dup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:36:17image-primary107450998e-7967-4447-af66-572bd4fdded4up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:36:17image-primary26c7d67ca-cef8-4a96-a25c-8d89a4671b00up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:36:17image-primary39cc4db57-5517-41a1-8d14-519d4043d1a5up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:36:17image-primary40203765c-0327-4045-b74a-e9bc90c8f4a0up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:36:17image-primary595f21a09-f11d-496c-bf07-0203910a05a6up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:36:17image-primary6dcf3f6ae-ecc8-4d77-9308-bede00122fcdup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:36:17image-primary7da6437a6-d0c6-4534-a0e0-60fb07f54f52up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:36:17image-primary8b415829c-3d1c-41f6-a1aa-e76e40e110d1up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:36:17image-primary91a8d4f4e-16a8-41c0-8b42-954a67b2e30dup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:36:17' 2024-09-12T13:36:18.605 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:36:18.609 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary10'\'']/last_update' 2024-09-12T13:36:18.639 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:36:17' 2024-09-12T13:36:18.639 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:36:17' 2024-09-12T13:36:18.640 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary2'\'']/state' 2024-09-12T13:36:18.640 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary10'\'']/state' 2024-09-12T13:36:18.643 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+unknown 2024-09-12T13:36:18.643 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+unknown 2024-09-12T13:36:18.644 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:36:18.644 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:36:18.644 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+unknown 2024-09-12T13:36:18.644 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+unknown 2024-09-12T13:36:18.646 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:36:18.646 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:36:18.647 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:36:18.647 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:36:18.648 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:36:18.648 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster2 mirror image-primary2 up+unknown 2024-09-12T13:36:18.648 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:36:18.648 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:36:18.648 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:36:18.648 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:36:18.648 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:36:18.648 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:36:18.648 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:36:18.648 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster2 mirror image-primary10 up+unknown 2024-09-12T13:36:18.648 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:36:18.648 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:36:18.648 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:36:18.648 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:36:18.649 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:36:18.649 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:36:18.649 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:36:18.649 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:36:18.649 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:36:18.649 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:36:19.649 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster2 mirror image-primary2 up+unknown '' '' 2024-09-12T13:36:19.650 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster2 mirror image-primary10 up+unknown '' '' 2024-09-12T13:36:19.650 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:36:19.650 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:36:19.650 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:36:19.650 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:36:19.650 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:36:19.650 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:36:19.650 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:36:19.650 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:36:19.650 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:36:19.650 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:36:19.651 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:36:19.651 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:36:19.651 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:36:19.651 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:36:19.651 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:36:19.652 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster2 mirror image status mirror/image-primary2 2024-09-12T13:36:19.652 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:36:19.652 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster2 mirror image status mirror/image-primary10 2024-09-12T13:36:19.721 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary10: 2024-09-12T13:36:19.721 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 7450998e-7967-4447-af66-572bd4fdded4 2024-09-12T13:36:19.721 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+unknown 2024-09-12T13:36:19.721 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: remote image is not primary 2024-09-12T13:36:19.722 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:36:19.722 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:36:17' 2024-09-12T13:36:19.722 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:36:19.722 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary2: 2024-09-12T13:36:19.722 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 6c7d67ca-cef8-4a96-a25c-8d89a4671b00 2024-09-12T13:36:19.723 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+unknown 2024-09-12T13:36:19.723 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: remote image is not primary 2024-09-12T13:36:19.723 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:36:19.723 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:36:17' 2024-09-12T13:36:19.723 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:36:19.723 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+unknown 2024-09-12T13:36:19.724 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:36:19.724 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+unknown 2024-09-12T13:36:19.724 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:36:19.726 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: remote image is not primary 2024-09-12T13:36:19.727 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:36:19.727 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: remote image is not primary 2024-09-12T13:36:19.728 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:36:19.728 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:36:19.728 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:36:19.728 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:36:19.728 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:36:19.729 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:36:19.730 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:36:19.730 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:36:19.730 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:36:19.732 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:36:19.733 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:36:19.733 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:36:19.733 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:36:19.733 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:36:19.734 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:36:19.738 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:36:17' 2024-09-12T13:36:19.739 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster2 mirror image-primary10 up+unknown '2024-09-12 13:36:17' 2024-09-12T13:36:19.739 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:36:19.739 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:36:19.739 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:36:19.739 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:36:19.739 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:36:17' 2024-09-12T13:36:19.739 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:36:19.739 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:36:17' 2024-09-12T13:36:19.739 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster2 mirror image-primary2 up+unknown '2024-09-12 13:36:17' 2024-09-12T13:36:19.739 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:36:19.739 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:36:19.739 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:36:19.740 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:36:19.740 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:36:17' 2024-09-12T13:36:19.740 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:36:19.740 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:36:19.740 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2024-09-12T13:36:19.740 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:36:19.740 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2024-09-12T13:36:19.863 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041924231mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary14ee25370-f977-4ecf-acda-dc9a3ae2ee9dup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:36:17image-primary107450998e-7967-4447-af66-572bd4fdded4up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:36:17image-primary26c7d67ca-cef8-4a96-a25c-8d89a4671b00up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:36:17image-primary39cc4db57-5517-41a1-8d14-519d4043d1a5up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:36:17image-primary40203765c-0327-4045-b74a-e9bc90c8f4a0up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:36:17image-primary595f21a09-f11d-496c-bf07-0203910a05a6up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:36:17image-primary6dcf3f6ae-ecc8-4d77-9308-bede00122fcdup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:36:17image-primary7da6437a6-d0c6-4534-a0e0-60fb07f54f52up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:36:17image-primary8b415829c-3d1c-41f6-a1aa-e76e40e110d1up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:36:17image-primary91a8d4f4e-16a8-41c0-8b42-954a67b2e30dup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:36:17' 2024-09-12T13:36:19.863 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:36:19.864 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary10'\'']/last_update' 2024-09-12T13:36:19.864 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041924231mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary14ee25370-f977-4ecf-acda-dc9a3ae2ee9dup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:36:17image-primary107450998e-7967-4447-af66-572bd4fdded4up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:36:17image-primary26c7d67ca-cef8-4a96-a25c-8d89a4671b00up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:36:17image-primary39cc4db57-5517-41a1-8d14-519d4043d1a5up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:36:17image-primary40203765c-0327-4045-b74a-e9bc90c8f4a0up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:36:17image-primary595f21a09-f11d-496c-bf07-0203910a05a6up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:36:17image-primary6dcf3f6ae-ecc8-4d77-9308-bede00122fcdup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:36:17image-primary7da6437a6-d0c6-4534-a0e0-60fb07f54f52up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:36:17image-primary8b415829c-3d1c-41f6-a1aa-e76e40e110d1up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:36:17image-primary91a8d4f4e-16a8-41c0-8b42-954a67b2e30dup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:36:17' 2024-09-12T13:36:19.864 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:36:19.865 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary2'\'']/last_update' 2024-09-12T13:36:19.868 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:36:17' 2024-09-12T13:36:19.868 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary10'\'']/state' 2024-09-12T13:36:19.869 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:36:17' 2024-09-12T13:36:19.869 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary2'\'']/state' 2024-09-12T13:36:19.872 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+unknown 2024-09-12T13:36:19.873 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:36:19.873 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+unknown 2024-09-12T13:36:19.873 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+unknown 2024-09-12T13:36:19.874 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:36:19.874 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+unknown 2024-09-12T13:36:19.875 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:36:19.876 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:36:19.876 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:36:19.876 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ promote_image cluster2 mirror image-primary10 2024-09-12T13:36:19.876 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:36:19.876 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:36:19.876 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:36:19.877 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local force= 2024-09-12T13:36:19.877 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 mirror image promote mirror/image-primary10 2024-09-12T13:36:19.877 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:36:19.878 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:36:19.878 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:36:19.878 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ promote_image cluster2 mirror image-primary2 2024-09-12T13:36:19.878 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:36:19.878 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:36:19.878 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:36:19.878 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local force= 2024-09-12T13:36:19.878 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 mirror image promote mirror/image-primary2 2024-09-12T13:36:20.127 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary8 up+unknown '' '' 2024-09-12T13:36:20.127 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:36:20.127 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:36:20.127 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:36:20.127 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:36:20.127 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:36:20.128 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:36:20.128 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:36:20.128 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:36:20.128 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary8 2024-09-12T13:36:20.201 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary8: 2024-09-12T13:36:20.202 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: b415829c-3d1c-41f6-a1aa-e76e40e110d1 2024-09-12T13:36:20.202 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+unknown 2024-09-12T13:36:20.202 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: remote image is not primary 2024-09-12T13:36:20.202 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:36:20.202 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:36:17' 2024-09-12T13:36:20.202 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:36:20.204 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+unknown 2024-09-12T13:36:20.204 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:36:20.207 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: remote image is not primary 2024-09-12T13:36:20.207 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:36:20.208 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:36:20.208 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:36:20.210 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:36:20.210 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:36:20.213 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:36:20.213 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:36:20.214 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:36:20.217 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:36:17' 2024-09-12T13:36:20.217 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary8 up+unknown '2024-09-12 13:36:17' 2024-09-12T13:36:20.218 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:36:20.218 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:36:20.218 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:36:20.218 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:36:20.218 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:36:17' 2024-09-12T13:36:20.218 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:36:20.218 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:36:20.218 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-09-12T13:36:20.276 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Image promoted to primary 2024-09-12T13:36:20.279 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Image promoted to primary 2024-09-12T13:36:20.288 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \n\b\d ]] 2024-09-12T13:36:20.288 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \k\r\b\d ]] 2024-09-12T13:36:20.289 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo rbd --cluster cluster2 device map -t krbd mirror/image-primary2 2024-09-12T13:36:20.291 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \n\b\d ]] 2024-09-12T13:36:20.291 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \k\r\b\d ]] 2024-09-12T13:36:20.292 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo rbd --cluster cluster2 device map -t krbd mirror/image-primary10 2024-09-12T13:36:20.335 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041934232mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary14ee25370-f977-4ecf-acda-dc9a3ae2ee9dup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:36:17image-primary107450998e-7967-4447-af66-572bd4fdded4up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:36:17image-primary26c7d67ca-cef8-4a96-a25c-8d89a4671b00up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:36:17image-primary39cc4db57-5517-41a1-8d14-519d4043d1a5up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:36:17image-primary40203765c-0327-4045-b74a-e9bc90c8f4a0up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:36:17image-primary595f21a09-f11d-496c-bf07-0203910a05a6up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:36:17image-primary6dcf3f6ae-ecc8-4d77-9308-bede00122fcdup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:36:17image-primary7da6437a6-d0c6-4534-a0e0-60fb07f54f52up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:36:17image-primary8b415829c-3d1c-41f6-a1aa-e76e40e110d1up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:36:17image-primary91a8d4f4e-16a8-41c0-8b42-954a67b2e30dup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:36:17' 2024-09-12T13:36:20.335 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:36:20.336 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary8'\'']/last_update' 2024-09-12T13:36:20.340 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:36:17' 2024-09-12T13:36:20.341 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary8'\'']/state' 2024-09-12T13:36:20.345 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+unknown 2024-09-12T13:36:20.345 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:36:20.346 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+unknown 2024-09-12T13:36:20.349 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:36:20.350 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:36:20.350 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:36:20.350 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster2 mirror image-primary8 up+unknown 2024-09-12T13:36:20.350 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:36:20.350 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:36:20.350 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:36:20.350 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:36:20.351 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:36:20.351 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:36:20.351 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:36:20.351 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:36:20.430 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ dev=/dev/rbd1 2024-09-12T13:36:20.431 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ dev=/dev/rbd0 2024-09-12T13:36:20.432 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo md5sum /dev/rbd1 2024-09-12T13:36:20.432 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo md5sum /dev/rbd0 2024-09-12T13:36:20.432 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ awk '{print $1}' 2024-09-12T13:36:20.432 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ awk '{print $1}' 2024-09-12T13:36:21.352 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster2 mirror image-primary8 up+unknown '' '' 2024-09-12T13:36:21.352 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:36:21.352 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:36:21.352 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:36:21.352 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:36:21.353 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:36:21.353 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:36:21.353 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:36:21.353 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:36:21.353 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster2 mirror image status mirror/image-primary8 2024-09-12T13:36:21.423 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary8: 2024-09-12T13:36:21.423 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: b415829c-3d1c-41f6-a1aa-e76e40e110d1 2024-09-12T13:36:21.423 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+unknown 2024-09-12T13:36:21.423 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: remote image is not primary 2024-09-12T13:36:21.423 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:36:21.423 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:36:17' 2024-09-12T13:36:21.423 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:36:21.425 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+unknown 2024-09-12T13:36:21.426 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:36:21.428 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: remote image is not primary 2024-09-12T13:36:21.428 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:36:21.429 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:36:21.429 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:36:21.431 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:36:21.431 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:36:21.434 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:36:21.434 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:36:21.435 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:36:21.439 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:36:17' 2024-09-12T13:36:21.439 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster2 mirror image-primary8 up+unknown '2024-09-12 13:36:17' 2024-09-12T13:36:21.439 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:36:21.439 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:36:21.439 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:36:21.439 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:36:21.439 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:36:17' 2024-09-12T13:36:21.439 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:36:21.439 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:36:21.440 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2024-09-12T13:36:21.554 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041924231mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary14ee25370-f977-4ecf-acda-dc9a3ae2ee9dup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:36:17image-primary107450998e-7967-4447-af66-572bd4fdded4up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:36:17image-primary26c7d67ca-cef8-4a96-a25c-8d89a4671b00up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:36:17image-primary39cc4db57-5517-41a1-8d14-519d4043d1a5up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:36:17image-primary40203765c-0327-4045-b74a-e9bc90c8f4a0up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:36:17image-primary595f21a09-f11d-496c-bf07-0203910a05a6up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:36:17image-primary6dcf3f6ae-ecc8-4d77-9308-bede00122fcdup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:36:17image-primary7da6437a6-d0c6-4534-a0e0-60fb07f54f52up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:36:17image-primary8b415829c-3d1c-41f6-a1aa-e76e40e110d1up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:36:17image-primary91a8d4f4e-16a8-41c0-8b42-954a67b2e30dup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:36:17' 2024-09-12T13:36:21.554 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:36:21.555 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary8'\'']/last_update' 2024-09-12T13:36:21.559 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:36:17' 2024-09-12T13:36:21.560 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary8'\'']/state' 2024-09-12T13:36:21.564 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+unknown 2024-09-12T13:36:21.565 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:36:21.565 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+unknown 2024-09-12T13:36:21.566 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:36:21.567 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:36:21.567 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:36:21.567 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ promote_image cluster2 mirror image-primary8 2024-09-12T13:36:21.568 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:36:21.568 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:36:21.568 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:36:21.568 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local force= 2024-09-12T13:36:21.568 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 mirror image promote mirror/image-primary8 2024-09-12T13:36:21.953 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary3 up+unknown '' '' 2024-09-12T13:36:21.954 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:36:21.954 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:36:21.954 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:36:21.954 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:36:21.954 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:36:21.954 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:36:21.954 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:36:21.955 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:36:21.955 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary3 2024-09-12T13:36:21.955 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary5 up+unknown '' '' 2024-09-12T13:36:21.955 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:36:21.955 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:36:21.955 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:36:21.955 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:36:21.955 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:36:21.955 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:36:21.955 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:36:21.956 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:36:21.956 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary5 2024-09-12T13:36:21.956 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary1 up+unknown '' '' 2024-09-12T13:36:21.956 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:36:21.957 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:36:21.957 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:36:21.957 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:36:21.957 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:36:21.957 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:36:21.957 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:36:21.957 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:36:21.957 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary1 2024-09-12T13:36:22.006 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary6 up+unknown '' '' 2024-09-12T13:36:22.006 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:36:22.006 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:36:22.006 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:36:22.006 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:36:22.006 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:36:22.006 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:36:22.006 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:36:22.007 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:36:22.007 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary6 2024-09-12T13:36:22.040 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary3: 2024-09-12T13:36:22.040 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 9cc4db57-5517-41a1-8d14-519d4043d1a5 2024-09-12T13:36:22.040 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+unknown 2024-09-12T13:36:22.040 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: remote image is not primary 2024-09-12T13:36:22.040 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:36:22.041 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:36:17' 2024-09-12T13:36:22.041 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:36:22.043 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+unknown 2024-09-12T13:36:22.043 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:36:22.048 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: remote image is not primary 2024-09-12T13:36:22.049 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:36:22.049 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:36:22.049 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:36:22.052 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:36:22.053 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:36:22.057 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:36:22.058 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:36:22.059 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:36:22.061 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary5: 2024-09-12T13:36:22.061 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 95f21a09-f11d-496c-bf07-0203910a05a6 2024-09-12T13:36:22.061 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+unknown 2024-09-12T13:36:22.061 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: remote image is not primary 2024-09-12T13:36:22.062 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:36:22.062 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:36:17' 2024-09-12T13:36:22.062 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:36:22.064 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary1: 2024-09-12T13:36:22.064 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 4ee25370-f977-4ecf-acda-dc9a3ae2ee9d 2024-09-12T13:36:22.064 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+unknown 2024-09-12T13:36:22.064 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: remote image is not primary 2024-09-12T13:36:22.064 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:36:22.064 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:36:17' 2024-09-12T13:36:22.064 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:36:22.065 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:36:17' 2024-09-12T13:36:22.065 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary3 up+unknown '2024-09-12 13:36:17' 2024-09-12T13:36:22.065 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:36:22.065 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:36:22.065 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:36:22.065 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:36:22.066 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:36:17' 2024-09-12T13:36:22.066 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:36:22.066 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:36:22.066 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-09-12T13:36:22.066 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+unknown 2024-09-12T13:36:22.066 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:36:22.067 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+unknown 2024-09-12T13:36:22.067 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:36:22.069 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: remote image is not primary 2024-09-12T13:36:22.070 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:36:22.070 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: remote image is not primary 2024-09-12T13:36:22.071 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:36:22.071 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:36:22.071 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:36:22.072 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:36:22.073 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:36:22.074 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:36:22.074 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:36:22.076 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:36:22.076 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:36:22.077 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:36:22.077 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:36:22.078 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:36:22.081 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:36:22.083 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:36:22.083 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:36:17' 2024-09-12T13:36:22.083 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary1 up+unknown '2024-09-12 13:36:17' 2024-09-12T13:36:22.083 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:36:22.083 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:36:22.083 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:36:22.083 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:36:22.083 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:36:17' 2024-09-12T13:36:22.083 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:36:22.083 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:36:22.084 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:36:22.084 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-09-12T13:36:22.087 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:36:17' 2024-09-12T13:36:22.087 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary5 up+unknown '2024-09-12 13:36:17' 2024-09-12T13:36:22.087 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:36:22.087 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:36:22.087 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:36:22.087 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:36:22.087 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:36:17' 2024-09-12T13:36:22.087 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:36:22.088 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:36:22.088 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-09-12T13:36:22.098 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary6: 2024-09-12T13:36:22.098 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: dcf3f6ae-ecc8-4d77-9308-bede00122fcd 2024-09-12T13:36:22.098 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+unknown 2024-09-12T13:36:22.098 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: remote image is not primary 2024-09-12T13:36:22.098 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:36:22.098 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:36:17' 2024-09-12T13:36:22.098 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:36:22.102 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+unknown 2024-09-12T13:36:22.102 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:36:22.106 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: remote image is not primary 2024-09-12T13:36:22.107 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:36:22.108 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:36:22.109 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:36:22.112 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:36:22.113 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:36:22.118 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:36:22.118 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:36:22.119 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:36:22.127 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:36:17' 2024-09-12T13:36:22.127 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary6 up+unknown '2024-09-12 13:36:17' 2024-09-12T13:36:22.127 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:36:22.127 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:36:22.128 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:36:22.128 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:36:22.128 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:36:17' 2024-09-12T13:36:22.128 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:36:22.128 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:36:22.129 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-09-12T13:36:22.251 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041934232mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary14ee25370-f977-4ecf-acda-dc9a3ae2ee9dup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:36:17image-primary107450998e-7967-4447-af66-572bd4fdded4up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:36:17image-primary26c7d67ca-cef8-4a96-a25c-8d89a4671b00up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:36:17image-primary39cc4db57-5517-41a1-8d14-519d4043d1a5up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:36:17image-primary40203765c-0327-4045-b74a-e9bc90c8f4a0up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:36:17image-primary595f21a09-f11d-496c-bf07-0203910a05a6up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:36:17image-primary6dcf3f6ae-ecc8-4d77-9308-bede00122fcdup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:36:17image-primary7da6437a6-d0c6-4534-a0e0-60fb07f54f52up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:36:17image-primary8b415829c-3d1c-41f6-a1aa-e76e40e110d1up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:36:17image-primary91a8d4f4e-16a8-41c0-8b42-954a67b2e30dup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:36:17' 2024-09-12T13:36:22.252 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:36:22.252 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary3'\'']/last_update' 2024-09-12T13:36:22.260 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:36:17' 2024-09-12T13:36:22.262 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary3'\'']/state' 2024-09-12T13:36:22.264 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041934232mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary14ee25370-f977-4ecf-acda-dc9a3ae2ee9dup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:36:17image-primary107450998e-7967-4447-af66-572bd4fdded4up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:36:17image-primary26c7d67ca-cef8-4a96-a25c-8d89a4671b00up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:36:17image-primary39cc4db57-5517-41a1-8d14-519d4043d1a5up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:36:17image-primary40203765c-0327-4045-b74a-e9bc90c8f4a0up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:36:17image-primary595f21a09-f11d-496c-bf07-0203910a05a6up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:36:17image-primary6dcf3f6ae-ecc8-4d77-9308-bede00122fcdup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:36:17image-primary7da6437a6-d0c6-4534-a0e0-60fb07f54f52up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:36:17image-primary8b415829c-3d1c-41f6-a1aa-e76e40e110d1up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:36:17image-primary91a8d4f4e-16a8-41c0-8b42-954a67b2e30dup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:36:17' 2024-09-12T13:36:22.264 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:36:22.265 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary1'\'']/last_update' 2024-09-12T13:36:22.271 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+unknown 2024-09-12T13:36:22.272 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:36:22.272 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:36:17' 2024-09-12T13:36:22.273 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+unknown 2024-09-12T13:36:22.274 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary1'\'']/state' 2024-09-12T13:36:22.276 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:36:22.279 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041934232mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary14ee25370-f977-4ecf-acda-dc9a3ae2ee9dup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:36:17image-primary107450998e-7967-4447-af66-572bd4fdded4up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:36:17image-primary26c7d67ca-cef8-4a96-a25c-8d89a4671b00up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:36:17image-primary39cc4db57-5517-41a1-8d14-519d4043d1a5up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:36:17image-primary40203765c-0327-4045-b74a-e9bc90c8f4a0up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:36:17image-primary595f21a09-f11d-496c-bf07-0203910a05a6up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:36:17image-primary6dcf3f6ae-ecc8-4d77-9308-bede00122fcdup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:36:17image-primary7da6437a6-d0c6-4534-a0e0-60fb07f54f52up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:36:17image-primary8b415829c-3d1c-41f6-a1aa-e76e40e110d1up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:36:17image-primary91a8d4f4e-16a8-41c0-8b42-954a67b2e30dup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:36:17' 2024-09-12T13:36:22.279 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:36:22.279 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:36:22.280 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:36:22.280 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster2 mirror image-primary3 up+unknown 2024-09-12T13:36:22.280 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:36:22.280 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:36:22.280 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:36:22.280 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:36:22.280 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:36:22.280 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:36:22.280 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:36:22.280 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:36:22.281 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary5'\'']/last_update' 2024-09-12T13:36:22.281 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+unknown 2024-09-12T13:36:22.282 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:36:22.283 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+unknown 2024-09-12T13:36:22.288 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:36:22.290 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:36:17' 2024-09-12T13:36:22.290 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:36:22.291 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:36:22.291 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster2 mirror image-primary1 up+unknown 2024-09-12T13:36:22.291 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:36:22.291 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:36:22.291 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:36:22.291 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:36:22.292 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:36:22.292 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:36:22.292 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:36:22.292 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:36:22.292 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary5'\'']/state' 2024-09-12T13:36:22.294 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+unknown 2024-09-12T13:36:22.295 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:36:22.295 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+unknown 2024-09-12T13:36:22.298 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:36:22.300 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:36:22.300 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:36:22.300 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster2 mirror image-primary5 up+unknown 2024-09-12T13:36:22.300 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:36:22.300 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:36:22.300 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:36:22.301 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:36:22.301 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:36:22.301 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:36:22.301 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:36:22.301 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:36:22.302 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Image promoted to primary 2024-09-12T13:36:22.309 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041934232mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary14ee25370-f977-4ecf-acda-dc9a3ae2ee9dup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:36:17image-primary107450998e-7967-4447-af66-572bd4fdded4up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:36:17image-primary26c7d67ca-cef8-4a96-a25c-8d89a4671b00up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:36:17image-primary39cc4db57-5517-41a1-8d14-519d4043d1a5up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:36:17image-primary40203765c-0327-4045-b74a-e9bc90c8f4a0up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:36:17image-primary595f21a09-f11d-496c-bf07-0203910a05a6up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:36:17image-primary6dcf3f6ae-ecc8-4d77-9308-bede00122fcdup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:36:17image-primary7da6437a6-d0c6-4534-a0e0-60fb07f54f52up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:36:17image-primary8b415829c-3d1c-41f6-a1aa-e76e40e110d1up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:36:17image-primary91a8d4f4e-16a8-41c0-8b42-954a67b2e30dup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:36:17' 2024-09-12T13:36:22.309 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:36:22.310 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary6'\'']/last_update' 2024-09-12T13:36:22.312 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \n\b\d ]] 2024-09-12T13:36:22.312 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \k\r\b\d ]] 2024-09-12T13:36:22.313 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo rbd --cluster cluster2 device map -t krbd mirror/image-primary8 2024-09-12T13:36:22.313 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:36:17' 2024-09-12T13:36:22.314 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary6'\'']/state' 2024-09-12T13:36:22.318 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+unknown 2024-09-12T13:36:22.319 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:36:22.320 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+unknown 2024-09-12T13:36:22.322 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:36:22.323 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:36:22.323 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:36:22.323 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster2 mirror image-primary6 up+unknown 2024-09-12T13:36:22.323 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:36:22.323 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:36:22.323 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:36:22.323 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:36:22.323 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:36:22.323 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:36:22.324 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:36:22.324 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:36:22.437 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ dev=/dev/rbd2 2024-09-12T13:36:22.438 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo md5sum /dev/rbd2 2024-09-12T13:36:22.439 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ awk '{print $1}' 2024-09-12T13:36:23.005 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary4 up+unknown '' '' 2024-09-12T13:36:23.005 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:36:23.005 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:36:23.005 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:36:23.005 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:36:23.005 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:36:23.005 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:36:23.005 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:36:23.006 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:36:23.006 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary4 2024-09-12T13:36:23.083 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary4: 2024-09-12T13:36:23.083 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 0203765c-0327-4045-b74a-e9bc90c8f4a0 2024-09-12T13:36:23.084 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+unknown 2024-09-12T13:36:23.084 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: remote image is not primary 2024-09-12T13:36:23.084 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:36:23.084 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:36:17' 2024-09-12T13:36:23.084 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:36:23.086 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+unknown 2024-09-12T13:36:23.086 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:36:23.089 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: remote image is not primary 2024-09-12T13:36:23.089 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:36:23.090 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:36:23.090 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:36:23.093 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:36:23.093 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:36:23.096 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:36:23.096 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:36:23.097 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:36:23.101 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:36:17' 2024-09-12T13:36:23.101 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary4 up+unknown '2024-09-12 13:36:17' 2024-09-12T13:36:23.101 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:36:23.102 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:36:23.102 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:36:23.102 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:36:23.102 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:36:17' 2024-09-12T13:36:23.102 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:36:23.102 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:36:23.102 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-09-12T13:36:23.215 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041934232mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary14ee25370-f977-4ecf-acda-dc9a3ae2ee9dup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:36:17image-primary107450998e-7967-4447-af66-572bd4fdded4up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:36:17image-primary26c7d67ca-cef8-4a96-a25c-8d89a4671b00up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:36:17image-primary39cc4db57-5517-41a1-8d14-519d4043d1a5up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:36:17image-primary40203765c-0327-4045-b74a-e9bc90c8f4a0up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:36:17image-primary595f21a09-f11d-496c-bf07-0203910a05a6up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:36:17image-primary6dcf3f6ae-ecc8-4d77-9308-bede00122fcdup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:36:17image-primary7da6437a6-d0c6-4534-a0e0-60fb07f54f52up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:36:17image-primary8b415829c-3d1c-41f6-a1aa-e76e40e110d1up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:36:17image-primary91a8d4f4e-16a8-41c0-8b42-954a67b2e30dup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:36:17' 2024-09-12T13:36:23.215 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:36:23.216 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary4'\'']/last_update' 2024-09-12T13:36:23.219 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:36:17' 2024-09-12T13:36:23.221 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary4'\'']/state' 2024-09-12T13:36:23.224 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+unknown 2024-09-12T13:36:23.225 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:36:23.226 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+unknown 2024-09-12T13:36:23.228 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:36:23.229 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:36:23.229 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:36:23.229 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster2 mirror image-primary4 up+unknown 2024-09-12T13:36:23.229 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:36:23.229 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:36:23.229 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:36:23.230 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:36:23.230 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:36:23.230 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:36:23.230 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:36:23.230 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:36:23.284 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster2 mirror image-primary3 up+unknown '' '' 2024-09-12T13:36:23.285 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:36:23.285 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:36:23.285 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:36:23.285 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:36:23.285 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:36:23.285 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:36:23.285 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:36:23.285 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:36:23.285 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster2 mirror image status mirror/image-primary3 2024-09-12T13:36:23.290 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster2 mirror image-primary1 up+unknown '' '' 2024-09-12T13:36:23.291 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:36:23.291 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:36:23.291 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:36:23.291 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:36:23.291 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:36:23.291 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:36:23.291 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:36:23.291 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:36:23.292 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster2 mirror image status mirror/image-primary1 2024-09-12T13:36:23.302 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster2 mirror image-primary5 up+unknown '' '' 2024-09-12T13:36:23.302 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:36:23.302 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:36:23.302 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:36:23.302 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:36:23.302 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:36:23.302 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:36:23.302 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:36:23.303 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:36:23.303 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster2 mirror image status mirror/image-primary5 2024-09-12T13:36:23.325 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster2 mirror image-primary6 up+unknown '' '' 2024-09-12T13:36:23.325 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:36:23.326 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:36:23.326 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:36:23.326 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:36:23.326 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:36:23.326 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:36:23.326 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:36:23.327 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:36:23.327 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster2 mirror image status mirror/image-primary6 2024-09-12T13:36:23.406 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary1: 2024-09-12T13:36:23.406 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 4ee25370-f977-4ecf-acda-dc9a3ae2ee9d 2024-09-12T13:36:23.406 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+unknown 2024-09-12T13:36:23.406 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: remote image is not primary 2024-09-12T13:36:23.406 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:36:23.406 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:36:17' 2024-09-12T13:36:23.406 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:36:23.406 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary3: 2024-09-12T13:36:23.406 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 9cc4db57-5517-41a1-8d14-519d4043d1a5 2024-09-12T13:36:23.406 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+unknown 2024-09-12T13:36:23.407 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: remote image is not primary 2024-09-12T13:36:23.407 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:36:23.407 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:36:17' 2024-09-12T13:36:23.407 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:36:23.409 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+unknown 2024-09-12T13:36:23.410 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:36:23.414 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+unknown 2024-09-12T13:36:23.414 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:36:23.415 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: remote image is not primary 2024-09-12T13:36:23.416 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:36:23.417 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:36:23.419 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: remote image is not primary 2024-09-12T13:36:23.419 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:36:23.419 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:36:23.420 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:36:23.420 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:36:23.422 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:36:23.422 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:36:23.423 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:36:23.424 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:36:23.427 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:36:23.428 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:36:23.428 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:36:23.429 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:36:23.429 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary5: 2024-09-12T13:36:23.430 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 95f21a09-f11d-496c-bf07-0203910a05a6 2024-09-12T13:36:23.430 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+unknown 2024-09-12T13:36:23.430 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: remote image is not primary 2024-09-12T13:36:23.430 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:36:23.430 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:36:17' 2024-09-12T13:36:23.430 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:36:23.430 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:36:23.431 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:36:23.433 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+unknown 2024-09-12T13:36:23.433 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:36:23.435 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:36:17' 2024-09-12T13:36:23.435 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster2 mirror image-primary3 up+unknown '2024-09-12 13:36:17' 2024-09-12T13:36:23.436 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:36:23.436 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:36:23.436 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:36:23.436 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:36:23.436 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:36:17' 2024-09-12T13:36:23.436 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:36:23.437 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:36:17' 2024-09-12T13:36:23.437 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster2 mirror image-primary1 up+unknown '2024-09-12 13:36:17' 2024-09-12T13:36:23.437 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:36:23.437 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2024-09-12T13:36:23.437 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:36:23.437 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:36:23.437 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:36:23.437 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:36:23.437 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:36:17' 2024-09-12T13:36:23.438 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:36:23.438 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:36:23.438 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: remote image is not primary 2024-09-12T13:36:23.439 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2024-09-12T13:36:23.439 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:36:23.440 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:36:23.441 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:36:23.444 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:36:23.445 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:36:23.449 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:36:23.450 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary6: 2024-09-12T13:36:23.450 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: dcf3f6ae-ecc8-4d77-9308-bede00122fcd 2024-09-12T13:36:23.450 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+unknown 2024-09-12T13:36:23.450 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: remote image is not primary 2024-09-12T13:36:23.450 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:36:23.450 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:36:17' 2024-09-12T13:36:23.450 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:36:23.451 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:36:23.451 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:36:23.454 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+unknown 2024-09-12T13:36:23.454 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:36:23.457 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:36:17' 2024-09-12T13:36:23.457 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster2 mirror image-primary5 up+unknown '2024-09-12 13:36:17' 2024-09-12T13:36:23.458 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:36:23.458 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:36:23.458 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:36:23.458 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:36:23.458 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:36:17' 2024-09-12T13:36:23.458 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:36:23.458 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:36:23.458 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2024-09-12T13:36:23.459 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: remote image is not primary 2024-09-12T13:36:23.459 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:36:23.461 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:36:23.461 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:36:23.464 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:36:23.465 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:36:23.469 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:36:23.470 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:36:23.471 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:36:23.478 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:36:17' 2024-09-12T13:36:23.478 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster2 mirror image-primary6 up+unknown '2024-09-12 13:36:17' 2024-09-12T13:36:23.478 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:36:23.478 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:36:23.478 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:36:23.478 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:36:23.479 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:36:17' 2024-09-12T13:36:23.479 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:36:23.479 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:36:23.479 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2024-09-12T13:36:23.689 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041924231mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary14ee25370-f977-4ecf-acda-dc9a3ae2ee9dup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:36:17image-primary107450998e-7967-4447-af66-572bd4fdded4up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:36:17image-primary26c7d67ca-cef8-4a96-a25c-8d89a4671b00up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:36:17image-primary39cc4db57-5517-41a1-8d14-519d4043d1a5up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:36:17image-primary40203765c-0327-4045-b74a-e9bc90c8f4a0up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:36:17image-primary595f21a09-f11d-496c-bf07-0203910a05a6up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:36:17image-primary6dcf3f6ae-ecc8-4d77-9308-bede00122fcdup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:36:17image-primary7da6437a6-d0c6-4534-a0e0-60fb07f54f52up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:36:17image-primary8b415829c-3d1c-41f6-a1aa-e76e40e110d1up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:36:17image-primary91a8d4f4e-16a8-41c0-8b42-954a67b2e30dup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:36:17' 2024-09-12T13:36:23.689 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:36:23.691 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary3'\'']/last_update' 2024-09-12T13:36:23.707 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:36:17' 2024-09-12T13:36:23.707 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary3'\'']/state' 2024-09-12T13:36:23.710 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041924231mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary14ee25370-f977-4ecf-acda-dc9a3ae2ee9dup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:36:17image-primary107450998e-7967-4447-af66-572bd4fdded4up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:36:17image-primary26c7d67ca-cef8-4a96-a25c-8d89a4671b00up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:36:17image-primary39cc4db57-5517-41a1-8d14-519d4043d1a5up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:36:17image-primary40203765c-0327-4045-b74a-e9bc90c8f4a0up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:36:17image-primary595f21a09-f11d-496c-bf07-0203910a05a6up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:36:17image-primary6dcf3f6ae-ecc8-4d77-9308-bede00122fcdup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:36:17image-primary7da6437a6-d0c6-4534-a0e0-60fb07f54f52up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:36:17image-primary8b415829c-3d1c-41f6-a1aa-e76e40e110d1up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:36:17image-primary91a8d4f4e-16a8-41c0-8b42-954a67b2e30dup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:36:17' 2024-09-12T13:36:23.710 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:36:23.711 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary1'\'']/last_update' 2024-09-12T13:36:23.722 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:36:17' 2024-09-12T13:36:23.722 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+unknown 2024-09-12T13:36:23.722 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:36:23.722 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary1'\'']/state' 2024-09-12T13:36:23.723 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+unknown 2024-09-12T13:36:23.727 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041924231mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary14ee25370-f977-4ecf-acda-dc9a3ae2ee9dup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:36:17image-primary107450998e-7967-4447-af66-572bd4fdded4up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:36:17image-primary26c7d67ca-cef8-4a96-a25c-8d89a4671b00up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:36:17image-primary39cc4db57-5517-41a1-8d14-519d4043d1a5up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:36:17image-primary40203765c-0327-4045-b74a-e9bc90c8f4a0up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:36:17image-primary595f21a09-f11d-496c-bf07-0203910a05a6up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:36:17image-primary6dcf3f6ae-ecc8-4d77-9308-bede00122fcdup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:36:17image-primary7da6437a6-d0c6-4534-a0e0-60fb07f54f52up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:36:17image-primary8b415829c-3d1c-41f6-a1aa-e76e40e110d1up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:36:17image-primary91a8d4f4e-16a8-41c0-8b42-954a67b2e30dup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:36:17' 2024-09-12T13:36:23.727 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:36:23.727 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:36:23.729 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary5'\'']/last_update' 2024-09-12T13:36:23.729 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:36:23.729 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:36:23.729 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ promote_image cluster2 mirror image-primary3 2024-09-12T13:36:23.729 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:36:23.729 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:36:23.729 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:36:23.729 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local force= 2024-09-12T13:36:23.729 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 mirror image promote mirror/image-primary3 2024-09-12T13:36:23.732 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+unknown 2024-09-12T13:36:23.733 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:36:23.733 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+unknown 2024-09-12T13:36:23.736 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:36:17' 2024-09-12T13:36:23.737 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:36:23.737 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:36:23.737 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:36:23.737 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ promote_image cluster2 mirror image-primary1 2024-09-12T13:36:23.738 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:36:23.738 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:36:23.738 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:36:23.738 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local force= 2024-09-12T13:36:23.738 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 mirror image promote mirror/image-primary1 2024-09-12T13:36:23.738 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary5'\'']/state' 2024-09-12T13:36:23.745 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+unknown 2024-09-12T13:36:23.745 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041924231mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary14ee25370-f977-4ecf-acda-dc9a3ae2ee9dup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:36:17image-primary107450998e-7967-4447-af66-572bd4fdded4up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:36:17image-primary26c7d67ca-cef8-4a96-a25c-8d89a4671b00up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:36:17image-primary39cc4db57-5517-41a1-8d14-519d4043d1a5up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:36:17image-primary40203765c-0327-4045-b74a-e9bc90c8f4a0up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:36:17image-primary595f21a09-f11d-496c-bf07-0203910a05a6up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:36:17image-primary6dcf3f6ae-ecc8-4d77-9308-bede00122fcdup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:36:17image-primary7da6437a6-d0c6-4534-a0e0-60fb07f54f52up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:36:17image-primary8b415829c-3d1c-41f6-a1aa-e76e40e110d1up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:36:17image-primary91a8d4f4e-16a8-41c0-8b42-954a67b2e30dup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:36:17' 2024-09-12T13:36:23.745 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:36:23.745 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:36:23.746 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+unknown 2024-09-12T13:36:23.746 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary6'\'']/last_update' 2024-09-12T13:36:23.749 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:36:23.751 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:36:23.751 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:36:23.751 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ promote_image cluster2 mirror image-primary5 2024-09-12T13:36:23.751 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:36:23.751 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:36:23.751 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:36:23.751 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local force= 2024-09-12T13:36:23.751 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 mirror image promote mirror/image-primary5 2024-09-12T13:36:23.753 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:36:17' 2024-09-12T13:36:23.754 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary6'\'']/state' 2024-09-12T13:36:23.761 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+unknown 2024-09-12T13:36:23.763 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:36:23.763 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+unknown 2024-09-12T13:36:23.766 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:36:23.768 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:36:23.768 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:36:23.768 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ promote_image cluster2 mirror image-primary6 2024-09-12T13:36:23.768 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:36:23.768 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:36:23.768 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:36:23.768 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local force= 2024-09-12T13:36:23.768 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 mirror image promote mirror/image-primary6 2024-09-12T13:36:23.967 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary7 up+unknown '' '' 2024-09-12T13:36:23.967 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:36:23.967 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:36:23.967 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:36:23.967 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:36:23.967 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:36:23.967 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:36:23.967 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:36:23.968 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:36:23.968 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary7 2024-09-12T13:36:24.052 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary7: 2024-09-12T13:36:24.052 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: da6437a6-d0c6-4534-a0e0-60fb07f54f52 2024-09-12T13:36:24.053 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+unknown 2024-09-12T13:36:24.053 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: remote image is not primary 2024-09-12T13:36:24.053 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:36:24.053 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:36:17' 2024-09-12T13:36:24.053 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:36:24.055 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+unknown 2024-09-12T13:36:24.055 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:36:24.058 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: remote image is not primary 2024-09-12T13:36:24.058 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:36:24.059 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:36:24.060 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:36:24.065 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:36:24.065 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:36:24.066 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:36:24.066 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:36:24.066 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:36:24.071 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:36:17' 2024-09-12T13:36:24.071 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary7 up+unknown '2024-09-12 13:36:17' 2024-09-12T13:36:24.071 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:36:24.071 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:36:24.071 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:36:24.072 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:36:24.072 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:36:17' 2024-09-12T13:36:24.072 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:36:24.072 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:36:24.072 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-09-12T13:36:24.188 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041934232mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary14ee25370-f977-4ecf-acda-dc9a3ae2ee9dup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:36:17image-primary107450998e-7967-4447-af66-572bd4fdded4up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:36:17image-primary26c7d67ca-cef8-4a96-a25c-8d89a4671b00up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:36:17image-primary39cc4db57-5517-41a1-8d14-519d4043d1a5up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:36:17image-primary40203765c-0327-4045-b74a-e9bc90c8f4a0up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:36:17image-primary595f21a09-f11d-496c-bf07-0203910a05a6up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:36:17image-primary6dcf3f6ae-ecc8-4d77-9308-bede00122fcdup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:36:17image-primary7da6437a6-d0c6-4534-a0e0-60fb07f54f52up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:36:17image-primary8b415829c-3d1c-41f6-a1aa-e76e40e110d1up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:36:17image-primary91a8d4f4e-16a8-41c0-8b42-954a67b2e30dup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:36:17' 2024-09-12T13:36:24.188 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:36:24.189 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary7'\'']/last_update' 2024-09-12T13:36:24.193 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:36:17' 2024-09-12T13:36:24.194 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary7'\'']/state' 2024-09-12T13:36:24.197 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+unknown 2024-09-12T13:36:24.198 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:36:24.199 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+unknown 2024-09-12T13:36:24.201 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:36:24.202 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:36:24.202 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:36:24.202 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster2 mirror image-primary7 up+unknown 2024-09-12T13:36:24.202 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:36:24.202 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:36:24.202 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:36:24.202 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:36:24.202 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:36:24.202 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:36:24.202 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:36:24.203 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:36:24.232 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster2 mirror image-primary4 up+unknown '' '' 2024-09-12T13:36:24.232 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:36:24.232 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:36:24.232 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:36:24.232 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:36:24.233 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:36:24.233 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:36:24.233 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:36:24.233 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:36:24.233 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster2 mirror image status mirror/image-primary4 2024-09-12T13:36:24.309 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Image promoted to primary 2024-09-12T13:36:24.315 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Image promoted to primary 2024-09-12T13:36:24.316 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Image promoted to primary 2024-09-12T13:36:24.333 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Image promoted to primary 2024-09-12T13:36:24.342 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \n\b\d ]] 2024-09-12T13:36:24.342 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \k\r\b\d ]] 2024-09-12T13:36:24.343 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \n\b\d ]] 2024-09-12T13:36:24.343 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \k\r\b\d ]] 2024-09-12T13:36:24.344 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo rbd --cluster cluster2 device map -t krbd mirror/image-primary1 2024-09-12T13:36:24.346 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo rbd --cluster cluster2 device map -t krbd mirror/image-primary5 2024-09-12T13:36:24.347 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \n\b\d ]] 2024-09-12T13:36:24.347 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \k\r\b\d ]] 2024-09-12T13:36:24.348 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo rbd --cluster cluster2 device map -t krbd mirror/image-primary6 2024-09-12T13:36:24.351 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary4: 2024-09-12T13:36:24.351 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 0203765c-0327-4045-b74a-e9bc90c8f4a0 2024-09-12T13:36:24.352 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+unknown 2024-09-12T13:36:24.352 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: remote image is not primary 2024-09-12T13:36:24.352 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:36:24.352 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:36:17' 2024-09-12T13:36:24.352 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:36:24.357 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+unknown 2024-09-12T13:36:24.358 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:36:24.359 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \n\b\d ]] 2024-09-12T13:36:24.359 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \k\r\b\d ]] 2024-09-12T13:36:24.360 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo rbd --cluster cluster2 device map -t krbd mirror/image-primary3 2024-09-12T13:36:24.363 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: remote image is not primary 2024-09-12T13:36:24.363 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:36:24.365 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:36:24.366 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:36:24.370 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:36:24.371 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:36:24.375 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:36:24.376 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:36:24.377 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:36:24.384 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:36:17' 2024-09-12T13:36:24.384 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster2 mirror image-primary4 up+unknown '2024-09-12 13:36:17' 2024-09-12T13:36:24.384 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:36:24.384 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:36:24.385 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:36:24.385 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:36:24.385 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:36:17' 2024-09-12T13:36:24.385 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:36:24.387 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:36:24.387 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2024-09-12T13:36:24.626 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041924231mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary14ee25370-f977-4ecf-acda-dc9a3ae2ee9dup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:36:17image-primary107450998e-7967-4447-af66-572bd4fdded4up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:36:17image-primary26c7d67ca-cef8-4a96-a25c-8d89a4671b00up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:36:17image-primary39cc4db57-5517-41a1-8d14-519d4043d1a5up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:36:17image-primary40203765c-0327-4045-b74a-e9bc90c8f4a0up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:36:17image-primary595f21a09-f11d-496c-bf07-0203910a05a6up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:36:17image-primary6dcf3f6ae-ecc8-4d77-9308-bede00122fcdup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:36:17image-primary7da6437a6-d0c6-4534-a0e0-60fb07f54f52up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:36:17image-primary8b415829c-3d1c-41f6-a1aa-e76e40e110d1up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:36:17image-primary91a8d4f4e-16a8-41c0-8b42-954a67b2e30dup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:36:17' 2024-09-12T13:36:24.626 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:36:24.628 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary4'\'']/last_update' 2024-09-12T13:36:24.636 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:36:17' 2024-09-12T13:36:24.638 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary4'\'']/state' 2024-09-12T13:36:24.640 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ dev=/dev/rbd4 2024-09-12T13:36:24.646 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo md5sum /dev/rbd4 2024-09-12T13:36:24.647 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+unknown 2024-09-12T13:36:24.649 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ awk '{print $1}' 2024-09-12T13:36:24.650 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:36:24.650 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ dev=/dev/rbd3 2024-09-12T13:36:24.651 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+unknown 2024-09-12T13:36:24.655 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ awk '{print $1}' 2024-09-12T13:36:24.655 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ dev=/dev/rbd5 2024-09-12T13:36:24.655 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:36:24.656 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:36:24.656 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:36:24.656 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ promote_image cluster2 mirror image-primary4 2024-09-12T13:36:24.656 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:36:24.656 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:36:24.656 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:36:24.656 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local force= 2024-09-12T13:36:24.657 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 mirror image promote mirror/image-primary4 2024-09-12T13:36:24.659 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo md5sum /dev/rbd3 2024-09-12T13:36:24.660 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ awk '{print $1}' 2024-09-12T13:36:24.660 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo md5sum /dev/rbd5 2024-09-12T13:36:24.661 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ dev=/dev/rbd6 2024-09-12T13:36:24.665 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo md5sum /dev/rbd6 2024-09-12T13:36:24.666 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ awk '{print $1}' 2024-09-12T13:36:24.909 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary9 up+unknown '' '' 2024-09-12T13:36:24.909 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:36:24.909 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:36:24.909 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:36:24.909 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:36:24.909 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:36:24.909 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:36:24.910 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:36:24.911 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:36:24.911 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary9 2024-09-12T13:36:25.033 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary9: 2024-09-12T13:36:25.033 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 1a8d4f4e-16a8-41c0-8b42-954a67b2e30d 2024-09-12T13:36:25.033 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+unknown 2024-09-12T13:36:25.033 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: remote image is not primary 2024-09-12T13:36:25.034 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:36:25.034 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:36:17' 2024-09-12T13:36:25.034 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:36:25.037 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+unknown 2024-09-12T13:36:25.038 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:36:25.043 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: remote image is not primary 2024-09-12T13:36:25.044 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:36:25.046 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:36:25.047 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:36:25.051 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:36:25.052 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:36:25.057 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:36:25.058 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:36:25.059 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:36:25.066 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:36:17' 2024-09-12T13:36:25.066 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary9 up+unknown '2024-09-12 13:36:17' 2024-09-12T13:36:25.067 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:36:25.067 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:36:25.067 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:36:25.067 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:36:25.067 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:36:17' 2024-09-12T13:36:25.067 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:36:25.068 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:36:25.068 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-09-12T13:36:25.205 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster2 mirror image-primary7 up+unknown '' '' 2024-09-12T13:36:25.205 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:36:25.205 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:36:25.205 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:36:25.205 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:36:25.205 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:36:25.206 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:36:25.206 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:36:25.209 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:36:25.209 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster2 mirror image status mirror/image-primary7 2024-09-12T13:36:25.283 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041934232mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary14ee25370-f977-4ecf-acda-dc9a3ae2ee9dup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:36:17image-primary107450998e-7967-4447-af66-572bd4fdded4up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:36:17image-primary26c7d67ca-cef8-4a96-a25c-8d89a4671b00up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:36:17image-primary39cc4db57-5517-41a1-8d14-519d4043d1a5up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:36:17image-primary40203765c-0327-4045-b74a-e9bc90c8f4a0up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:36:17image-primary595f21a09-f11d-496c-bf07-0203910a05a6up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:36:17image-primary6dcf3f6ae-ecc8-4d77-9308-bede00122fcdup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:36:17image-primary7da6437a6-d0c6-4534-a0e0-60fb07f54f52up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:36:17image-primary8b415829c-3d1c-41f6-a1aa-e76e40e110d1up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:36:17image-primary91a8d4f4e-16a8-41c0-8b42-954a67b2e30dup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:36:17' 2024-09-12T13:36:25.283 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:36:25.286 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary9'\'']/last_update' 2024-09-12T13:36:25.294 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:36:17' 2024-09-12T13:36:25.295 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary9'\'']/state' 2024-09-12T13:36:25.304 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+unknown 2024-09-12T13:36:25.305 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:36:25.305 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+unknown 2024-09-12T13:36:25.309 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:36:25.311 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:36:25.311 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:36:25.311 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster2 mirror image-primary9 up+unknown 2024-09-12T13:36:25.311 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:36:25.311 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:36:25.311 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:36:25.311 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:36:25.312 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:36:25.312 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:36:25.312 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:36:25.312 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:36:25.369 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Image promoted to primary 2024-09-12T13:36:25.408 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary7: 2024-09-12T13:36:25.408 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: da6437a6-d0c6-4534-a0e0-60fb07f54f52 2024-09-12T13:36:25.408 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+unknown 2024-09-12T13:36:25.408 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: remote image is not primary 2024-09-12T13:36:25.408 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:36:25.408 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:36:17' 2024-09-12T13:36:25.408 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:36:25.415 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+unknown 2024-09-12T13:36:25.416 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:36:25.418 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \n\b\d ]] 2024-09-12T13:36:25.419 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \k\r\b\d ]] 2024-09-12T13:36:25.420 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo rbd --cluster cluster2 device map -t krbd mirror/image-primary4 2024-09-12T13:36:25.423 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: remote image is not primary 2024-09-12T13:36:25.425 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:36:25.427 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:36:25.429 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:36:25.435 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:36:25.440 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:36:25.449 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:36:25.450 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:36:25.451 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:36:25.468 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:36:17' 2024-09-12T13:36:25.468 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster2 mirror image-primary7 up+unknown '2024-09-12 13:36:17' 2024-09-12T13:36:25.468 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:36:25.468 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:36:25.468 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:36:25.468 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:36:25.468 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:36:17' 2024-09-12T13:36:25.468 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:36:25.471 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:36:25.471 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2024-09-12T13:36:25.739 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ dev=/dev/rbd7 2024-09-12T13:36:25.744 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo md5sum /dev/rbd7 2024-09-12T13:36:25.745 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ awk '{print $1}' 2024-09-12T13:36:25.777 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041924231mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary14ee25370-f977-4ecf-acda-dc9a3ae2ee9dup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:36:17image-primary107450998e-7967-4447-af66-572bd4fdded4up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:36:17image-primary26c7d67ca-cef8-4a96-a25c-8d89a4671b00up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:36:17image-primary39cc4db57-5517-41a1-8d14-519d4043d1a5up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:36:17image-primary40203765c-0327-4045-b74a-e9bc90c8f4a0up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:36:17image-primary595f21a09-f11d-496c-bf07-0203910a05a6up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:36:17image-primary6dcf3f6ae-ecc8-4d77-9308-bede00122fcdup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:36:17image-primary7da6437a6-d0c6-4534-a0e0-60fb07f54f52up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:36:17image-primary8b415829c-3d1c-41f6-a1aa-e76e40e110d1up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:36:17image-primary91a8d4f4e-16a8-41c0-8b42-954a67b2e30dup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:36:17' 2024-09-12T13:36:25.777 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:36:25.778 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary7'\'']/last_update' 2024-09-12T13:36:25.789 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:36:17' 2024-09-12T13:36:25.790 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary7'\'']/state' 2024-09-12T13:36:25.807 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+unknown 2024-09-12T13:36:25.809 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:36:25.812 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+unknown 2024-09-12T13:36:25.819 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:36:25.820 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:36:25.822 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:36:25.822 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ promote_image cluster2 mirror image-primary7 2024-09-12T13:36:25.822 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:36:25.822 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:36:25.822 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:36:25.823 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local force= 2024-09-12T13:36:25.823 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 mirror image promote mirror/image-primary7 2024-09-12T13:36:26.321 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster2 mirror image-primary9 up+unknown '' '' 2024-09-12T13:36:26.321 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:36:26.321 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:36:26.321 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:36:26.321 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:36:26.321 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:36:26.321 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:36:26.321 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:36:26.322 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:36:26.322 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster2 mirror image status mirror/image-primary9 2024-09-12T13:36:26.367 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Image promoted to primary 2024-09-12T13:36:26.410 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \n\b\d ]] 2024-09-12T13:36:26.410 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \k\r\b\d ]] 2024-09-12T13:36:26.412 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo rbd --cluster cluster2 device map -t krbd mirror/image-primary7 2024-09-12T13:36:26.585 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary9: 2024-09-12T13:36:26.585 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 1a8d4f4e-16a8-41c0-8b42-954a67b2e30d 2024-09-12T13:36:26.585 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+unknown 2024-09-12T13:36:26.585 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: remote image is not primary 2024-09-12T13:36:26.585 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:36:26.585 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:36:17' 2024-09-12T13:36:26.585 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:36:26.594 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+unknown 2024-09-12T13:36:26.595 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:36:26.607 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: remote image is not primary 2024-09-12T13:36:26.610 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:36:26.612 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:36:26.613 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:36:26.618 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:36:26.621 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:36:26.632 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:36:26.638 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:36:26.642 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:36:26.663 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:36:17' 2024-09-12T13:36:26.664 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster2 mirror image-primary9 up+unknown '2024-09-12 13:36:17' 2024-09-12T13:36:26.664 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:36:26.664 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:36:26.664 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:36:26.664 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:36:26.664 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:36:17' 2024-09-12T13:36:26.664 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:36:26.667 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:36:26.667 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2024-09-12T13:36:26.834 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ dev=/dev/rbd8 2024-09-12T13:36:26.838 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo md5sum /dev/rbd8 2024-09-12T13:36:26.839 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ awk '{print $1}' 2024-09-12T13:36:27.170 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041924231mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary14ee25370-f977-4ecf-acda-dc9a3ae2ee9dup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:36:17image-primary107450998e-7967-4447-af66-572bd4fdded4up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:36:17image-primary26c7d67ca-cef8-4a96-a25c-8d89a4671b00up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:36:17image-primary39cc4db57-5517-41a1-8d14-519d4043d1a5up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:36:17image-primary40203765c-0327-4045-b74a-e9bc90c8f4a0up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:36:17image-primary595f21a09-f11d-496c-bf07-0203910a05a6up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:36:17image-primary6dcf3f6ae-ecc8-4d77-9308-bede00122fcdup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:36:17image-primary7da6437a6-d0c6-4534-a0e0-60fb07f54f52up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:36:17image-primary8b415829c-3d1c-41f6-a1aa-e76e40e110d1up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:36:17image-primary91a8d4f4e-16a8-41c0-8b42-954a67b2e30dup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:36:17' 2024-09-12T13:36:27.170 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:36:27.171 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary9'\'']/last_update' 2024-09-12T13:36:27.195 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:36:17' 2024-09-12T13:36:27.197 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary9'\'']/state' 2024-09-12T13:36:27.220 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+unknown 2024-09-12T13:36:27.224 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:36:27.225 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+unknown 2024-09-12T13:36:27.231 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:36:27.232 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:36:27.232 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:36:27.232 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ promote_image cluster2 mirror image-primary9 2024-09-12T13:36:27.232 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:36:27.232 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:36:27.232 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:36:27.233 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local force= 2024-09-12T13:36:27.233 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 mirror image promote mirror/image-primary9 2024-09-12T13:36:28.356 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Image promoted to primary 2024-09-12T13:36:28.393 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \n\b\d ]] 2024-09-12T13:36:28.393 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \k\r\b\d ]] 2024-09-12T13:36:28.394 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo rbd --cluster cluster2 device map -t krbd mirror/image-primary9 2024-09-12T13:36:28.697 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ dev=/dev/rbd9 2024-09-12T13:36:28.701 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo md5sum /dev/rbd9 2024-09-12T13:36:28.702 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ awk '{print $1}' 2024-09-12T13:37:08.528 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ promote_md5=c766e1b9b40887e252f808b7c8765512 2024-09-12T13:37:08.529 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo rbd --cluster cluster2 device unmap -t krbd /dev/rbd0 2024-09-12T13:37:09.146 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ c766e1b9b40887e252f808b7c8765512 != \c\7\6\6\e\1\b\9\b\4\0\8\8\7\e\2\5\2\f\8\0\8\b\7\c\8\7\6\5\5\1\2 ]] 2024-09-12T13:37:13.710 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ promote_md5=639b299f6db074096373eb56deafce73 2024-09-12T13:37:13.710 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo rbd --cluster cluster2 device unmap -t krbd /dev/rbd1 2024-09-12T13:37:13.982 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ 639b299f6db074096373eb56deafce73 != \6\3\9\b\2\9\9\f\6\d\b\0\7\4\0\9\6\3\7\3\e\b\5\6\d\e\a\f\c\e\7\3 ]] 2024-09-12T13:37:15.550 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ promote_md5=d6bd25670a3cb91d0ea71320af1306ab 2024-09-12T13:37:15.550 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo rbd --cluster cluster2 device unmap -t krbd /dev/rbd2 2024-09-12T13:37:15.779 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ d6bd25670a3cb91d0ea71320af1306ab != \d\6\b\d\2\5\6\7\0\a\3\c\b\9\1\d\0\e\a\7\1\3\2\0\a\f\1\3\0\6\a\b ]] 2024-09-12T13:37:16.279 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ promote_md5=89b6dc0f5d258aa7b152b3b7f3ae9ffe 2024-09-12T13:37:16.279 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo rbd --cluster cluster2 device unmap -t krbd /dev/rbd3 2024-09-12T13:37:16.447 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ 89b6dc0f5d258aa7b152b3b7f3ae9ffe != \8\9\b\6\d\c\0\f\5\d\2\5\8\a\a\7\b\1\5\2\b\3\b\7\f\3\a\e\9\f\f\e ]] 2024-09-12T13:37:16.448 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${COMPARE_PIDS[@]} 2024-09-12T13:37:16.448 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 90046 2024-09-12T13:37:16.448 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${COMPARE_PIDS[@]} 2024-09-12T13:37:16.448 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 90048 2024-09-12T13:37:16.877 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ promote_md5=c158e231ec70974d8764bac597b398f3 2024-09-12T13:37:16.878 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo rbd --cluster cluster2 device unmap -t krbd /dev/rbd5 2024-09-12T13:37:16.989 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ c158e231ec70974d8764bac597b398f3 != \c\1\5\8\e\2\3\1\e\c\7\0\9\7\4\d\8\7\6\4\b\a\c\5\9\7\b\3\9\8\f\3 ]] 2024-09-12T13:37:16.990 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${COMPARE_PIDS[@]} 2024-09-12T13:37:16.990 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 90050 2024-09-12T13:37:19.143 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ promote_md5=85577d212ced413d69d145a64948a111 2024-09-12T13:37:19.143 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo rbd --cluster cluster2 device unmap -t krbd /dev/rbd9 2024-09-12T13:37:19.154 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ promote_md5=3d029a8d927783cfa30b5ad87d7fd36d 2024-09-12T13:37:19.154 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo rbd --cluster cluster2 device unmap -t krbd /dev/rbd6 2024-09-12T13:37:19.247 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ promote_md5=c9ae793770c6420b1bb1ca9b01a292ed 2024-09-12T13:37:19.248 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo rbd --cluster cluster2 device unmap -t krbd /dev/rbd7 2024-09-12T13:37:19.301 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ 3d029a8d927783cfa30b5ad87d7fd36d != \3\d\0\2\9\a\8\d\9\2\7\7\8\3\c\f\a\3\0\b\5\a\d\8\7\d\7\f\d\3\6\d ]] 2024-09-12T13:37:19.309 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ 85577d212ced413d69d145a64948a111 != \8\5\5\7\7\d\2\1\2\c\e\d\4\1\3\d\6\9\d\1\4\5\a\6\4\9\4\8\a\1\1\1 ]] 2024-09-12T13:37:19.374 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ c9ae793770c6420b1bb1ca9b01a292ed != \c\9\a\e\7\9\3\7\7\0\c\6\4\2\0\b\1\b\b\1\c\a\9\b\0\1\a\2\9\2\e\d ]] 2024-09-12T13:37:19.374 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${COMPARE_PIDS[@]} 2024-09-12T13:37:19.375 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 90052 2024-09-12T13:37:19.589 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ promote_md5=9fcb612174a84a4dc5dc4639dffa95bd 2024-09-12T13:37:19.590 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo rbd --cluster cluster2 device unmap -t krbd /dev/rbd4 2024-09-12T13:37:19.712 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ 9fcb612174a84a4dc5dc4639dffa95bd != \9\f\c\b\6\1\2\1\7\4\a\8\4\a\4\d\c\5\d\c\4\6\3\9\d\f\f\a\9\5\b\d ]] 2024-09-12T13:37:19.712 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${COMPARE_PIDS[@]} 2024-09-12T13:37:19.712 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 90054 2024-09-12T13:37:19.713 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${COMPARE_PIDS[@]} 2024-09-12T13:37:19.713 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 90056 2024-09-12T13:37:20.128 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ promote_md5=bc43079dacea4896e58f625c488ff1a1 2024-09-12T13:37:20.128 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo rbd --cluster cluster2 device unmap -t krbd /dev/rbd8 2024-09-12T13:37:20.216 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ bc43079dacea4896e58f625c488ff1a1 != \b\c\4\3\0\7\9\d\a\c\e\a\4\8\9\6\e\5\8\f\6\2\5\c\4\8\8\f\f\1\a\1 ]] 2024-09-12T13:37:20.216 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${COMPARE_PIDS[@]} 2024-09-12T13:37:20.216 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 90058 2024-09-12T13:37:20.216 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${COMPARE_PIDS[@]} 2024-09-12T13:37:20.216 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 90060 2024-09-12T13:37:20.216 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${COMPARE_PIDS[@]} 2024-09-12T13:37:20.216 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 90061 2024-09-12T13:37:20.216 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ (( RET != 0 )) 2024-09-12T13:37:20.216 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:37:20.216 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ IMG=image-primary1 2024-09-12T13:37:20.217 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_replaying_status_in_pool_dir cluster1 mirror image-primary1 2024-09-12T13:37:20.217 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:37:20.217 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:37:20.217 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:37:20.217 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' snapshot = journal ']' 2024-09-12T13:37:20.217 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary1 up+replaying 2024-09-12T13:37:20.217 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:37:20.217 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:37:20.217 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:37:20.217 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:37:20.217 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:37:20.217 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:37:20.217 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:37:20.217 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:37:21.237 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary1 up+replaying '' '' 2024-09-12T13:37:21.238 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:37:21.238 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:37:21.238 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:37:21.238 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:37:21.238 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:37:21.238 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:37:21.238 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:37:21.238 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:37:21.239 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary1 2024-09-12T13:37:21.320 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary1: 2024-09-12T13:37:21.320 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 4ee25370-f977-4ecf-acda-dc9a3ae2ee9d 2024-09-12T13:37:21.321 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+replaying 2024-09-12T13:37:21.321 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: replaying, {"bytes_per_second":3259801.6,"bytes_per_snapshot":65196032.0,"last_snapshot_bytes":65196032,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726148184,"remote_snapshot_timestamp":1726148184,"replay_state":"idle"} 2024-09-12T13:37:21.321 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:37:21.321 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:37:17' 2024-09-12T13:37:21.321 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+replaying' 2024-09-12T13:37:21.323 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+replaying 2024-09-12T13:37:21.323 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:37:21.326 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: replaying, {"bytes_per_second":3259801.6,"bytes_per_snapshot":65196032.0,"last_snapshot_bytes":65196032,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726148184,"remote_snapshot_timestamp":1726148184,"replay_state":"idle"} 2024-09-12T13:37:21.326 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:37:21.326 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+replaying 2024-09-12T13:37:21.327 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:37:21.329 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+replaying 2024-09-12T13:37:21.329 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:37:21.332 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:37:21.332 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:37:21.333 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:37:21.337 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:37:17' 2024-09-12T13:37:21.337 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary1 up+replaying '2024-09-12 13:37:17' 2024-09-12T13:37:21.337 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:37:21.337 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:37:21.337 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:37:21.338 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:37:21.338 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:37:17' 2024-09-12T13:37:21.338 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:37:21.338 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:37:21.338 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-09-12T13:37:21.448 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041934232mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary14ee25370-f977-4ecf-acda-dc9a3ae2ee9dup+replayingreplaying, {"bytes_per_second":3259801.6,"bytes_per_snapshot":65196032.0,"last_snapshot_bytes":65196032,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726148184,"remote_snapshot_timestamp":1726148184,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:37:17image-primary107450998e-7967-4447-af66-572bd4fdded4up+replayingreplaying, {"bytes_per_second":3175424.0,"bytes_per_snapshot":63508480.0,"last_snapshot_bytes":63508480,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726148180,"remote_snapshot_timestamp":1726148180,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:37:17image-primary26c7d67ca-cef8-4a96-a25c-8d89a4671b00up+replayingreplaying, {"bytes_per_second":3169689.6,"bytes_per_snapshot":63393792.0,"last_snapshot_bytes":63393792,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726148180,"remote_snapshot_timestamp":1726148180,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:37:17image-primary39cc4db57-5517-41a1-8d14-519d4043d1a5up+replayingreplaying, {"bytes_per_second":3338444.8,"bytes_per_snapshot":66768896.0,"last_snapshot_bytes":66768896,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726148184,"remote_snapshot_timestamp":1726148184,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:37:17image-primary40203765c-0327-4045-b74a-e9bc90c8f4a0up+replayingreplaying, {"bytes_per_second":3304448.0,"bytes_per_snapshot":66088960.0,"last_snapshot_bytes":66088960,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726148185,"remote_snapshot_timestamp":1726148185,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:37:17image-primary595f21a09-f11d-496c-bf07-0203910a05a6up+replayingreplaying, {"bytes_per_second":3241779.2,"bytes_per_snapshot":64835584.0,"last_snapshot_bytes":64835584,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726148184,"remote_snapshot_timestamp":1726148184,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:37:17image-primary6dcf3f6ae-ecc8-4d77-9308-bede00122fcdup+replayingreplaying, {"bytes_per_second":3414630.4,"bytes_per_snapshot":68292608.0,"last_snapshot_bytes":68292608,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726148184,"remote_snapshot_timestamp":1726148184,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:37:17image-primary7da6437a6-d0c6-4534-a0e0-60fb07f54f52up+replayingreplaying, {"bytes_per_second":3090841.6,"bytes_per_snapshot":61816832.0,"last_snapshot_bytes":61816832,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726148186,"remote_snapshot_timestamp":1726148186,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:37:17image-primary8b415829c-3d1c-41f6-a1aa-e76e40e110d1up+replayingreplaying, {"bytes_per_second":3153305.6,"bytes_per_snapshot":63066112.0,"last_snapshot_bytes":63066112,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726148182,"remote_snapshot_timestamp":1726148182,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:37:17image-primary91a8d4f4e-16a8-41c0-8b42-954a67b2e30dup+replayingreplaying, {"bytes_per_second":3334348.8,"bytes_per_snapshot":66686976.0,"last_snapshot_bytes":66686976,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726148188,"remote_snapshot_timestamp":1726148188,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:37:17' 2024-09-12T13:37:21.448 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:37:21.448 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary1'\'']/last_update' 2024-09-12T13:37:21.479 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:37:17' 2024-09-12T13:37:21.479 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary1'\'']/state' 2024-09-12T13:37:21.483 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+replaying 2024-09-12T13:37:21.484 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+replaying 2024-09-12T13:37:21.484 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+replaying 2024-09-12T13:37:21.486 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+replaying 2024-09-12T13:37:21.487 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:37:21.487 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:37:21.487 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ remove_image cluster2 mirror image-primary1 2024-09-12T13:37:21.487 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:37:21.487 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:37:21.487 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:37:21.487 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 snap purge mirror/image-primary1 2024-09-12T13:37:21.547 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 rm mirror/image-primary1 2024-09-12T13:37:21.685 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:37:21.685+0000 7f67e5200640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x56055b865d40 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-09-12T13:37:21.693 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:37:21.685+0000 7f67ed400640 -1 rbd::mirror::ImageReplayer: 0x56055c198c80 [3/4ee25370-f977-4ecf-acda-dc9a3ae2ee9d] operator(): start failed: (85) Interrupted system call should be restarted 2024-09-12T13:37:23.488 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-09-12T13:37:23.494 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_image_removal cluster1 mirror image-primary1 2024-09-12T13:37:23.494 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:37:23.494 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:37:23.494 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:37:23.494 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:37:23.495 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 ls mirror 2024-09-12T13:37:23.495 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep -wq image-primary1 2024-09-12T13:37:23.545 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:37:23.545 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rm -rf test-primary1 2024-09-12T13:37:23.560 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:37:23.560 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ IMG=image-primary2 2024-09-12T13:37:23.560 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_replaying_status_in_pool_dir cluster1 mirror image-primary2 2024-09-12T13:37:23.560 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:37:23.560 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:37:23.560 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:37:23.560 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' snapshot = journal ']' 2024-09-12T13:37:23.560 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary2 up+replaying 2024-09-12T13:37:23.560 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:37:23.560 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:37:23.560 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:37:23.561 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:37:23.561 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:37:23.561 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:37:23.561 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:37:23.561 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:37:24.562 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary2 up+replaying '' '' 2024-09-12T13:37:24.562 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:37:24.562 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:37:24.562 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:37:24.562 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:37:24.563 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:37:24.563 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:37:24.563 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:37:24.563 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:37:24.563 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary2 2024-09-12T13:37:24.644 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary2: 2024-09-12T13:37:24.645 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 6c7d67ca-cef8-4a96-a25c-8d89a4671b00 2024-09-12T13:37:24.645 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+replaying 2024-09-12T13:37:24.645 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: replaying, {"bytes_per_second":3169689.6,"bytes_per_snapshot":63393792.0,"last_snapshot_bytes":63393792,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726148180,"remote_snapshot_timestamp":1726148180,"replay_state":"idle"} 2024-09-12T13:37:24.645 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:37:24.645 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:37:17' 2024-09-12T13:37:24.645 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+replaying' 2024-09-12T13:37:24.647 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+replaying 2024-09-12T13:37:24.647 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:37:24.650 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: replaying, {"bytes_per_second":3169689.6,"bytes_per_snapshot":63393792.0,"last_snapshot_bytes":63393792,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726148180,"remote_snapshot_timestamp":1726148180,"replay_state":"idle"} 2024-09-12T13:37:24.650 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:37:24.651 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+replaying 2024-09-12T13:37:24.651 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:37:24.653 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+replaying 2024-09-12T13:37:24.653 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:37:24.656 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:37:24.657 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:37:24.657 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:37:24.661 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:37:17' 2024-09-12T13:37:24.661 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary2 up+replaying '2024-09-12 13:37:17' 2024-09-12T13:37:24.661 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:37:24.661 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:37:24.661 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:37:24.661 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:37:24.662 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:37:17' 2024-09-12T13:37:24.662 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:37:24.662 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:37:24.662 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-09-12T13:37:24.768 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING941934232mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary107450998e-7967-4447-af66-572bd4fdded4up+replayingreplaying, {"bytes_per_second":3175424.0,"bytes_per_snapshot":63508480.0,"last_snapshot_bytes":63508480,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726148180,"remote_snapshot_timestamp":1726148180,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:37:17image-primary26c7d67ca-cef8-4a96-a25c-8d89a4671b00up+replayingreplaying, {"bytes_per_second":3169689.6,"bytes_per_snapshot":63393792.0,"last_snapshot_bytes":63393792,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726148180,"remote_snapshot_timestamp":1726148180,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:37:17image-primary39cc4db57-5517-41a1-8d14-519d4043d1a5up+replayingreplaying, {"bytes_per_second":3338444.8,"bytes_per_snapshot":66768896.0,"last_snapshot_bytes":66768896,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726148184,"remote_snapshot_timestamp":1726148184,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:37:17image-primary40203765c-0327-4045-b74a-e9bc90c8f4a0up+replayingreplaying, {"bytes_per_second":3304448.0,"bytes_per_snapshot":66088960.0,"last_snapshot_bytes":66088960,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726148185,"remote_snapshot_timestamp":1726148185,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:37:17image-primary595f21a09-f11d-496c-bf07-0203910a05a6up+replayingreplaying, {"bytes_per_second":3241779.2,"bytes_per_snapshot":64835584.0,"last_snapshot_bytes":64835584,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726148184,"remote_snapshot_timestamp":1726148184,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:37:17image-primary6dcf3f6ae-ecc8-4d77-9308-bede00122fcdup+replayingreplaying, {"bytes_per_second":3414630.4,"bytes_per_snapshot":68292608.0,"last_snapshot_bytes":68292608,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726148184,"remote_snapshot_timestamp":1726148184,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:37:17image-primary7da6437a6-d0c6-4534-a0e0-60fb07f54f52up+replayingreplaying, {"bytes_per_second":3090841.6,"bytes_per_snapshot":61816832.0,"last_snapshot_bytes":61816832,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726148186,"remote_snapshot_timestamp":1726148186,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:37:17image-primary8b415829c-3d1c-41f6-a1aa-e76e40e110d1up+replayingreplaying, {"bytes_per_second":3153305.6,"bytes_per_snapshot":63066112.0,"last_snapshot_bytes":63066112,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726148182,"remote_snapshot_timestamp":1726148182,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:37:17image-primary91a8d4f4e-16a8-41c0-8b42-954a67b2e30dup+replayingreplaying, {"bytes_per_second":3334348.8,"bytes_per_snapshot":66686976.0,"last_snapshot_bytes":66686976,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726148188,"remote_snapshot_timestamp":1726148188,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:37:17' 2024-09-12T13:37:24.768 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:37:24.768 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary2'\'']/last_update' 2024-09-12T13:37:24.772 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:37:17' 2024-09-12T13:37:24.773 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary2'\'']/state' 2024-09-12T13:37:24.777 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+replaying 2024-09-12T13:37:24.777 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+replaying 2024-09-12T13:37:24.778 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+replaying 2024-09-12T13:37:24.780 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+replaying 2024-09-12T13:37:24.781 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:37:24.781 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:37:24.781 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ remove_image cluster2 mirror image-primary2 2024-09-12T13:37:24.781 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:37:24.781 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:37:24.781 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:37:24.782 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 snap purge mirror/image-primary2 2024-09-12T13:37:24.840 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 rm mirror/image-primary2 2024-09-12T13:37:24.982 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:37:24.982+0000 7f67e5200640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x5605571a04e0 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-09-12T13:37:24.982 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:37:24.982+0000 7f67eca00640 -1 rbd::mirror::ImageReplayer: 0x5605561acf00 [3/6c7d67ca-cef8-4a96-a25c-8d89a4671b00] operator(): start failed: (85) Interrupted system call should be restarted 2024-09-12T13:37:26.497 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-09-12T13:37:26.503 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_image_removal cluster1 mirror image-primary2 2024-09-12T13:37:26.503 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:37:26.503 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:37:26.503 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:37:26.503 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:37:26.504 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 ls mirror 2024-09-12T13:37:26.504 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep -wq image-primary2 2024-09-12T13:37:26.553 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:37:26.553 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rm -rf test-primary2 2024-09-12T13:37:26.556 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:37:26.556 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ IMG=image-primary3 2024-09-12T13:37:26.556 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_replaying_status_in_pool_dir cluster1 mirror image-primary3 2024-09-12T13:37:26.557 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:37:26.557 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:37:26.557 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:37:26.557 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' snapshot = journal ']' 2024-09-12T13:37:26.557 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary3 up+replaying 2024-09-12T13:37:26.557 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:37:26.557 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:37:26.557 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:37:26.557 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:37:26.557 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:37:26.558 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:37:26.558 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:37:26.558 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:37:27.558 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary3 up+replaying '' '' 2024-09-12T13:37:27.558 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:37:27.558 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:37:27.558 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:37:27.558 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:37:27.558 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:37:27.558 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:37:27.558 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:37:27.559 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:37:27.559 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary3 2024-09-12T13:37:27.642 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary3: 2024-09-12T13:37:27.642 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 9cc4db57-5517-41a1-8d14-519d4043d1a5 2024-09-12T13:37:27.642 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+replaying 2024-09-12T13:37:27.642 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: replaying, {"bytes_per_second":3338444.8,"bytes_per_snapshot":66768896.0,"last_snapshot_bytes":66768896,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726148184,"remote_snapshot_timestamp":1726148184,"replay_state":"idle"} 2024-09-12T13:37:27.642 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:37:27.642 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:37:17' 2024-09-12T13:37:27.643 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+replaying' 2024-09-12T13:37:27.645 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+replaying 2024-09-12T13:37:27.645 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:37:27.648 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: replaying, {"bytes_per_second":3338444.8,"bytes_per_snapshot":66768896.0,"last_snapshot_bytes":66768896,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726148184,"remote_snapshot_timestamp":1726148184,"replay_state":"idle"} 2024-09-12T13:37:27.648 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:37:27.649 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+replaying 2024-09-12T13:37:27.649 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:37:27.651 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+replaying 2024-09-12T13:37:27.651 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:37:27.654 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:37:27.654 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:37:27.655 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:37:27.659 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:37:17' 2024-09-12T13:37:27.659 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary3 up+replaying '2024-09-12 13:37:17' 2024-09-12T13:37:27.659 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:37:27.659 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:37:27.660 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:37:27.660 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:37:27.660 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:37:17' 2024-09-12T13:37:27.660 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:37:27.660 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:37:27.661 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-09-12T13:37:27.764 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING841934232mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary107450998e-7967-4447-af66-572bd4fdded4up+replayingreplaying, {"bytes_per_second":3175424.0,"bytes_per_snapshot":63508480.0,"last_snapshot_bytes":63508480,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726148180,"remote_snapshot_timestamp":1726148180,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:37:17image-primary39cc4db57-5517-41a1-8d14-519d4043d1a5up+replayingreplaying, {"bytes_per_second":3338444.8,"bytes_per_snapshot":66768896.0,"last_snapshot_bytes":66768896,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726148184,"remote_snapshot_timestamp":1726148184,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:37:17image-primary40203765c-0327-4045-b74a-e9bc90c8f4a0up+replayingreplaying, {"bytes_per_second":3304448.0,"bytes_per_snapshot":66088960.0,"last_snapshot_bytes":66088960,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726148185,"remote_snapshot_timestamp":1726148185,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:37:17image-primary595f21a09-f11d-496c-bf07-0203910a05a6up+replayingreplaying, {"bytes_per_second":3241779.2,"bytes_per_snapshot":64835584.0,"last_snapshot_bytes":64835584,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726148184,"remote_snapshot_timestamp":1726148184,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:37:17image-primary6dcf3f6ae-ecc8-4d77-9308-bede00122fcdup+replayingreplaying, {"bytes_per_second":3414630.4,"bytes_per_snapshot":68292608.0,"last_snapshot_bytes":68292608,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726148184,"remote_snapshot_timestamp":1726148184,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:37:17image-primary7da6437a6-d0c6-4534-a0e0-60fb07f54f52up+replayingreplaying, {"bytes_per_second":3090841.6,"bytes_per_snapshot":61816832.0,"last_snapshot_bytes":61816832,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726148186,"remote_snapshot_timestamp":1726148186,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:37:17image-primary8b415829c-3d1c-41f6-a1aa-e76e40e110d1up+replayingreplaying, {"bytes_per_second":3153305.6,"bytes_per_snapshot":63066112.0,"last_snapshot_bytes":63066112,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726148182,"remote_snapshot_timestamp":1726148182,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:37:17image-primary91a8d4f4e-16a8-41c0-8b42-954a67b2e30dup+replayingreplaying, {"bytes_per_second":3334348.8,"bytes_per_snapshot":66686976.0,"last_snapshot_bytes":66686976,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726148188,"remote_snapshot_timestamp":1726148188,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:37:17' 2024-09-12T13:37:27.764 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:37:27.764 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary3'\'']/last_update' 2024-09-12T13:37:27.768 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:37:17' 2024-09-12T13:37:27.769 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary3'\'']/state' 2024-09-12T13:37:27.773 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+replaying 2024-09-12T13:37:27.773 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+replaying 2024-09-12T13:37:27.774 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+replaying 2024-09-12T13:37:27.776 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+replaying 2024-09-12T13:37:27.777 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:37:27.777 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:37:27.777 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ remove_image cluster2 mirror image-primary3 2024-09-12T13:37:27.777 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:37:27.777 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:37:27.777 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:37:27.777 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 snap purge mirror/image-primary3 2024-09-12T13:37:27.836 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 rm mirror/image-primary3 2024-09-12T13:37:27.978 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:37:27.977+0000 7f67e5200640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x5605557796c0 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-09-12T13:37:27.978 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:37:27.977+0000 7f67eca00640 -1 rbd::mirror::ImageReplayer: 0x5605573cac80 [3/9cc4db57-5517-41a1-8d14-519d4043d1a5] operator(): start failed: (85) Interrupted system call should be restarted 2024-09-12T13:37:29.497 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-09-12T13:37:29.503 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_image_removal cluster1 mirror image-primary3 2024-09-12T13:37:29.503 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:37:29.504 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:37:29.504 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:37:29.504 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:37:29.505 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 ls mirror 2024-09-12T13:37:29.505 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep -wq image-primary3 2024-09-12T13:37:29.558 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:37:29.558 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rm -rf test-primary3 2024-09-12T13:37:29.560 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:37:29.560 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ IMG=image-primary4 2024-09-12T13:37:29.560 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_replaying_status_in_pool_dir cluster1 mirror image-primary4 2024-09-12T13:37:29.560 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:37:29.561 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:37:29.561 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:37:29.561 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' snapshot = journal ']' 2024-09-12T13:37:29.561 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary4 up+replaying 2024-09-12T13:37:29.561 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:37:29.561 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:37:29.561 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:37:29.561 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:37:29.561 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:37:29.561 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:37:29.561 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:37:29.561 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:37:30.562 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary4 up+replaying '' '' 2024-09-12T13:37:30.563 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:37:30.563 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:37:30.563 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:37:30.563 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:37:30.563 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:37:30.563 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:37:30.563 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:37:30.563 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:37:30.563 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary4 2024-09-12T13:37:30.639 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary4: 2024-09-12T13:37:30.639 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 0203765c-0327-4045-b74a-e9bc90c8f4a0 2024-09-12T13:37:30.639 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+replaying 2024-09-12T13:37:30.639 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: replaying, {"bytes_per_second":3304448.0,"bytes_per_snapshot":66088960.0,"last_snapshot_bytes":66088960,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726148185,"remote_snapshot_timestamp":1726148185,"replay_state":"idle"} 2024-09-12T13:37:30.639 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:37:30.639 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:37:17' 2024-09-12T13:37:30.639 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+replaying' 2024-09-12T13:37:30.641 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+replaying 2024-09-12T13:37:30.642 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:37:30.644 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: replaying, {"bytes_per_second":3304448.0,"bytes_per_snapshot":66088960.0,"last_snapshot_bytes":66088960,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726148185,"remote_snapshot_timestamp":1726148185,"replay_state":"idle"} 2024-09-12T13:37:30.644 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:37:30.645 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+replaying 2024-09-12T13:37:30.645 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:37:30.647 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+replaying 2024-09-12T13:37:30.647 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:37:30.650 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:37:30.650 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:37:30.651 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:37:30.655 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:37:17' 2024-09-12T13:37:30.655 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary4 up+replaying '2024-09-12 13:37:17' 2024-09-12T13:37:30.655 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:37:30.655 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:37:30.655 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:37:30.655 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:37:30.655 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:37:17' 2024-09-12T13:37:30.655 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:37:30.656 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:37:30.656 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-09-12T13:37:30.753 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING741934232mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary107450998e-7967-4447-af66-572bd4fdded4up+replayingreplaying, {"bytes_per_second":3175424.0,"bytes_per_snapshot":63508480.0,"last_snapshot_bytes":63508480,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726148180,"remote_snapshot_timestamp":1726148180,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:37:17image-primary40203765c-0327-4045-b74a-e9bc90c8f4a0up+replayingreplaying, {"bytes_per_second":3304448.0,"bytes_per_snapshot":66088960.0,"last_snapshot_bytes":66088960,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726148185,"remote_snapshot_timestamp":1726148185,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:37:17image-primary595f21a09-f11d-496c-bf07-0203910a05a6up+replayingreplaying, {"bytes_per_second":3241779.2,"bytes_per_snapshot":64835584.0,"last_snapshot_bytes":64835584,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726148184,"remote_snapshot_timestamp":1726148184,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:37:17image-primary6dcf3f6ae-ecc8-4d77-9308-bede00122fcdup+replayingreplaying, {"bytes_per_second":3414630.4,"bytes_per_snapshot":68292608.0,"last_snapshot_bytes":68292608,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726148184,"remote_snapshot_timestamp":1726148184,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:37:17image-primary7da6437a6-d0c6-4534-a0e0-60fb07f54f52up+replayingreplaying, {"bytes_per_second":3090841.6,"bytes_per_snapshot":61816832.0,"last_snapshot_bytes":61816832,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726148186,"remote_snapshot_timestamp":1726148186,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:37:17image-primary8b415829c-3d1c-41f6-a1aa-e76e40e110d1up+replayingreplaying, {"bytes_per_second":3153305.6,"bytes_per_snapshot":63066112.0,"last_snapshot_bytes":63066112,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726148182,"remote_snapshot_timestamp":1726148182,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:37:17image-primary91a8d4f4e-16a8-41c0-8b42-954a67b2e30dup+replayingreplaying, {"bytes_per_second":3334348.8,"bytes_per_snapshot":66686976.0,"last_snapshot_bytes":66686976,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726148188,"remote_snapshot_timestamp":1726148188,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:37:17' 2024-09-12T13:37:30.753 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:37:30.754 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary4'\'']/last_update' 2024-09-12T13:37:30.758 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:37:17' 2024-09-12T13:37:30.758 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary4'\'']/state' 2024-09-12T13:37:30.762 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+replaying 2024-09-12T13:37:30.763 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+replaying 2024-09-12T13:37:30.763 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+replaying 2024-09-12T13:37:30.765 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+replaying 2024-09-12T13:37:30.766 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:37:30.766 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:37:30.766 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ remove_image cluster2 mirror image-primary4 2024-09-12T13:37:30.766 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:37:30.766 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:37:30.766 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:37:30.767 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 snap purge mirror/image-primary4 2024-09-12T13:37:30.828 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 rm mirror/image-primary4 2024-09-12T13:37:30.969 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:37:30.969+0000 7f67e5200640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x56055b8656c0 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-09-12T13:37:30.969 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:37:30.969+0000 7f67eca00640 -1 rbd::mirror::ImageReplayer: 0x56055c198500 [3/0203765c-0327-4045-b74a-e9bc90c8f4a0] operator(): start failed: (85) Interrupted system call should be restarted 2024-09-12T13:37:32.523 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-09-12T13:37:32.529 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_image_removal cluster1 mirror image-primary4 2024-09-12T13:37:32.530 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:37:32.530 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:37:32.530 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:37:32.530 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:37:32.530 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 ls mirror 2024-09-12T13:37:32.530 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep -wq image-primary4 2024-09-12T13:37:32.580 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:37:32.580 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rm -rf test-primary4 2024-09-12T13:37:32.582 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:37:32.583 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ IMG=image-primary5 2024-09-12T13:37:32.583 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_replaying_status_in_pool_dir cluster1 mirror image-primary5 2024-09-12T13:37:32.583 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:37:32.583 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:37:32.583 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:37:32.583 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' snapshot = journal ']' 2024-09-12T13:37:32.583 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary5 up+replaying 2024-09-12T13:37:32.583 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:37:32.583 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:37:32.583 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:37:32.583 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:37:32.583 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:37:32.583 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:37:32.584 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:37:32.584 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:37:33.585 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary5 up+replaying '' '' 2024-09-12T13:37:33.585 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:37:33.585 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:37:33.585 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:37:33.585 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:37:33.585 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:37:33.585 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:37:33.585 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:37:33.586 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:37:33.586 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary5 2024-09-12T13:37:33.664 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary5: 2024-09-12T13:37:33.664 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 95f21a09-f11d-496c-bf07-0203910a05a6 2024-09-12T13:37:33.664 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+replaying 2024-09-12T13:37:33.664 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: replaying, {"bytes_per_second":3241779.2,"bytes_per_snapshot":64835584.0,"last_snapshot_bytes":64835584,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726148184,"remote_snapshot_timestamp":1726148184,"replay_state":"idle"} 2024-09-12T13:37:33.664 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:37:33.664 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:37:17' 2024-09-12T13:37:33.665 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+replaying' 2024-09-12T13:37:33.666 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+replaying 2024-09-12T13:37:33.667 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:37:33.669 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: replaying, {"bytes_per_second":3241779.2,"bytes_per_snapshot":64835584.0,"last_snapshot_bytes":64835584,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726148184,"remote_snapshot_timestamp":1726148184,"replay_state":"idle"} 2024-09-12T13:37:33.670 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:37:33.670 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+replaying 2024-09-12T13:37:33.671 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:37:33.672 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+replaying 2024-09-12T13:37:33.673 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:37:33.675 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:37:33.676 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:37:33.676 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:37:33.680 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:37:17' 2024-09-12T13:37:33.681 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary5 up+replaying '2024-09-12 13:37:17' 2024-09-12T13:37:33.681 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:37:33.681 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:37:33.681 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:37:33.681 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:37:33.681 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:37:17' 2024-09-12T13:37:33.681 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:37:33.681 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:37:33.681 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-09-12T13:37:33.777 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING641934232mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary107450998e-7967-4447-af66-572bd4fdded4up+replayingreplaying, {"bytes_per_second":3175424.0,"bytes_per_snapshot":63508480.0,"last_snapshot_bytes":63508480,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726148180,"remote_snapshot_timestamp":1726148180,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:37:17image-primary595f21a09-f11d-496c-bf07-0203910a05a6up+replayingreplaying, {"bytes_per_second":3241779.2,"bytes_per_snapshot":64835584.0,"last_snapshot_bytes":64835584,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726148184,"remote_snapshot_timestamp":1726148184,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:37:17image-primary6dcf3f6ae-ecc8-4d77-9308-bede00122fcdup+replayingreplaying, {"bytes_per_second":3414630.4,"bytes_per_snapshot":68292608.0,"last_snapshot_bytes":68292608,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726148184,"remote_snapshot_timestamp":1726148184,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:37:17image-primary7da6437a6-d0c6-4534-a0e0-60fb07f54f52up+replayingreplaying, {"bytes_per_second":3090841.6,"bytes_per_snapshot":61816832.0,"last_snapshot_bytes":61816832,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726148186,"remote_snapshot_timestamp":1726148186,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:37:17image-primary8b415829c-3d1c-41f6-a1aa-e76e40e110d1up+replayingreplaying, {"bytes_per_second":3153305.6,"bytes_per_snapshot":63066112.0,"last_snapshot_bytes":63066112,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726148182,"remote_snapshot_timestamp":1726148182,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:37:17image-primary91a8d4f4e-16a8-41c0-8b42-954a67b2e30dup+replayingreplaying, {"bytes_per_second":3334348.8,"bytes_per_snapshot":66686976.0,"last_snapshot_bytes":66686976,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726148188,"remote_snapshot_timestamp":1726148188,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:37:17' 2024-09-12T13:37:33.777 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:37:33.778 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary5'\'']/last_update' 2024-09-12T13:37:33.781 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:37:17' 2024-09-12T13:37:33.782 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary5'\'']/state' 2024-09-12T13:37:33.786 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+replaying 2024-09-12T13:37:33.786 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+replaying 2024-09-12T13:37:33.787 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+replaying 2024-09-12T13:37:33.789 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+replaying 2024-09-12T13:37:33.789 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:37:33.790 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:37:33.790 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ remove_image cluster2 mirror image-primary5 2024-09-12T13:37:33.790 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:37:33.790 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:37:33.790 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:37:33.790 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 snap purge mirror/image-primary5 2024-09-12T13:37:33.854 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 rm mirror/image-primary5 2024-09-12T13:37:34.003 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:37:34.003+0000 7f67e4800640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x56055627bba0 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-09-12T13:37:34.003 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:37:34.003+0000 7f67eca00640 -1 rbd::mirror::ImageReplayer: 0x5605573caa00 [3/95f21a09-f11d-496c-bf07-0203910a05a6] operator(): start failed: (85) Interrupted system call should be restarted 2024-09-12T13:37:35.200 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-09-12T13:37:35.207 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_image_removal cluster1 mirror image-primary5 2024-09-12T13:37:35.207 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:37:35.207 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:37:35.207 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:37:35.207 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:37:35.207 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 ls mirror 2024-09-12T13:37:35.208 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep -wq image-primary5 2024-09-12T13:37:35.255 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:37:35.255 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rm -rf test-primary5 2024-09-12T13:37:35.258 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:37:35.258 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ IMG=image-primary6 2024-09-12T13:37:35.258 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_replaying_status_in_pool_dir cluster1 mirror image-primary6 2024-09-12T13:37:35.258 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:37:35.258 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:37:35.259 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:37:35.259 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' snapshot = journal ']' 2024-09-12T13:37:35.259 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary6 up+replaying 2024-09-12T13:37:35.259 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:37:35.259 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:37:35.259 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:37:35.259 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:37:35.259 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:37:35.259 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:37:35.259 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:37:35.259 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:37:36.261 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary6 up+replaying '' '' 2024-09-12T13:37:36.261 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:37:36.261 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:37:36.261 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:37:36.261 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:37:36.261 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:37:36.261 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:37:36.261 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:37:36.262 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:37:36.262 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary6 2024-09-12T13:37:36.339 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary6: 2024-09-12T13:37:36.339 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: dcf3f6ae-ecc8-4d77-9308-bede00122fcd 2024-09-12T13:37:36.339 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+replaying 2024-09-12T13:37:36.339 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: replaying, {"bytes_per_second":3414630.4,"bytes_per_snapshot":68292608.0,"last_snapshot_bytes":68292608,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726148184,"remote_snapshot_timestamp":1726148184,"replay_state":"idle"} 2024-09-12T13:37:36.339 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:37:36.339 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:37:17' 2024-09-12T13:37:36.339 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+replaying' 2024-09-12T13:37:36.344 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+replaying 2024-09-12T13:37:36.345 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:37:36.345 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: replaying, {"bytes_per_second":3414630.4,"bytes_per_snapshot":68292608.0,"last_snapshot_bytes":68292608,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726148184,"remote_snapshot_timestamp":1726148184,"replay_state":"idle"} 2024-09-12T13:37:36.345 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:37:36.346 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+replaying 2024-09-12T13:37:36.346 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:37:36.347 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+replaying 2024-09-12T13:37:36.347 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:37:36.350 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:37:36.350 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:37:36.351 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:37:36.355 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:37:17' 2024-09-12T13:37:36.355 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary6 up+replaying '2024-09-12 13:37:17' 2024-09-12T13:37:36.355 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:37:36.355 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:37:36.355 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:37:36.355 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:37:36.355 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:37:17' 2024-09-12T13:37:36.355 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:37:36.356 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:37:36.356 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-09-12T13:37:36.446 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING541934232mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary107450998e-7967-4447-af66-572bd4fdded4up+replayingreplaying, {"bytes_per_second":3175424.0,"bytes_per_snapshot":63508480.0,"last_snapshot_bytes":63508480,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726148180,"remote_snapshot_timestamp":1726148180,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:37:17image-primary6dcf3f6ae-ecc8-4d77-9308-bede00122fcdup+replayingreplaying, {"bytes_per_second":3414630.4,"bytes_per_snapshot":68292608.0,"last_snapshot_bytes":68292608,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726148184,"remote_snapshot_timestamp":1726148184,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:37:17image-primary7da6437a6-d0c6-4534-a0e0-60fb07f54f52up+replayingreplaying, {"bytes_per_second":3090841.6,"bytes_per_snapshot":61816832.0,"last_snapshot_bytes":61816832,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726148186,"remote_snapshot_timestamp":1726148186,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:37:17image-primary8b415829c-3d1c-41f6-a1aa-e76e40e110d1up+replayingreplaying, {"bytes_per_second":3153305.6,"bytes_per_snapshot":63066112.0,"last_snapshot_bytes":63066112,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726148182,"remote_snapshot_timestamp":1726148182,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:37:17image-primary91a8d4f4e-16a8-41c0-8b42-954a67b2e30dup+replayingreplaying, {"bytes_per_second":3334348.8,"bytes_per_snapshot":66686976.0,"last_snapshot_bytes":66686976,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726148188,"remote_snapshot_timestamp":1726148188,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:37:17' 2024-09-12T13:37:36.447 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:37:36.447 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary6'\'']/last_update' 2024-09-12T13:37:36.451 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:37:17' 2024-09-12T13:37:36.452 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary6'\'']/state' 2024-09-12T13:37:36.455 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+replaying 2024-09-12T13:37:36.457 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+replaying 2024-09-12T13:37:36.457 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+replaying 2024-09-12T13:37:36.458 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+replaying 2024-09-12T13:37:36.459 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:37:36.459 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:37:36.460 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ remove_image cluster2 mirror image-primary6 2024-09-12T13:37:36.460 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:37:36.460 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:37:36.460 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:37:36.460 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 snap purge mirror/image-primary6 2024-09-12T13:37:36.522 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 rm mirror/image-primary6 2024-09-12T13:37:36.662 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:37:36.661+0000 7f67e4800640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x56055c053860 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-09-12T13:37:36.662 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:37:36.661+0000 7f67eca00640 -1 rbd::mirror::ImageReplayer: 0x5605573cbb80 [3/dcf3f6ae-ecc8-4d77-9308-bede00122fcd] operator(): start failed: (85) Interrupted system call should be restarted 2024-09-12T13:37:38.520 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-09-12T13:37:38.526 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_image_removal cluster1 mirror image-primary6 2024-09-12T13:37:38.526 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:37:38.526 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:37:38.526 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:37:38.526 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:37:38.527 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 ls mirror 2024-09-12T13:37:38.527 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep -wq image-primary6 2024-09-12T13:37:38.581 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:37:38.581 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rm -rf test-primary6 2024-09-12T13:37:38.584 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:37:38.584 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ IMG=image-primary7 2024-09-12T13:37:38.584 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_replaying_status_in_pool_dir cluster1 mirror image-primary7 2024-09-12T13:37:38.584 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:37:38.584 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:37:38.584 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:37:38.584 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' snapshot = journal ']' 2024-09-12T13:37:38.584 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary7 up+replaying 2024-09-12T13:37:38.584 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:37:38.584 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:37:38.584 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:37:38.585 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:37:38.585 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:37:38.585 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:37:38.585 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:37:38.585 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:37:39.586 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary7 up+replaying '' '' 2024-09-12T13:37:39.586 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:37:39.586 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:37:39.586 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:37:39.586 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:37:39.587 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:37:39.587 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:37:39.587 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:37:39.587 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:37:39.587 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary7 2024-09-12T13:37:39.662 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary7: 2024-09-12T13:37:39.662 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: da6437a6-d0c6-4534-a0e0-60fb07f54f52 2024-09-12T13:37:39.662 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+replaying 2024-09-12T13:37:39.662 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: replaying, {"bytes_per_second":3090841.6,"bytes_per_snapshot":61816832.0,"last_snapshot_bytes":61816832,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726148186,"remote_snapshot_timestamp":1726148186,"replay_state":"idle"} 2024-09-12T13:37:39.663 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:37:39.663 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:37:17' 2024-09-12T13:37:39.663 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+replaying' 2024-09-12T13:37:39.664 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+replaying 2024-09-12T13:37:39.665 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:37:39.667 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: replaying, {"bytes_per_second":3090841.6,"bytes_per_snapshot":61816832.0,"last_snapshot_bytes":61816832,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726148186,"remote_snapshot_timestamp":1726148186,"replay_state":"idle"} 2024-09-12T13:37:39.668 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:37:39.668 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+replaying 2024-09-12T13:37:39.668 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:37:39.670 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+replaying 2024-09-12T13:37:39.670 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:37:39.673 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:37:39.673 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:37:39.674 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:37:39.678 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:37:17' 2024-09-12T13:37:39.678 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary7 up+replaying '2024-09-12 13:37:17' 2024-09-12T13:37:39.678 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:37:39.678 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:37:39.678 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:37:39.678 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:37:39.678 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:37:17' 2024-09-12T13:37:39.678 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:37:39.679 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:37:39.679 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-09-12T13:37:39.767 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING441934232mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary107450998e-7967-4447-af66-572bd4fdded4up+replayingreplaying, {"bytes_per_second":3175424.0,"bytes_per_snapshot":63508480.0,"last_snapshot_bytes":63508480,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726148180,"remote_snapshot_timestamp":1726148180,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:37:17image-primary7da6437a6-d0c6-4534-a0e0-60fb07f54f52up+replayingreplaying, {"bytes_per_second":3090841.6,"bytes_per_snapshot":61816832.0,"last_snapshot_bytes":61816832,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726148186,"remote_snapshot_timestamp":1726148186,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:37:17image-primary8b415829c-3d1c-41f6-a1aa-e76e40e110d1up+replayingreplaying, {"bytes_per_second":3153305.6,"bytes_per_snapshot":63066112.0,"last_snapshot_bytes":63066112,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726148182,"remote_snapshot_timestamp":1726148182,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:37:17image-primary91a8d4f4e-16a8-41c0-8b42-954a67b2e30dup+replayingreplaying, {"bytes_per_second":3334348.8,"bytes_per_snapshot":66686976.0,"last_snapshot_bytes":66686976,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726148188,"remote_snapshot_timestamp":1726148188,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:37:17' 2024-09-12T13:37:39.767 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:37:39.768 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary7'\'']/last_update' 2024-09-12T13:37:39.771 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:37:17' 2024-09-12T13:37:39.772 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary7'\'']/state' 2024-09-12T13:37:39.776 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+replaying 2024-09-12T13:37:39.777 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+replaying 2024-09-12T13:37:39.777 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+replaying 2024-09-12T13:37:39.779 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+replaying 2024-09-12T13:37:39.780 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:37:39.780 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:37:39.780 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ remove_image cluster2 mirror image-primary7 2024-09-12T13:37:39.780 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:37:39.780 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:37:39.780 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:37:39.780 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 snap purge mirror/image-primary7 2024-09-12T13:37:39.840 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 rm mirror/image-primary7 2024-09-12T13:37:39.983 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:37:39.982+0000 7f67e5200640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x56055c053040 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-09-12T13:37:39.983 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:37:39.982+0000 7f67eca00640 -1 rbd::mirror::ImageReplayer: 0x5605573cb180 [3/da6437a6-d0c6-4534-a0e0-60fb07f54f52] operator(): start failed: (85) Interrupted system call should be restarted 2024-09-12T13:37:41.200 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-09-12T13:37:41.206 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_image_removal cluster1 mirror image-primary7 2024-09-12T13:37:41.207 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:37:41.207 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:37:41.207 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:37:41.207 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:37:41.207 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 ls mirror 2024-09-12T13:37:41.207 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep -wq image-primary7 2024-09-12T13:37:41.258 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:37:41.258 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rm -rf test-primary7 2024-09-12T13:37:41.260 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:37:41.260 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ IMG=image-primary8 2024-09-12T13:37:41.261 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_replaying_status_in_pool_dir cluster1 mirror image-primary8 2024-09-12T13:37:41.261 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:37:41.261 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:37:41.261 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:37:41.261 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' snapshot = journal ']' 2024-09-12T13:37:41.261 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary8 up+replaying 2024-09-12T13:37:41.261 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:37:41.261 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:37:41.261 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:37:41.261 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:37:41.261 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:37:41.261 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:37:41.261 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:37:41.262 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:37:42.263 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary8 up+replaying '' '' 2024-09-12T13:37:42.263 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:37:42.263 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:37:42.263 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:37:42.263 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:37:42.263 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:37:42.263 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:37:42.263 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:37:42.264 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:37:42.264 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary8 2024-09-12T13:37:42.339 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary8: 2024-09-12T13:37:42.339 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: b415829c-3d1c-41f6-a1aa-e76e40e110d1 2024-09-12T13:37:42.339 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+replaying 2024-09-12T13:37:42.340 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: replaying, {"bytes_per_second":3153305.6,"bytes_per_snapshot":63066112.0,"last_snapshot_bytes":63066112,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726148182,"remote_snapshot_timestamp":1726148182,"replay_state":"idle"} 2024-09-12T13:37:42.340 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:37:42.340 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:37:17' 2024-09-12T13:37:42.340 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+replaying' 2024-09-12T13:37:42.342 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+replaying 2024-09-12T13:37:42.342 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:37:42.345 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: replaying, {"bytes_per_second":3153305.6,"bytes_per_snapshot":63066112.0,"last_snapshot_bytes":63066112,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726148182,"remote_snapshot_timestamp":1726148182,"replay_state":"idle"} 2024-09-12T13:37:42.345 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:37:42.346 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+replaying 2024-09-12T13:37:42.346 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:37:42.348 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+replaying 2024-09-12T13:37:42.348 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:37:42.351 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:37:42.351 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:37:42.352 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:37:42.356 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:37:17' 2024-09-12T13:37:42.356 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary8 up+replaying '2024-09-12 13:37:17' 2024-09-12T13:37:42.356 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:37:42.356 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:37:42.356 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:37:42.356 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:37:42.356 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:37:17' 2024-09-12T13:37:42.356 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:37:42.357 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:37:42.357 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-09-12T13:37:42.442 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING341934232mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary107450998e-7967-4447-af66-572bd4fdded4up+replayingreplaying, {"bytes_per_second":3175424.0,"bytes_per_snapshot":63508480.0,"last_snapshot_bytes":63508480,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726148180,"remote_snapshot_timestamp":1726148180,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:37:17image-primary8b415829c-3d1c-41f6-a1aa-e76e40e110d1up+replayingreplaying, {"bytes_per_second":3153305.6,"bytes_per_snapshot":63066112.0,"last_snapshot_bytes":63066112,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726148182,"remote_snapshot_timestamp":1726148182,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:37:17image-primary91a8d4f4e-16a8-41c0-8b42-954a67b2e30dup+replayingreplaying, {"bytes_per_second":3334348.8,"bytes_per_snapshot":66686976.0,"last_snapshot_bytes":66686976,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726148188,"remote_snapshot_timestamp":1726148188,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:37:17' 2024-09-12T13:37:42.442 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:37:42.443 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary8'\'']/last_update' 2024-09-12T13:37:42.447 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:37:17' 2024-09-12T13:37:42.447 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary8'\'']/state' 2024-09-12T13:37:42.451 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+replaying 2024-09-12T13:37:42.452 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+replaying 2024-09-12T13:37:42.452 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+replaying 2024-09-12T13:37:42.454 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+replaying 2024-09-12T13:37:42.455 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:37:42.455 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:37:42.455 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ remove_image cluster2 mirror image-primary8 2024-09-12T13:37:42.455 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:37:42.455 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:37:42.455 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:37:42.455 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 snap purge mirror/image-primary8 2024-09-12T13:37:42.515 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 rm mirror/image-primary8 2024-09-12T13:37:42.660 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:37:42.660+0000 7f67e4800640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x560556582680 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-09-12T13:37:42.660 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:37:42.660+0000 7f67ed400640 -1 rbd::mirror::ImageReplayer: 0x5605561ad400 [3/b415829c-3d1c-41f6-a1aa-e76e40e110d1] operator(): start failed: (85) Interrupted system call should be restarted 2024-09-12T13:37:44.213 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-09-12T13:37:44.220 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_image_removal cluster1 mirror image-primary8 2024-09-12T13:37:44.220 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:37:44.220 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:37:44.220 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:37:44.220 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:37:44.220 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 ls mirror 2024-09-12T13:37:44.221 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep -wq image-primary8 2024-09-12T13:37:44.270 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:37:44.270 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rm -rf test-primary8 2024-09-12T13:37:44.273 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:37:44.273 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ IMG=image-primary9 2024-09-12T13:37:44.273 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_replaying_status_in_pool_dir cluster1 mirror image-primary9 2024-09-12T13:37:44.273 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:37:44.273 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:37:44.273 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:37:44.273 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' snapshot = journal ']' 2024-09-12T13:37:44.273 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary9 up+replaying 2024-09-12T13:37:44.273 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:37:44.273 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:37:44.273 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:37:44.273 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:37:44.273 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:37:44.274 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:37:44.274 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:37:44.274 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:37:45.275 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary9 up+replaying '' '' 2024-09-12T13:37:45.275 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:37:45.275 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:37:45.275 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:37:45.275 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:37:45.276 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:37:45.276 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:37:45.276 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:37:45.276 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:37:45.276 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary9 2024-09-12T13:37:45.364 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary9: 2024-09-12T13:37:45.364 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 1a8d4f4e-16a8-41c0-8b42-954a67b2e30d 2024-09-12T13:37:45.364 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+replaying 2024-09-12T13:37:45.364 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: replaying, {"bytes_per_second":3334348.8,"bytes_per_snapshot":66686976.0,"last_snapshot_bytes":66686976,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726148188,"remote_snapshot_timestamp":1726148188,"replay_state":"idle"} 2024-09-12T13:37:45.365 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:37:45.365 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:37:17' 2024-09-12T13:37:45.365 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+replaying' 2024-09-12T13:37:45.367 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+replaying 2024-09-12T13:37:45.367 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:37:45.370 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: replaying, {"bytes_per_second":3334348.8,"bytes_per_snapshot":66686976.0,"last_snapshot_bytes":66686976,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726148188,"remote_snapshot_timestamp":1726148188,"replay_state":"idle"} 2024-09-12T13:37:45.370 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:37:45.371 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+replaying 2024-09-12T13:37:45.371 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:37:45.373 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+replaying 2024-09-12T13:37:45.373 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:37:45.376 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:37:45.376 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:37:45.377 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:37:45.381 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:37:17' 2024-09-12T13:37:45.381 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary9 up+replaying '2024-09-12 13:37:17' 2024-09-12T13:37:45.381 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:37:45.381 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:37:45.381 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:37:45.381 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:37:45.382 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:37:17' 2024-09-12T13:37:45.382 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:37:45.382 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:37:45.382 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-09-12T13:37:45.467 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING241934232mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary107450998e-7967-4447-af66-572bd4fdded4up+replayingreplaying, {"bytes_per_second":3175424.0,"bytes_per_snapshot":63508480.0,"last_snapshot_bytes":63508480,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726148180,"remote_snapshot_timestamp":1726148180,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:37:17image-primary91a8d4f4e-16a8-41c0-8b42-954a67b2e30dup+replayingreplaying, {"bytes_per_second":3334348.8,"bytes_per_snapshot":66686976.0,"last_snapshot_bytes":66686976,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726148188,"remote_snapshot_timestamp":1726148188,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:37:17' 2024-09-12T13:37:45.467 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:37:45.467 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary9'\'']/last_update' 2024-09-12T13:37:45.471 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:37:17' 2024-09-12T13:37:45.471 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary9'\'']/state' 2024-09-12T13:37:45.475 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+replaying 2024-09-12T13:37:45.476 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+replaying 2024-09-12T13:37:45.476 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+replaying 2024-09-12T13:37:45.478 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+replaying 2024-09-12T13:37:45.479 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:37:45.479 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:37:45.479 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ remove_image cluster2 mirror image-primary9 2024-09-12T13:37:45.479 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:37:45.479 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:37:45.480 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:37:45.480 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 snap purge mirror/image-primary9 2024-09-12T13:37:45.539 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 rm mirror/image-primary9 2024-09-12T13:37:45.680 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:37:45.680+0000 7f67e5200640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x56055bf60b60 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-09-12T13:37:45.680 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:37:45.680+0000 7f67ed400640 -1 rbd::mirror::ImageReplayer: 0x560556cef180 [3/1a8d4f4e-16a8-41c0-8b42-954a67b2e30d] operator(): start failed: (85) Interrupted system call should be restarted 2024-09-12T13:37:47.213 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-09-12T13:37:47.219 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_image_removal cluster1 mirror image-primary9 2024-09-12T13:37:47.219 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:37:47.219 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:37:47.219 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:37:47.219 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:37:47.220 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 ls mirror 2024-09-12T13:37:47.220 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep -wq image-primary9 2024-09-12T13:37:47.275 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:37:47.275 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rm -rf test-primary9 2024-09-12T13:37:47.278 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:37:47.278 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ IMG=image-primary10 2024-09-12T13:37:47.278 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_replaying_status_in_pool_dir cluster1 mirror image-primary10 2024-09-12T13:37:47.278 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:37:47.278 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:37:47.278 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:37:47.278 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' snapshot = journal ']' 2024-09-12T13:37:47.278 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary10 up+replaying 2024-09-12T13:37:47.278 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:37:47.278 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:37:47.279 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:37:47.279 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:37:47.279 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:37:47.279 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:37:47.279 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:37:47.279 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:37:48.280 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary10 up+replaying '' '' 2024-09-12T13:37:48.280 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:37:48.280 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:37:48.280 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:37:48.281 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:37:48.281 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:37:48.281 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:37:48.281 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:37:48.281 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:37:48.281 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary10 2024-09-12T13:37:48.353 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary10: 2024-09-12T13:37:48.354 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 7450998e-7967-4447-af66-572bd4fdded4 2024-09-12T13:37:48.354 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+replaying 2024-09-12T13:37:48.354 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: replaying, {"bytes_per_second":0.0,"bytes_per_snapshot":63508480.0,"last_snapshot_bytes":63508480,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726148180,"remote_snapshot_timestamp":1726148180,"replay_state":"idle"} 2024-09-12T13:37:48.354 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:37:48.354 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:37:47' 2024-09-12T13:37:48.354 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+replaying' 2024-09-12T13:37:48.356 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+replaying 2024-09-12T13:37:48.356 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:37:48.359 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: replaying, {"bytes_per_second":0.0,"bytes_per_snapshot":63508480.0,"last_snapshot_bytes":63508480,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726148180,"remote_snapshot_timestamp":1726148180,"replay_state":"idle"} 2024-09-12T13:37:48.359 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:37:48.360 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+replaying 2024-09-12T13:37:48.360 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:37:48.362 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+replaying 2024-09-12T13:37:48.362 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:37:48.365 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:37:48.366 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:37:48.366 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:37:48.370 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:37:47' 2024-09-12T13:37:48.370 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary10 up+replaying '2024-09-12 13:37:47' 2024-09-12T13:37:48.370 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:37:48.370 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:37:48.370 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:37:48.370 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:37:48.370 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:37:47' 2024-09-12T13:37:48.370 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:37:48.371 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:37:48.371 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-09-12T13:37:48.447 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING141934232mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary107450998e-7967-4447-af66-572bd4fdded4up+replayingreplaying, {"bytes_per_second":0.0,"bytes_per_snapshot":63508480.0,"last_snapshot_bytes":63508480,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726148180,"remote_snapshot_timestamp":1726148180,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:37:47' 2024-09-12T13:37:48.447 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:37:48.447 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary10'\'']/last_update' 2024-09-12T13:37:48.451 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:37:47' 2024-09-12T13:37:48.452 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary10'\'']/state' 2024-09-12T13:37:48.455 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+replaying 2024-09-12T13:37:48.456 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+replaying 2024-09-12T13:37:48.456 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+replaying 2024-09-12T13:37:48.458 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+replaying 2024-09-12T13:37:48.459 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:37:48.459 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:37:48.459 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ remove_image cluster2 mirror image-primary10 2024-09-12T13:37:48.459 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:37:48.459 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:37:48.459 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:37:48.460 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 snap purge mirror/image-primary10 2024-09-12T13:37:48.519 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 rm mirror/image-primary10 2024-09-12T13:37:48.651 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:37:48.651+0000 7f67e5200640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x56055b8ca680 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-09-12T13:37:48.651 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:37:48.651+0000 7f67eca00640 -1 rbd::mirror::ImageReplayer: 0x5605561ac500 [3/7450998e-7967-4447-af66-572bd4fdded4] operator(): start failed: (85) Interrupted system call should be restarted 2024-09-12T13:37:50.234 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-09-12T13:37:50.240 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_image_removal cluster1 mirror image-primary10 2024-09-12T13:37:50.240 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:37:50.241 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:37:50.241 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:37:50.241 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:37:50.241 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 ls mirror 2024-09-12T13:37:50.241 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep -wq image-primary10 2024-09-12T13:37:50.292 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:37:50.292 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rm -rf test-primary10 2024-09-12T13:37:50.295 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..10} 2024-09-12T13:37:50.295 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEVS=() 2024-09-12T13:37:50.295 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ SNAP_PIDS=() 2024-09-12T13:37:50.295 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ COMPARE_PIDS=() 2024-09-12T13:37:50.295 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ WORKLOAD_PIDS=() 2024-09-12T13:37:50.295 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ RET=0 2024-09-12T13:37:50.295 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:37:50.295 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ IMG=image-primary1 2024-09-12T13:37:50.296 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ MNTPT=test-primary1 2024-09-12T13:37:50.296 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ create_image_and_enable_mirror cluster1 mirror image-primary1 snapshot 10G 2024-09-12T13:37:50.296 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:37:50.296 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:37:50.296 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:37:50.296 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:37:50.296 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:37:50.296 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:37:50.296 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mode=snapshot 2024-09-12T13:37:50.296 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n snapshot ']' 2024-09-12T13:37:50.296 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:37:50.296 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ create_image cluster1 mirror image-primary1 10G 2024-09-12T13:37:50.297 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:37:50.297 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:37:50.297 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:37:50.297 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:37:50.297 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:37:50.297 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:37:50.297 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local size=128 2024-09-12T13:37:50.297 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n 10G ']' 2024-09-12T13:37:50.297 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ size=10G 2024-09-12T13:37:50.297 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:37:50.297 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 create --size 10G --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image-primary1 2024-09-12T13:37:50.369 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ is_pool_mirror_mode_image mirror 2024-09-12T13:37:50.369 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:37:50.369 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' image = image ']' 2024-09-12T13:37:50.369 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:37:50.369 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ enable_mirror cluster1 mirror image-primary1 snapshot 2024-09-12T13:37:50.369 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:37:50.369 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:37:50.369 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:37:50.369 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mode=snapshot 2024-09-12T13:37:50.369 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 mirror image enable mirror/image-primary1 snapshot 2024-09-12T13:37:51.154 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Mirroring enabled 2024-09-12T13:37:51.170 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 info mirror/image-primary1 2024-09-12T13:37:51.227 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:rbd image 'image-primary1': 2024-09-12T13:37:51.228 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: size 10 GiB in 2560 objects 2024-09-12T13:37:51.228 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: order 22 (4 MiB objects) 2024-09-12T13:37:51.228 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: snapshot_count: 1 2024-09-12T13:37:51.228 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: id: 2954ef0b9e 2024-09-12T13:37:51.228 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: block_name_prefix: rbd_data.2954ef0b9e 2024-09-12T13:37:51.228 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: format: 2 2024-09-12T13:37:51.228 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: features: layering, exclusive-lock, object-map, fast-diff 2024-09-12T13:37:51.228 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: op_features: 2024-09-12T13:37:51.228 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: flags: 2024-09-12T13:37:51.228 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: create_timestamp: Thu Sep 12 13:37:50 2024 2024-09-12T13:37:51.228 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: access_timestamp: Thu Sep 12 13:37:50 2024 2024-09-12T13:37:51.229 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: modify_timestamp: Thu Sep 12 13:37:50 2024 2024-09-12T13:37:51.229 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring state: enabled 2024-09-12T13:37:51.229 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring mode: snapshot 2024-09-12T13:37:51.229 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring global id: 99b2aeba-005c-4935-8fc2-c4175a9a85c1 2024-09-12T13:37:51.229 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring primary: true 2024-09-12T13:37:51.233 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \n\b\d ]] 2024-09-12T13:37:51.233 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \k\r\b\d ]] 2024-09-12T13:37:51.234 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo rbd --cluster cluster1 device map -t krbd mirror/image-primary1 2024-09-12T13:37:51.370 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEV=/dev/rbd0 2024-09-12T13:37:51.371 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEVS+=($DEV) 2024-09-12T13:37:51.371 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo mkfs.ext4 /dev/rbd0 2024-09-12T13:37:51.423 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:mke2fs 1.46.5 (30-Dec-2021) 2024-09-12T13:37:51.436 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Discarding device blocks: 0/2621440 done 2024-09-12T13:37:51.436 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Creating filesystem with 2621440 4k blocks and 655360 inodes 2024-09-12T13:37:51.436 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Filesystem UUID: bf22f348-e8a1-4077-9138-574774065b45 2024-09-12T13:37:51.436 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Superblock backups stored on blocks: 2024-09-12T13:37:51.436 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 2024-09-12T13:37:51.436 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 2024-09-12T13:37:51.436 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Allocating group tables: 0/80 done 2024-09-12T13:37:51.436 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Writing inode tables: 0/80 done 2024-09-12T13:37:51.441 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Creating journal (16384 blocks): done 2024-09-12T13:37:51.500 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Writing superblocks and filesystem accounting information: 0/80 done 2024-09-12T13:37:51.501 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 2024-09-12T13:37:51.504 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mkdir test-primary1 2024-09-12T13:37:51.515 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo mount /dev/rbd0 test-primary1 2024-09-12T13:37:51.571 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ whoami 2024-09-12T13:37:51.575 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo chown ubuntu test-primary1 2024-09-12T13:37:51.607 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ SNAP_PIDS+=($!) 2024-09-12T13:37:51.607 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ take_mirror_snapshots cluster1 mirror image-primary1 2024-09-12T13:37:51.607 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:37:51.608 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:37:51.608 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:37:51.608 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:37:51.608 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:37:51.608 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ WORKLOAD_PIDS+=($!) 2024-09-12T13:37:51.608 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:37:51.608 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:37:51.608 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:37:51.608 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:37:51.608 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:37:51.608 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ IMG=image-primary2 2024-09-12T13:37:51.608 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ slow_untar_workload test-primary1 2024-09-12T13:37:51.608 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ MNTPT=test-primary2 2024-09-12T13:37:51.609 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ create_image_and_enable_mirror cluster1 mirror image-primary2 snapshot 10G 2024-09-12T13:37:51.609 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:37:51.609 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:37:51.609 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:37:51.609 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:37:51.609 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mountpt=test-primary1 2024-09-12T13:37:51.609 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:37:51.609 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:37:51.609 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mode=snapshot 2024-09-12T13:37:51.609 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n snapshot ']' 2024-09-12T13:37:51.609 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ cp linux-5.4.tar.gz test-primary1 2024-09-12T13:37:51.609 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:37:51.609 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ create_image cluster1 mirror image-primary2 10G 2024-09-12T13:37:51.610 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:37:51.610 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:37:51.610 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:37:51.610 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:37:51.610 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:37:51.610 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:37:51.610 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local size=128 2024-09-12T13:37:51.610 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n 10G ']' 2024-09-12T13:37:51.610 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ size=10G 2024-09-12T13:37:51.610 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:37:51.610 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 create --size 10G --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image-primary2 2024-09-12T13:37:51.680 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ is_pool_mirror_mode_image mirror 2024-09-12T13:37:51.680 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:37:51.680 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' image = image ']' 2024-09-12T13:37:51.680 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:37:51.680 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ enable_mirror cluster1 mirror image-primary2 snapshot 2024-09-12T13:37:51.680 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:37:51.681 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:37:51.681 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:37:51.681 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mode=snapshot 2024-09-12T13:37:51.681 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 mirror image enable mirror/image-primary2 snapshot 2024-09-12T13:37:51.895 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local ret=0 2024-09-12T13:37:51.895 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ timeout 5m bash -c 'zcat test-primary1/linux-5.4.tar.gz | pv -L 256K | tar xf - -C test-primary1' 2024-09-12T13:37:52.157 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Mirroring enabled 2024-09-12T13:37:52.160 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1654 2024-09-12T13:37:52.171 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:37:52.174 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 info mirror/image-primary2 2024-09-12T13:37:52.229 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:rbd image 'image-primary2': 2024-09-12T13:37:52.229 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: size 10 GiB in 2560 objects 2024-09-12T13:37:52.229 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: order 22 (4 MiB objects) 2024-09-12T13:37:52.229 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: snapshot_count: 1 2024-09-12T13:37:52.229 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: id: 2964ad6c9b 2024-09-12T13:37:52.230 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: block_name_prefix: rbd_data.2964ad6c9b 2024-09-12T13:37:52.230 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: format: 2 2024-09-12T13:37:52.230 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: features: layering, exclusive-lock, object-map, fast-diff 2024-09-12T13:37:52.230 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: op_features: 2024-09-12T13:37:52.230 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: flags: 2024-09-12T13:37:52.230 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: create_timestamp: Thu Sep 12 13:37:51 2024 2024-09-12T13:37:52.230 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: access_timestamp: Thu Sep 12 13:37:51 2024 2024-09-12T13:37:52.230 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: modify_timestamp: Thu Sep 12 13:37:51 2024 2024-09-12T13:37:52.230 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring state: enabled 2024-09-12T13:37:52.230 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring mode: snapshot 2024-09-12T13:37:52.230 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring global id: 603c11d9-89c3-4bd2-8c39-0d4ed9ed3093 2024-09-12T13:37:52.230 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring primary: true 2024-09-12T13:37:52.234 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \n\b\d ]] 2024-09-12T13:37:52.234 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \k\r\b\d ]] 2024-09-12T13:37:52.235 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo rbd --cluster cluster1 device map -t krbd mirror/image-primary2 2024-09-12T13:37:52.354 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEV=/dev/rbd1 2024-09-12T13:37:52.355 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEVS+=($DEV) 2024-09-12T13:37:52.355 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo mkfs.ext4 /dev/rbd1 2024-09-12T13:37:52.375 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:mke2fs 1.46.5 (30-Dec-2021) 2024-09-12T13:37:52.387 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Discarding device blocks: 0/2621440 done 2024-09-12T13:37:52.387 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Creating filesystem with 2621440 4k blocks and 655360 inodes 2024-09-12T13:37:52.387 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Filesystem UUID: b9e26b56-433c-4981-b586-12b8a670189d 2024-09-12T13:37:52.387 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Superblock backups stored on blocks: 2024-09-12T13:37:52.387 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 2024-09-12T13:37:52.387 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 2024-09-12T13:37:52.387 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Allocating group tables: 0/80 done 2024-09-12T13:37:52.387 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Writing inode tables: 0/80 done 2024-09-12T13:37:52.392 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Creating journal (16384 blocks): done 2024-09-12T13:37:52.451 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Writing superblocks and filesystem accounting information: 0/80 done 2024-09-12T13:37:52.451 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 2024-09-12T13:37:52.454 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mkdir test-primary2 2024-09-12T13:37:52.457 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo mount /dev/rbd1 test-primary2 2024-09-12T13:37:52.512 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ whoami 2024-09-12T13:37:52.514 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo chown ubuntu test-primary2 2024-09-12T13:37:52.538 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ SNAP_PIDS+=($!) 2024-09-12T13:37:52.538 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ take_mirror_snapshots cluster1 mirror image-primary2 2024-09-12T13:37:52.538 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:37:52.538 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:37:52.539 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:37:52.539 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:37:52.539 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:37:52.539 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:37:52.539 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:37:52.539 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:37:52.539 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ WORKLOAD_PIDS+=($!) 2024-09-12T13:37:52.539 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:37:52.539 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:37:52.539 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ IMG=image-primary3 2024-09-12T13:37:52.539 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ MNTPT=test-primary3 2024-09-12T13:37:52.539 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ create_image_and_enable_mirror cluster1 mirror image-primary3 snapshot 10G 2024-09-12T13:37:52.539 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ slow_untar_workload test-primary2 2024-09-12T13:37:52.540 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:37:52.540 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:37:52.540 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:37:52.540 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:37:52.540 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:37:52.540 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:37:52.540 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mode=snapshot 2024-09-12T13:37:52.540 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n snapshot ']' 2024-09-12T13:37:52.540 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:37:52.540 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ create_image cluster1 mirror image-primary3 10G 2024-09-12T13:37:52.540 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mountpt=test-primary2 2024-09-12T13:37:52.540 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:37:52.541 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:37:52.541 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:37:52.541 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:37:52.541 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:37:52.541 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:37:52.541 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local size=128 2024-09-12T13:37:52.541 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n 10G ']' 2024-09-12T13:37:52.541 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ size=10G 2024-09-12T13:37:52.541 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:37:52.541 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ cp linux-5.4.tar.gz test-primary2 2024-09-12T13:37:52.541 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 create --size 10G --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image-primary3 2024-09-12T13:37:52.609 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ is_pool_mirror_mode_image mirror 2024-09-12T13:37:52.609 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:37:52.609 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' image = image ']' 2024-09-12T13:37:52.609 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:37:52.609 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ enable_mirror cluster1 mirror image-primary3 snapshot 2024-09-12T13:37:52.609 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:37:52.609 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:37:52.609 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:37:52.609 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mode=snapshot 2024-09-12T13:37:52.610 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 mirror image enable mirror/image-primary3 snapshot 2024-09-12T13:37:52.831 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local ret=0 2024-09-12T13:37:52.831 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ timeout 5m bash -c 'zcat test-primary2/linux-5.4.tar.gz | pv -L 256K | tar xf - -C test-primary2' 2024-09-12T13:37:53.143 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Mirroring enabled 2024-09-12T13:37:53.152 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1656 2024-09-12T13:37:53.160 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 info mirror/image-primary3 2024-09-12T13:37:53.163 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:37:53.219 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:rbd image 'image-primary3': 2024-09-12T13:37:53.219 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: size 10 GiB in 2560 objects 2024-09-12T13:37:53.219 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: order 22 (4 MiB objects) 2024-09-12T13:37:53.219 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: snapshot_count: 1 2024-09-12T13:37:53.219 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: id: 2973d767a4df 2024-09-12T13:37:53.219 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: block_name_prefix: rbd_data.2973d767a4df 2024-09-12T13:37:53.219 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: format: 2 2024-09-12T13:37:53.219 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: features: layering, exclusive-lock, object-map, fast-diff 2024-09-12T13:37:53.219 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: op_features: 2024-09-12T13:37:53.220 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: flags: 2024-09-12T13:37:53.220 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: create_timestamp: Thu Sep 12 13:37:52 2024 2024-09-12T13:37:53.220 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: access_timestamp: Thu Sep 12 13:37:52 2024 2024-09-12T13:37:53.220 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: modify_timestamp: Thu Sep 12 13:37:52 2024 2024-09-12T13:37:53.220 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring state: enabled 2024-09-12T13:37:53.220 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring mode: snapshot 2024-09-12T13:37:53.220 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring global id: 6eed8ffb-1fae-4e8b-9794-cf7af72eafbc 2024-09-12T13:37:53.220 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring primary: true 2024-09-12T13:37:53.224 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \n\b\d ]] 2024-09-12T13:37:53.224 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \k\r\b\d ]] 2024-09-12T13:37:53.225 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo rbd --cluster cluster1 device map -t krbd mirror/image-primary3 2024-09-12T13:37:53.348 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEV=/dev/rbd2 2024-09-12T13:37:53.348 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEVS+=($DEV) 2024-09-12T13:37:53.348 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo mkfs.ext4 /dev/rbd2 2024-09-12T13:37:53.368 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:mke2fs 1.46.5 (30-Dec-2021) 2024-09-12T13:37:53.381 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Discarding device blocks: 0/2621440 done 2024-09-12T13:37:53.381 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Creating filesystem with 2621440 4k blocks and 655360 inodes 2024-09-12T13:37:53.381 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Filesystem UUID: fdb213dd-8353-46fc-b8c3-d27ece60d2e2 2024-09-12T13:37:53.381 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Superblock backups stored on blocks: 2024-09-12T13:37:53.382 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 2024-09-12T13:37:53.382 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 2024-09-12T13:37:53.382 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Allocating group tables: 0/80 done 2024-09-12T13:37:53.382 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Writing inode tables: 0/80 done 2024-09-12T13:37:53.385 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Creating journal (16384 blocks): done 2024-09-12T13:37:53.440 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Writing superblocks and filesystem accounting information: 0/80 done 2024-09-12T13:37:53.441 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 2024-09-12T13:37:53.443 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mkdir test-primary3 2024-09-12T13:37:53.446 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo mount /dev/rbd2 test-primary3 2024-09-12T13:37:53.503 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ whoami 2024-09-12T13:37:53.505 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo chown ubuntu test-primary3 2024-09-12T13:37:53.529 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ SNAP_PIDS+=($!) 2024-09-12T13:37:53.529 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ take_mirror_snapshots cluster1 mirror image-primary3 2024-09-12T13:37:53.531 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:37:53.531 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:37:53.531 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:37:53.531 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:37:53.531 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:37:53.531 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:37:53.531 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:37:53.532 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:37:53.532 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ WORKLOAD_PIDS+=($!) 2024-09-12T13:37:53.532 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:37:53.532 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:37:53.532 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ IMG=image-primary4 2024-09-12T13:37:53.532 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ MNTPT=test-primary4 2024-09-12T13:37:53.532 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ create_image_and_enable_mirror cluster1 mirror image-primary4 snapshot 10G 2024-09-12T13:37:53.532 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ slow_untar_workload test-primary3 2024-09-12T13:37:53.532 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:37:53.532 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:37:53.532 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:37:53.532 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:37:53.533 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:37:53.533 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:37:53.533 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mode=snapshot 2024-09-12T13:37:53.533 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n snapshot ']' 2024-09-12T13:37:53.533 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:37:53.533 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ create_image cluster1 mirror image-primary4 10G 2024-09-12T13:37:53.533 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:37:53.533 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:37:53.533 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:37:53.533 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:37:53.533 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:37:53.534 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mountpt=test-primary3 2024-09-12T13:37:53.534 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:37:53.534 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local size=128 2024-09-12T13:37:53.534 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n 10G ']' 2024-09-12T13:37:53.534 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ size=10G 2024-09-12T13:37:53.534 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:37:53.534 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ cp linux-5.4.tar.gz test-primary3 2024-09-12T13:37:53.534 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 create --size 10G --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image-primary4 2024-09-12T13:37:53.601 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ is_pool_mirror_mode_image mirror 2024-09-12T13:37:53.601 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:37:53.601 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' image = image ']' 2024-09-12T13:37:53.601 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:37:53.601 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ enable_mirror cluster1 mirror image-primary4 snapshot 2024-09-12T13:37:53.602 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:37:53.602 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:37:53.602 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:37:53.602 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mode=snapshot 2024-09-12T13:37:53.602 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 mirror image enable mirror/image-primary4 snapshot 2024-09-12T13:37:53.816 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local ret=0 2024-09-12T13:37:53.816 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ timeout 5m bash -c 'zcat test-primary3/linux-5.4.tar.gz | pv -L 256K | tar xf - -C test-primary3' 2024-09-12T13:37:54.177 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Mirroring enabled 2024-09-12T13:37:54.182 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1658 2024-09-12T13:37:54.193 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:37:54.194 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 info mirror/image-primary4 2024-09-12T13:37:54.257 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:rbd image 'image-primary4': 2024-09-12T13:37:54.257 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: size 10 GiB in 2560 objects 2024-09-12T13:37:54.257 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: order 22 (4 MiB objects) 2024-09-12T13:37:54.257 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: snapshot_count: 1 2024-09-12T13:37:54.257 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: id: 2982dbb1bdec 2024-09-12T13:37:54.257 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: block_name_prefix: rbd_data.2982dbb1bdec 2024-09-12T13:37:54.257 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: format: 2 2024-09-12T13:37:54.257 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: features: layering, exclusive-lock, object-map, fast-diff 2024-09-12T13:37:54.257 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: op_features: 2024-09-12T13:37:54.258 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: flags: 2024-09-12T13:37:54.258 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: create_timestamp: Thu Sep 12 13:37:53 2024 2024-09-12T13:37:54.258 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: access_timestamp: Thu Sep 12 13:37:53 2024 2024-09-12T13:37:54.258 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: modify_timestamp: Thu Sep 12 13:37:53 2024 2024-09-12T13:37:54.258 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring state: enabled 2024-09-12T13:37:54.258 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring mode: snapshot 2024-09-12T13:37:54.258 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring global id: 747f0ef0-de52-4d30-a780-d09d65c61a3b 2024-09-12T13:37:54.258 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring primary: true 2024-09-12T13:37:54.262 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \n\b\d ]] 2024-09-12T13:37:54.262 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \k\r\b\d ]] 2024-09-12T13:37:54.263 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo rbd --cluster cluster1 device map -t krbd mirror/image-primary4 2024-09-12T13:37:54.408 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEV=/dev/rbd3 2024-09-12T13:37:54.409 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEVS+=($DEV) 2024-09-12T13:37:54.409 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo mkfs.ext4 /dev/rbd3 2024-09-12T13:37:54.429 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:mke2fs 1.46.5 (30-Dec-2021) 2024-09-12T13:37:54.443 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Discarding device blocks: 0/2621440 done 2024-09-12T13:37:54.443 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Creating filesystem with 2621440 4k blocks and 655360 inodes 2024-09-12T13:37:54.443 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Filesystem UUID: 88936c7f-f632-4dbe-add8-d44485cfd49a 2024-09-12T13:37:54.443 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Superblock backups stored on blocks: 2024-09-12T13:37:54.443 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 2024-09-12T13:37:54.443 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 2024-09-12T13:37:54.443 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Allocating group tables: 0/80 done 2024-09-12T13:37:54.444 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Writing inode tables: 0/80 done 2024-09-12T13:37:54.448 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Creating journal (16384 blocks): done 2024-09-12T13:37:54.508 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Writing superblocks and filesystem accounting information: 0/80 done 2024-09-12T13:37:54.508 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 2024-09-12T13:37:54.511 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mkdir test-primary4 2024-09-12T13:37:54.513 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo mount /dev/rbd3 test-primary4 2024-09-12T13:37:54.573 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ whoami 2024-09-12T13:37:54.573 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo chown ubuntu test-primary4 2024-09-12T13:37:54.598 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ SNAP_PIDS+=($!) 2024-09-12T13:37:54.598 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ take_mirror_snapshots cluster1 mirror image-primary4 2024-09-12T13:37:54.598 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:37:54.598 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:37:54.598 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:37:54.599 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:37:54.599 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:37:54.599 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:37:54.599 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:37:54.599 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:37:54.599 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:37:54.599 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ WORKLOAD_PIDS+=($!) 2024-09-12T13:37:54.599 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:37:54.599 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ IMG=image-primary5 2024-09-12T13:37:54.599 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ MNTPT=test-primary5 2024-09-12T13:37:54.599 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ create_image_and_enable_mirror cluster1 mirror image-primary5 snapshot 10G 2024-09-12T13:37:54.599 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:37:54.599 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:37:54.599 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:37:54.600 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:37:54.600 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:37:54.600 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:37:54.600 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ slow_untar_workload test-primary4 2024-09-12T13:37:54.600 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mode=snapshot 2024-09-12T13:37:54.601 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n snapshot ']' 2024-09-12T13:37:54.601 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:37:54.601 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ create_image cluster1 mirror image-primary5 10G 2024-09-12T13:37:54.602 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:37:54.602 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:37:54.602 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:37:54.602 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:37:54.602 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:37:54.603 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:37:54.603 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local size=128 2024-09-12T13:37:54.603 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n 10G ']' 2024-09-12T13:37:54.603 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ size=10G 2024-09-12T13:37:54.603 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:37:54.603 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 create --size 10G --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image-primary5 2024-09-12T13:37:54.603 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mountpt=test-primary4 2024-09-12T13:37:54.604 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ cp linux-5.4.tar.gz test-primary4 2024-09-12T13:37:54.668 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ is_pool_mirror_mode_image mirror 2024-09-12T13:37:54.668 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:37:54.668 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' image = image ']' 2024-09-12T13:37:54.668 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:37:54.668 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ enable_mirror cluster1 mirror image-primary5 snapshot 2024-09-12T13:37:54.668 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:37:54.668 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:37:54.668 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:37:54.668 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mode=snapshot 2024-09-12T13:37:54.668 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 mirror image enable mirror/image-primary5 snapshot 2024-09-12T13:37:54.884 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local ret=0 2024-09-12T13:37:54.884 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ timeout 5m bash -c 'zcat test-primary4/linux-5.4.tar.gz | pv -L 256K | tar xf - -C test-primary4' 2024-09-12T13:37:55.173 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:37:55.174 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:37:55.174 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:37:55.174 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:37:55.174 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:37:55.174 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:37:55.188 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Mirroring enabled 2024-09-12T13:37:55.213 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1660 2024-09-12T13:37:55.225 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 info mirror/image-primary5 2024-09-12T13:37:55.228 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:37:55.300 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:rbd image 'image-primary5': 2024-09-12T13:37:55.300 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: size 10 GiB in 2560 objects 2024-09-12T13:37:55.300 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: order 22 (4 MiB objects) 2024-09-12T13:37:55.301 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: snapshot_count: 1 2024-09-12T13:37:55.301 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: id: 29919f160089 2024-09-12T13:37:55.301 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: block_name_prefix: rbd_data.29919f160089 2024-09-12T13:37:55.301 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: format: 2 2024-09-12T13:37:55.301 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: features: layering, exclusive-lock, object-map, fast-diff 2024-09-12T13:37:55.301 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: op_features: 2024-09-12T13:37:55.301 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: flags: 2024-09-12T13:37:55.301 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: create_timestamp: Thu Sep 12 13:37:54 2024 2024-09-12T13:37:55.301 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: access_timestamp: Thu Sep 12 13:37:54 2024 2024-09-12T13:37:55.301 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: modify_timestamp: Thu Sep 12 13:37:54 2024 2024-09-12T13:37:55.301 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring state: enabled 2024-09-12T13:37:55.301 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring mode: snapshot 2024-09-12T13:37:55.301 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring global id: 19fef5ab-79b5-4712-8250-75fffa450f0a 2024-09-12T13:37:55.301 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring primary: true 2024-09-12T13:37:55.306 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \n\b\d ]] 2024-09-12T13:37:55.306 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \k\r\b\d ]] 2024-09-12T13:37:55.306 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo rbd --cluster cluster1 device map -t krbd mirror/image-primary5 2024-09-12T13:37:55.462 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEV=/dev/rbd4 2024-09-12T13:37:55.462 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEVS+=($DEV) 2024-09-12T13:37:55.462 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo mkfs.ext4 /dev/rbd4 2024-09-12T13:37:55.483 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:mke2fs 1.46.5 (30-Dec-2021) 2024-09-12T13:37:55.496 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Discarding device blocks: 0/2621440 done 2024-09-12T13:37:55.496 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Creating filesystem with 2621440 4k blocks and 655360 inodes 2024-09-12T13:37:55.496 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Filesystem UUID: 3e535770-f84c-453a-838c-63409075ab00 2024-09-12T13:37:55.496 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Superblock backups stored on blocks: 2024-09-12T13:37:55.496 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 2024-09-12T13:37:55.496 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 2024-09-12T13:37:55.496 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Allocating group tables: 0/80 done 2024-09-12T13:37:55.496 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Writing inode tables: 0/80 done 2024-09-12T13:37:55.501 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Creating journal (16384 blocks): done 2024-09-12T13:37:55.594 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Writing superblocks and filesystem accounting information: 0/80 done 2024-09-12T13:37:55.594 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 2024-09-12T13:37:55.597 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mkdir test-primary5 2024-09-12T13:37:55.601 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo mount /dev/rbd4 test-primary5 2024-09-12T13:37:55.697 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ whoami 2024-09-12T13:37:55.698 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo chown ubuntu test-primary5 2024-09-12T13:37:55.725 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ SNAP_PIDS+=($!) 2024-09-12T13:37:55.725 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ take_mirror_snapshots cluster1 mirror image-primary5 2024-09-12T13:37:55.725 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:37:55.725 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:37:55.726 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:37:55.726 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:37:55.726 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:37:55.726 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:37:55.726 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:37:55.726 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:37:55.726 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:37:55.726 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ WORKLOAD_PIDS+=($!) 2024-09-12T13:37:55.726 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ slow_untar_workload test-primary5 2024-09-12T13:37:55.727 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:37:55.727 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ IMG=image-primary6 2024-09-12T13:37:55.727 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ MNTPT=test-primary6 2024-09-12T13:37:55.727 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ create_image_and_enable_mirror cluster1 mirror image-primary6 snapshot 10G 2024-09-12T13:37:55.727 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mountpt=test-primary5 2024-09-12T13:37:55.727 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ cp linux-5.4.tar.gz test-primary5 2024-09-12T13:37:55.727 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:37:55.727 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:37:55.727 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:37:55.727 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:37:55.728 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:37:55.728 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:37:55.728 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mode=snapshot 2024-09-12T13:37:55.728 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n snapshot ']' 2024-09-12T13:37:55.728 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:37:55.728 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ create_image cluster1 mirror image-primary6 10G 2024-09-12T13:37:55.729 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:37:55.729 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:37:55.729 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:37:55.729 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:37:55.729 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:37:55.729 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:37:55.730 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local size=128 2024-09-12T13:37:55.730 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n 10G ']' 2024-09-12T13:37:55.730 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ size=10G 2024-09-12T13:37:55.730 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:37:55.730 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 create --size 10G --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image-primary6 2024-09-12T13:37:55.828 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ is_pool_mirror_mode_image mirror 2024-09-12T13:37:55.829 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:37:55.829 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' image = image ']' 2024-09-12T13:37:55.829 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:37:55.829 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ enable_mirror cluster1 mirror image-primary6 snapshot 2024-09-12T13:37:55.829 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:37:55.829 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:37:55.829 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:37:55.829 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mode=snapshot 2024-09-12T13:37:55.829 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 mirror image enable mirror/image-primary6 snapshot 2024-09-12T13:37:56.021 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local ret=0 2024-09-12T13:37:56.021 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ timeout 5m bash -c 'zcat test-primary5/linux-5.4.tar.gz | pv -L 256K | tar xf - -C test-primary5' 2024-09-12T13:37:56.166 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:37:56.166 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:37:56.166 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:37:56.166 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:37:56.167 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:37:56.167 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:37:56.212 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Mirroring enabled 2024-09-12T13:37:56.229 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1663 2024-09-12T13:37:56.242 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:37:56.249 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1662 2024-09-12T13:37:56.250 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 info mirror/image-primary6 2024-09-12T13:37:56.264 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:37:56.335 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:rbd image 'image-primary6': 2024-09-12T13:37:56.335 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: size 10 GiB in 2560 objects 2024-09-12T13:37:56.335 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: order 22 (4 MiB objects) 2024-09-12T13:37:56.336 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: snapshot_count: 1 2024-09-12T13:37:56.336 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: id: 29a218f11c71 2024-09-12T13:37:56.336 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: block_name_prefix: rbd_data.29a218f11c71 2024-09-12T13:37:56.336 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: format: 2 2024-09-12T13:37:56.336 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: features: layering, exclusive-lock, object-map, fast-diff 2024-09-12T13:37:56.336 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: op_features: 2024-09-12T13:37:56.337 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: flags: 2024-09-12T13:37:56.337 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: create_timestamp: Thu Sep 12 13:37:55 2024 2024-09-12T13:37:56.337 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: access_timestamp: Thu Sep 12 13:37:55 2024 2024-09-12T13:37:56.337 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: modify_timestamp: Thu Sep 12 13:37:55 2024 2024-09-12T13:37:56.337 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring state: enabled 2024-09-12T13:37:56.337 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring mode: snapshot 2024-09-12T13:37:56.338 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring global id: 85abb8e7-1989-4d4a-a874-9bc560cf4e0d 2024-09-12T13:37:56.338 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring primary: true 2024-09-12T13:37:56.340 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \n\b\d ]] 2024-09-12T13:37:56.340 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \k\r\b\d ]] 2024-09-12T13:37:56.341 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo rbd --cluster cluster1 device map -t krbd mirror/image-primary6 2024-09-12T13:37:56.508 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEV=/dev/rbd5 2024-09-12T13:37:56.508 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEVS+=($DEV) 2024-09-12T13:37:56.508 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo mkfs.ext4 /dev/rbd5 2024-09-12T13:37:56.530 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:mke2fs 1.46.5 (30-Dec-2021) 2024-09-12T13:37:56.545 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Discarding device blocks: 0/2621440 done 2024-09-12T13:37:56.545 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Creating filesystem with 2621440 4k blocks and 655360 inodes 2024-09-12T13:37:56.545 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Filesystem UUID: 6de8e084-4338-4d85-86f7-584a343c2515 2024-09-12T13:37:56.545 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Superblock backups stored on blocks: 2024-09-12T13:37:56.545 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 2024-09-12T13:37:56.545 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 2024-09-12T13:37:56.545 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Allocating group tables: 0/80 done 2024-09-12T13:37:56.546 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Writing inode tables: 0/80 done 2024-09-12T13:37:56.550 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Creating journal (16384 blocks): done 2024-09-12T13:37:56.693 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Writing superblocks and filesystem accounting information: 0/80 done 2024-09-12T13:37:56.694 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 2024-09-12T13:37:56.696 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mkdir test-primary6 2024-09-12T13:37:56.699 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo mount /dev/rbd5 test-primary6 2024-09-12T13:37:56.791 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ whoami 2024-09-12T13:37:56.792 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo chown ubuntu test-primary6 2024-09-12T13:37:56.818 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ SNAP_PIDS+=($!) 2024-09-12T13:37:56.818 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ take_mirror_snapshots cluster1 mirror image-primary6 2024-09-12T13:37:56.818 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:37:56.818 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:37:56.819 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:37:56.819 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:37:56.819 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:37:56.819 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:37:56.819 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ WORKLOAD_PIDS+=($!) 2024-09-12T13:37:56.819 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:37:56.819 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:37:56.819 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:37:56.819 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:37:56.819 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ IMG=image-primary7 2024-09-12T13:37:56.819 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ MNTPT=test-primary7 2024-09-12T13:37:56.819 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ slow_untar_workload test-primary6 2024-09-12T13:37:56.819 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ create_image_and_enable_mirror cluster1 mirror image-primary7 snapshot 10G 2024-09-12T13:37:56.819 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:37:56.820 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:37:56.820 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:37:56.820 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:37:56.820 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:37:56.820 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:37:56.820 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mode=snapshot 2024-09-12T13:37:56.820 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n snapshot ']' 2024-09-12T13:37:56.820 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:37:56.820 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ create_image cluster1 mirror image-primary7 10G 2024-09-12T13:37:56.820 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:37:56.820 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:37:56.820 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mountpt=test-primary6 2024-09-12T13:37:56.820 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:37:56.820 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:37:56.821 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:37:56.821 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:37:56.821 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local size=128 2024-09-12T13:37:56.821 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n 10G ']' 2024-09-12T13:37:56.821 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ size=10G 2024-09-12T13:37:56.821 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:37:56.821 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ cp linux-5.4.tar.gz test-primary6 2024-09-12T13:37:56.821 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 create --size 10G --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image-primary7 2024-09-12T13:37:56.926 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ is_pool_mirror_mode_image mirror 2024-09-12T13:37:56.926 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:37:56.926 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' image = image ']' 2024-09-12T13:37:56.926 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:37:56.926 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ enable_mirror cluster1 mirror image-primary7 snapshot 2024-09-12T13:37:56.926 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:37:56.926 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:37:56.927 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:37:56.927 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mode=snapshot 2024-09-12T13:37:56.927 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 mirror image enable mirror/image-primary7 snapshot 2024-09-12T13:37:57.140 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local ret=0 2024-09-12T13:37:57.140 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ timeout 5m bash -c 'zcat test-primary6/linux-5.4.tar.gz | pv -L 256K | tar xf - -C test-primary6' 2024-09-12T13:37:57.197 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:37:57.197 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:37:57.197 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:37:57.197 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:37:57.197 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:37:57.197 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:37:57.233 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1666 2024-09-12T13:37:57.233 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Mirroring enabled 2024-09-12T13:37:57.237 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1665 2024-09-12T13:37:57.250 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:37:57.252 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:37:57.270 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 info mirror/image-primary7 2024-09-12T13:37:57.344 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:rbd image 'image-primary7': 2024-09-12T13:37:57.344 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: size 10 GiB in 2560 objects 2024-09-12T13:37:57.345 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: order 22 (4 MiB objects) 2024-09-12T13:37:57.345 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: snapshot_count: 1 2024-09-12T13:37:57.345 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: id: 29b24acea35f 2024-09-12T13:37:57.345 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: block_name_prefix: rbd_data.29b24acea35f 2024-09-12T13:37:57.345 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: format: 2 2024-09-12T13:37:57.345 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: features: layering, exclusive-lock, object-map, fast-diff 2024-09-12T13:37:57.345 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: op_features: 2024-09-12T13:37:57.345 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: flags: 2024-09-12T13:37:57.345 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: create_timestamp: Thu Sep 12 13:37:56 2024 2024-09-12T13:37:57.345 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: access_timestamp: Thu Sep 12 13:37:56 2024 2024-09-12T13:37:57.345 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: modify_timestamp: Thu Sep 12 13:37:56 2024 2024-09-12T13:37:57.345 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring state: enabled 2024-09-12T13:37:57.345 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring mode: snapshot 2024-09-12T13:37:57.345 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring global id: 6ca46845-7741-492c-99fb-642d3ab53fab 2024-09-12T13:37:57.346 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring primary: true 2024-09-12T13:37:57.350 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \n\b\d ]] 2024-09-12T13:37:57.350 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \k\r\b\d ]] 2024-09-12T13:37:57.350 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo rbd --cluster cluster1 device map -t krbd mirror/image-primary7 2024-09-12T13:37:57.502 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEV=/dev/rbd6 2024-09-12T13:37:57.502 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEVS+=($DEV) 2024-09-12T13:37:57.502 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo mkfs.ext4 /dev/rbd6 2024-09-12T13:37:57.523 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:mke2fs 1.46.5 (30-Dec-2021) 2024-09-12T13:37:57.537 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Discarding device blocks: 0/2621440 done 2024-09-12T13:37:57.537 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Creating filesystem with 2621440 4k blocks and 655360 inodes 2024-09-12T13:37:57.538 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Filesystem UUID: 560a0069-3880-42dd-8756-1ae637927295 2024-09-12T13:37:57.538 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Superblock backups stored on blocks: 2024-09-12T13:37:57.538 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 2024-09-12T13:37:57.538 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 2024-09-12T13:37:57.538 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Allocating group tables: 0/80 done 2024-09-12T13:37:57.538 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Writing inode tables: 0/80 done 2024-09-12T13:37:57.543 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Creating journal (16384 blocks): done 2024-09-12T13:37:57.677 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Writing superblocks and filesystem accounting information: 0/80 done 2024-09-12T13:37:57.677 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 2024-09-12T13:37:57.679 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mkdir test-primary7 2024-09-12T13:37:57.682 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo mount /dev/rbd6 test-primary7 2024-09-12T13:37:57.740 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ whoami 2024-09-12T13:37:57.742 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo chown ubuntu test-primary7 2024-09-12T13:37:57.767 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ SNAP_PIDS+=($!) 2024-09-12T13:37:57.767 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ take_mirror_snapshots cluster1 mirror image-primary7 2024-09-12T13:37:57.767 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:37:57.767 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:37:57.767 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:37:57.767 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:37:57.768 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:37:57.768 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:37:57.768 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:37:57.768 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ WORKLOAD_PIDS+=($!) 2024-09-12T13:37:57.768 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:37:57.768 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:37:57.768 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ IMG=image-primary8 2024-09-12T13:37:57.768 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ MNTPT=test-primary8 2024-09-12T13:37:57.768 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ slow_untar_workload test-primary7 2024-09-12T13:37:57.768 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ create_image_and_enable_mirror cluster1 mirror image-primary8 snapshot 10G 2024-09-12T13:37:57.768 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:37:57.768 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:37:57.768 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:37:57.769 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:37:57.769 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:37:57.769 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:37:57.769 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:37:57.769 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mode=snapshot 2024-09-12T13:37:57.769 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n snapshot ']' 2024-09-12T13:37:57.769 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:37:57.769 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ create_image cluster1 mirror image-primary8 10G 2024-09-12T13:37:57.769 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:37:57.769 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:37:57.769 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mountpt=test-primary7 2024-09-12T13:37:57.769 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:37:57.769 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:37:57.770 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:37:57.770 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:37:57.770 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local size=128 2024-09-12T13:37:57.770 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n 10G ']' 2024-09-12T13:37:57.770 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ size=10G 2024-09-12T13:37:57.770 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ cp linux-5.4.tar.gz test-primary7 2024-09-12T13:37:57.770 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:37:57.770 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 create --size 10G --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image-primary8 2024-09-12T13:37:57.840 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ is_pool_mirror_mode_image mirror 2024-09-12T13:37:57.841 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:37:57.841 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' image = image ']' 2024-09-12T13:37:57.841 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:37:57.841 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ enable_mirror cluster1 mirror image-primary8 snapshot 2024-09-12T13:37:57.841 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:37:57.841 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:37:57.841 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:37:57.841 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mode=snapshot 2024-09-12T13:37:57.841 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 mirror image enable mirror/image-primary8 snapshot 2024-09-12T13:37:58.073 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local ret=0 2024-09-12T13:37:58.073 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ timeout 5m bash -c 'zcat test-primary7/linux-5.4.tar.gz | pv -L 256K | tar xf - -C test-primary7' 2024-09-12T13:37:58.138 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1669 2024-09-12T13:37:58.139 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Mirroring enabled 2024-09-12T13:37:58.153 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1668 2024-09-12T13:37:58.161 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:37:58.163 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 info mirror/image-primary8 2024-09-12T13:37:58.167 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:37:58.231 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:37:58.231 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:37:58.232 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:37:58.232 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:37:58.232 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:37:58.232 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:37:58.237 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:rbd image 'image-primary8': 2024-09-12T13:37:58.237 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: size 10 GiB in 2560 objects 2024-09-12T13:37:58.237 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: order 22 (4 MiB objects) 2024-09-12T13:37:58.237 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: snapshot_count: 1 2024-09-12T13:37:58.237 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: id: 29c1f1c15744 2024-09-12T13:37:58.237 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: block_name_prefix: rbd_data.29c1f1c15744 2024-09-12T13:37:58.237 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: format: 2 2024-09-12T13:37:58.237 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: features: layering, exclusive-lock, object-map, fast-diff 2024-09-12T13:37:58.237 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: op_features: 2024-09-12T13:37:58.237 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: flags: 2024-09-12T13:37:58.237 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: create_timestamp: Thu Sep 12 13:37:57 2024 2024-09-12T13:37:58.237 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: access_timestamp: Thu Sep 12 13:37:57 2024 2024-09-12T13:37:58.238 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: modify_timestamp: Thu Sep 12 13:37:57 2024 2024-09-12T13:37:58.238 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring state: enabled 2024-09-12T13:37:58.238 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring mode: snapshot 2024-09-12T13:37:58.238 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring global id: 11646481-daec-461e-b7dc-ed4d9a9fd543 2024-09-12T13:37:58.238 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring primary: true 2024-09-12T13:37:58.242 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \n\b\d ]] 2024-09-12T13:37:58.242 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \k\r\b\d ]] 2024-09-12T13:37:58.243 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo rbd --cluster cluster1 device map -t krbd mirror/image-primary8 2024-09-12T13:37:58.365 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEV=/dev/rbd7 2024-09-12T13:37:58.365 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEVS+=($DEV) 2024-09-12T13:37:58.365 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo mkfs.ext4 /dev/rbd7 2024-09-12T13:37:58.385 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:mke2fs 1.46.5 (30-Dec-2021) 2024-09-12T13:37:58.399 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Discarding device blocks: 0/2621440 done 2024-09-12T13:37:58.399 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Creating filesystem with 2621440 4k blocks and 655360 inodes 2024-09-12T13:37:58.399 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Filesystem UUID: f93024bd-c5c8-412f-8e4c-8a0513f41cd6 2024-09-12T13:37:58.399 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Superblock backups stored on blocks: 2024-09-12T13:37:58.399 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 2024-09-12T13:37:58.399 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 2024-09-12T13:37:58.400 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Allocating group tables: 0/80 done 2024-09-12T13:37:58.400 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Writing inode tables: 0/80 done 2024-09-12T13:37:58.404 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Creating journal (16384 blocks): done 2024-09-12T13:37:58.461 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Writing superblocks and filesystem accounting information: 0/80 done 2024-09-12T13:37:58.461 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 2024-09-12T13:37:58.462 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mkdir test-primary8 2024-09-12T13:37:58.465 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo mount /dev/rbd7 test-primary8 2024-09-12T13:37:58.521 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ whoami 2024-09-12T13:37:58.523 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo chown ubuntu test-primary8 2024-09-12T13:37:58.553 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ SNAP_PIDS+=($!) 2024-09-12T13:37:58.553 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ take_mirror_snapshots cluster1 mirror image-primary8 2024-09-12T13:37:58.553 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:37:58.553 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:37:58.553 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:37:58.553 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:37:58.554 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:37:58.554 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:37:58.554 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:37:58.554 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:37:58.554 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:37:58.554 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ WORKLOAD_PIDS+=($!) 2024-09-12T13:37:58.554 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:37:58.554 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ IMG=image-primary9 2024-09-12T13:37:58.554 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ MNTPT=test-primary9 2024-09-12T13:37:58.554 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ create_image_and_enable_mirror cluster1 mirror image-primary9 snapshot 10G 2024-09-12T13:37:58.554 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ slow_untar_workload test-primary8 2024-09-12T13:37:58.554 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:37:58.555 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:37:58.555 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:37:58.555 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:37:58.555 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:37:58.555 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:37:58.555 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mode=snapshot 2024-09-12T13:37:58.555 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n snapshot ']' 2024-09-12T13:37:58.555 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:37:58.555 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ create_image cluster1 mirror image-primary9 10G 2024-09-12T13:37:58.555 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:37:58.555 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:37:58.555 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:37:58.555 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:37:58.556 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:37:58.556 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:37:58.556 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local size=128 2024-09-12T13:37:58.556 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n 10G ']' 2024-09-12T13:37:58.556 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ size=10G 2024-09-12T13:37:58.556 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:37:58.556 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 create --size 10G --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image-primary9 2024-09-12T13:37:58.556 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mountpt=test-primary8 2024-09-12T13:37:58.556 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ cp linux-5.4.tar.gz test-primary8 2024-09-12T13:37:58.841 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ is_pool_mirror_mode_image mirror 2024-09-12T13:37:58.841 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:37:58.841 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' image = image ']' 2024-09-12T13:37:58.841 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:37:58.841 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ enable_mirror cluster1 mirror image-primary9 snapshot 2024-09-12T13:37:58.841 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:37:58.841 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:37:58.841 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:37:58.842 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mode=snapshot 2024-09-12T13:37:58.842 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 mirror image enable mirror/image-primary9 snapshot 2024-09-12T13:37:58.860 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local ret=0 2024-09-12T13:37:58.860 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ timeout 5m bash -c 'zcat test-primary8/linux-5.4.tar.gz | pv -L 256K | tar xf - -C test-primary8' 2024-09-12T13:37:59.197 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1672 2024-09-12T13:37:59.198 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Mirroring enabled 2024-09-12T13:37:59.204 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1671 2024-09-12T13:37:59.219 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:37:59.229 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:37:59.229 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 info mirror/image-primary9 2024-09-12T13:37:59.245 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:37:59.245 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:37:59.245 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:37:59.246 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:37:59.246 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:37:59.246 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:37:59.267 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:37:59.267 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:37:59.267 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:37:59.267 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:37:59.267 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:37:59.267 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:37:59.318 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:rbd image 'image-primary9': 2024-09-12T13:37:59.318 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: size 10 GiB in 2560 objects 2024-09-12T13:37:59.318 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: order 22 (4 MiB objects) 2024-09-12T13:37:59.318 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: snapshot_count: 1 2024-09-12T13:37:59.319 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: id: 29d38d11649d 2024-09-12T13:37:59.319 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: block_name_prefix: rbd_data.29d38d11649d 2024-09-12T13:37:59.319 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: format: 2 2024-09-12T13:37:59.319 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: features: layering, exclusive-lock, object-map, fast-diff 2024-09-12T13:37:59.319 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: op_features: 2024-09-12T13:37:59.320 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: flags: 2024-09-12T13:37:59.320 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: create_timestamp: Thu Sep 12 13:37:58 2024 2024-09-12T13:37:59.320 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: access_timestamp: Thu Sep 12 13:37:58 2024 2024-09-12T13:37:59.320 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: modify_timestamp: Thu Sep 12 13:37:58 2024 2024-09-12T13:37:59.320 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring state: enabled 2024-09-12T13:37:59.320 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring mode: snapshot 2024-09-12T13:37:59.320 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring global id: b98c3fed-4807-46cf-b9ef-4cbefc529d10 2024-09-12T13:37:59.320 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring primary: true 2024-09-12T13:37:59.324 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \n\b\d ]] 2024-09-12T13:37:59.324 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \k\r\b\d ]] 2024-09-12T13:37:59.325 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo rbd --cluster cluster1 device map -t krbd mirror/image-primary9 2024-09-12T13:37:59.447 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEV=/dev/rbd8 2024-09-12T13:37:59.447 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEVS+=($DEV) 2024-09-12T13:37:59.447 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo mkfs.ext4 /dev/rbd8 2024-09-12T13:37:59.468 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:mke2fs 1.46.5 (30-Dec-2021) 2024-09-12T13:37:59.481 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Discarding device blocks: 0/2621440 done 2024-09-12T13:37:59.481 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Creating filesystem with 2621440 4k blocks and 655360 inodes 2024-09-12T13:37:59.481 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Filesystem UUID: 677507b3-6c1d-4980-a94e-787466ace64d 2024-09-12T13:37:59.482 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Superblock backups stored on blocks: 2024-09-12T13:37:59.482 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 2024-09-12T13:37:59.482 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 2024-09-12T13:37:59.482 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Allocating group tables: 0/80 done 2024-09-12T13:37:59.482 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Writing inode tables: 0/80 done 2024-09-12T13:37:59.485 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Creating journal (16384 blocks): done 2024-09-12T13:37:59.539 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Writing superblocks and filesystem accounting information: 0/80 done 2024-09-12T13:37:59.539 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 2024-09-12T13:37:59.542 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mkdir test-primary9 2024-09-12T13:37:59.544 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo mount /dev/rbd8 test-primary9 2024-09-12T13:37:59.695 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ whoami 2024-09-12T13:37:59.697 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo chown ubuntu test-primary9 2024-09-12T13:37:59.726 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ SNAP_PIDS+=($!) 2024-09-12T13:37:59.726 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ take_mirror_snapshots cluster1 mirror image-primary9 2024-09-12T13:37:59.726 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:37:59.726 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:37:59.727 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:37:59.727 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:37:59.727 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:37:59.727 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:37:59.727 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:37:59.728 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:37:59.728 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ WORKLOAD_PIDS+=($!) 2024-09-12T13:37:59.728 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:37:59.728 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:37:59.728 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ IMG=image-primary10 2024-09-12T13:37:59.728 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ MNTPT=test-primary10 2024-09-12T13:37:59.729 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ create_image_and_enable_mirror cluster1 mirror image-primary10 snapshot 10G 2024-09-12T13:37:59.729 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ slow_untar_workload test-primary9 2024-09-12T13:37:59.729 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:37:59.729 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:37:59.729 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:37:59.730 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:37:59.730 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:37:59.730 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:37:59.730 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mode=snapshot 2024-09-12T13:37:59.730 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n snapshot ']' 2024-09-12T13:37:59.730 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:37:59.730 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mountpt=test-primary9 2024-09-12T13:37:59.730 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ create_image cluster1 mirror image-primary10 10G 2024-09-12T13:37:59.730 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ cp linux-5.4.tar.gz test-primary9 2024-09-12T13:37:59.730 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:37:59.730 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:37:59.730 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:37:59.730 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:37:59.730 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:37:59.731 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:37:59.731 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local size=128 2024-09-12T13:37:59.731 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n 10G ']' 2024-09-12T13:37:59.731 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ size=10G 2024-09-12T13:37:59.731 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:37:59.731 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 create --size 10G --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image-primary10 2024-09-12T13:37:59.795 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ is_pool_mirror_mode_image mirror 2024-09-12T13:37:59.795 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:37:59.795 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' image = image ']' 2024-09-12T13:37:59.796 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:37:59.796 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ enable_mirror cluster1 mirror image-primary10 snapshot 2024-09-12T13:37:59.796 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:37:59.796 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:37:59.796 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:37:59.796 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mode=snapshot 2024-09-12T13:37:59.796 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 mirror image enable mirror/image-primary10 snapshot 2024-09-12T13:38:00.016 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local ret=0 2024-09-12T13:38:00.016 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ timeout 5m bash -c 'zcat test-primary9/linux-5.4.tar.gz | pv -L 256K | tar xf - -C test-primary9' 2024-09-12T13:38:00.202 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Mirroring enabled 2024-09-12T13:38:00.206 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1675 2024-09-12T13:38:00.207 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1674 2024-09-12T13:38:00.221 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1676 2024-09-12T13:38:00.233 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:38:00.238 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 info mirror/image-primary10 2024-09-12T13:38:00.242 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:38:00.242 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:38:00.253 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:38:00.253 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:38:00.253 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:38:00.253 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:38:00.253 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:38:00.254 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:38:00.255 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:38:00.255 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:38:00.255 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:38:00.255 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:38:00.255 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:38:00.256 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:38:00.319 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:rbd image 'image-primary10': 2024-09-12T13:38:00.319 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: size 10 GiB in 2560 objects 2024-09-12T13:38:00.319 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: order 22 (4 MiB objects) 2024-09-12T13:38:00.320 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: snapshot_count: 1 2024-09-12T13:38:00.320 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: id: 29e7c018cd96 2024-09-12T13:38:00.320 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: block_name_prefix: rbd_data.29e7c018cd96 2024-09-12T13:38:00.320 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: format: 2 2024-09-12T13:38:00.320 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: features: layering, exclusive-lock, object-map, fast-diff 2024-09-12T13:38:00.320 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: op_features: 2024-09-12T13:38:00.320 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: flags: 2024-09-12T13:38:00.320 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: create_timestamp: Thu Sep 12 13:37:59 2024 2024-09-12T13:38:00.320 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: access_timestamp: Thu Sep 12 13:37:59 2024 2024-09-12T13:38:00.320 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: modify_timestamp: Thu Sep 12 13:37:59 2024 2024-09-12T13:38:00.321 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring state: enabled 2024-09-12T13:38:00.321 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring mode: snapshot 2024-09-12T13:38:00.321 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring global id: 2e48042f-ba7f-45c0-9287-a6bae29dd722 2024-09-12T13:38:00.321 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring primary: true 2024-09-12T13:38:00.326 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \n\b\d ]] 2024-09-12T13:38:00.327 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \k\r\b\d ]] 2024-09-12T13:38:00.327 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo rbd --cluster cluster1 device map -t krbd mirror/image-primary10 2024-09-12T13:38:00.450 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEV=/dev/rbd9 2024-09-12T13:38:00.451 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEVS+=($DEV) 2024-09-12T13:38:00.451 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo mkfs.ext4 /dev/rbd9 2024-09-12T13:38:00.472 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:mke2fs 1.46.5 (30-Dec-2021) 2024-09-12T13:38:00.485 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Discarding device blocks: 0/2621440 done 2024-09-12T13:38:00.485 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Creating filesystem with 2621440 4k blocks and 655360 inodes 2024-09-12T13:38:00.485 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Filesystem UUID: ba78e6df-7e89-4d5b-a112-14fd610a6ebc 2024-09-12T13:38:00.485 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Superblock backups stored on blocks: 2024-09-12T13:38:00.485 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 2024-09-12T13:38:00.486 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 2024-09-12T13:38:00.486 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Allocating group tables: 0/80 done 2024-09-12T13:38:00.486 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Writing inode tables: 0/80 done 2024-09-12T13:38:00.490 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Creating journal (16384 blocks): done 2024-09-12T13:38:00.545 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Writing superblocks and filesystem accounting information: 0/80 done 2024-09-12T13:38:00.545 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 2024-09-12T13:38:00.548 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mkdir test-primary10 2024-09-12T13:38:00.550 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo mount /dev/rbd9 test-primary10 2024-09-12T13:38:00.699 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ whoami 2024-09-12T13:38:00.701 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo chown ubuntu test-primary10 2024-09-12T13:38:00.726 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ SNAP_PIDS+=($!) 2024-09-12T13:38:00.726 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ take_mirror_snapshots cluster1 mirror image-primary10 2024-09-12T13:38:00.726 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:38:00.726 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:38:00.726 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:38:00.726 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:38:00.726 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:38:00.726 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:38:00.726 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:38:00.726 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:38:00.727 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ WORKLOAD_PIDS+=($!) 2024-09-12T13:38:00.727 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:38:00.727 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${SNAP_PIDS[@]} 2024-09-12T13:38:00.727 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 94535 2024-09-12T13:38:00.727 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ slow_untar_workload test-primary10 2024-09-12T13:38:00.727 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mountpt=test-primary10 2024-09-12T13:38:00.727 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ cp linux-5.4.tar.gz test-primary10 2024-09-12T13:38:01.009 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local ret=0 2024-09-12T13:38:01.009 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ timeout 5m bash -c 'zcat test-primary10/linux-5.4.tar.gz | pv -L 256K | tar xf - -C test-primary10' 2024-09-12T13:38:01.170 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:38:01.170 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:38:01.171 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:38:01.171 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:38:01.171 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:38:01.171 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:38:01.171 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:38:01.171 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:38:01.177 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:38:01.177 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:38:01.177 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:38:01.177 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:38:01.232 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1680 2024-09-12T13:38:01.232 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1678 2024-09-12T13:38:01.232 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1679 2024-09-12T13:38:01.262 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:38:01.272 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:38:01.290 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:38:02.222 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:38:02.223 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:38:02.223 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:38:02.223 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:38:02.223 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:38:02.223 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:38:02.233 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:38:02.234 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:38:02.234 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:38:02.234 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:38:02.234 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:38:02.234 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:38:02.295 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1681 2024-09-12T13:38:02.301 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1682 2024-09-12T13:38:02.361 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:38:02.366 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:38:03.164 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1684 2024-09-12T13:38:03.184 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:38:03.202 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1683 2024-09-12T13:38:03.233 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:38:03.236 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:38:03.236 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:38:03.237 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:38:03.237 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:38:03.237 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:38:03.237 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:38:03.245 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:38:03.248 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:38:03.248 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:38:03.248 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:38:03.248 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:38:03.248 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:38:03.248 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:38:03.248 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:38:03.248 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:38:03.248 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:38:03.249 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:38:03.249 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:38:04.168 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1687 2024-09-12T13:38:04.215 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1686 2024-09-12T13:38:04.229 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:38:04.246 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1685 2024-09-12T13:38:04.271 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:38:04.272 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:38:04.272 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:38:04.272 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:38:04.272 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:38:04.272 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:38:04.278 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:38:04.278 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:38:04.278 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:38:04.278 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:38:04.278 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:38:04.278 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:38:04.310 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:38:04.310 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:38:04.310 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:38:04.310 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:38:04.311 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:38:04.311 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:38:04.334 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:38:04.358 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:38:05.265 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1688 2024-09-12T13:38:05.289 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1689 2024-09-12T13:38:05.297 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1690 2024-09-12T13:38:05.316 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:38:05.325 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:38:05.336 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:38:05.366 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:38:05.367 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:38:05.367 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:38:05.367 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:38:05.367 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:38:05.367 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:38:05.370 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:38:05.371 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:38:05.371 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:38:05.371 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:38:05.371 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:38:05.371 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:38:06.189 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:38:06.189 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:38:06.189 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:38:06.190 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:38:06.190 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:38:06.190 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:38:06.237 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:38:06.237 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:38:06.237 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:38:06.237 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:38:06.238 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:38:06.238 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:38:06.308 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1692 2024-09-12T13:38:06.325 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1691 2024-09-12T13:38:06.365 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:38:06.383 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:38:07.234 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:38:07.234 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:38:07.234 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:38:07.234 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:38:07.234 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:38:07.234 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:38:07.252 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1693 2024-09-12T13:38:07.260 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1694 2024-09-12T13:38:07.296 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:38:07.309 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:38:07.339 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:38:07.340 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:38:07.340 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:38:07.340 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:38:07.340 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:38:07.340 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:38:07.366 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:38:07.366 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:38:07.366 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:38:07.367 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:38:07.367 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:38:07.367 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:38:08.147 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1696 2024-09-12T13:38:08.169 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1697 2024-09-12T13:38:08.169 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1695 2024-09-12T13:38:08.194 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:38:08.194 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:38:08.199 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:38:08.320 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:38:08.321 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:38:08.321 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:38:08.321 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:38:08.321 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:38:08.321 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:38:08.329 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:38:08.329 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:38:08.330 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:38:08.330 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:38:08.330 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:38:08.330 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:38:08.343 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:38:08.343 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:38:08.343 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:38:08.343 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:38:08.344 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:38:08.344 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:38:09.144 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1699 2024-09-12T13:38:09.146 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1700 2024-09-12T13:38:09.172 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:38:09.178 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:38:09.226 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1698 2024-09-12T13:38:09.263 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:38:09.379 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:38:09.379 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:38:09.379 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:38:09.379 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:38:09.379 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:38:09.379 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:38:09.386 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:38:09.386 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:38:09.386 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:38:09.386 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:38:09.386 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:38:09.386 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:38:10.156 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1701 2024-09-12T13:38:10.157 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1702 2024-09-12T13:38:10.174 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:38:10.206 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:38:10.307 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:38:10.307 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:38:10.307 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:38:10.307 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:38:10.307 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:38:10.307 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:38:10.316 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:38:10.316 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:38:10.317 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:38:10.317 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:38:10.317 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:38:10.317 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:38:11.159 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1704 2024-09-12T13:38:11.170 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1703 2024-09-12T13:38:11.185 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:38:11.198 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:38:11.199 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:38:11.199 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:38:11.199 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:38:11.199 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:38:11.199 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:38:11.204 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:38:11.204 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:38:11.204 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:38:11.204 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:38:11.204 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:38:11.204 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:38:11.209 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:38:11.212 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:38:11.212 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:38:11.212 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:38:11.212 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:38:11.212 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:38:11.212 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:38:12.175 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:38:12.175 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:38:12.175 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:38:12.175 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:38:12.176 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:38:12.176 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:38:12.183 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:38:12.183 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:38:12.183 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:38:12.184 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:38:12.184 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:38:12.184 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:38:12.208 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1707 2024-09-12T13:38:12.214 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1705 2024-09-12T13:38:12.250 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:38:12.251 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:38:12.271 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:38:12.271 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:38:12.271 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:38:12.271 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:38:12.271 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:38:12.272 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:38:13.177 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:38:13.177 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:38:13.177 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:38:13.177 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:38:13.177 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:38:13.177 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:38:13.211 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:38:13.211 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:38:13.211 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:38:13.211 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:38:13.211 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:38:13.211 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:38:13.227 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1709 2024-09-12T13:38:13.319 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1708 2024-09-12T13:38:13.325 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:38:13.395 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:38:14.151 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1706 2024-09-12T13:38:14.192 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:38:14.193 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:38:14.193 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:38:14.193 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:38:14.193 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:38:14.193 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:38:14.214 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:38:14.215 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:38:14.215 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:38:14.215 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:38:14.215 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:38:14.215 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:38:14.232 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1710 2024-09-12T13:38:14.409 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:38:14.409 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1712 2024-09-12T13:38:14.410 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:38:14.410 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:38:15.255 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:38:15.255 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:38:15.255 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:38:15.256 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:38:15.256 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:38:15.256 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:38:15.257 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:38:15.257 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:38:15.257 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:38:15.257 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:38:15.257 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:38:15.258 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:38:15.281 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1713 2024-09-12T13:38:15.297 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:38:16.147 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1711 2024-09-12T13:38:16.186 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:38:16.217 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1715 2024-09-12T13:38:16.240 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:38:16.330 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:38:16.330 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:38:16.330 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:38:16.330 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:38:16.331 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:38:16.331 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:38:16.398 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:38:16.398 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:38:16.399 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:38:16.399 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:38:16.399 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:38:16.399 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:38:17.143 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1714 2024-09-12T13:38:17.185 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:38:17.235 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1717 2024-09-12T13:38:17.262 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:38:17.273 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:38:17.273 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:38:17.273 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:38:17.273 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:38:17.273 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:38:17.274 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:38:17.311 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:38:17.311 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:38:17.311 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:38:17.311 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:38:17.311 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:38:17.312 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:38:17.349 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:38:17.349 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:38:17.349 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:38:17.349 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:38:17.349 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:38:17.349 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:38:18.123 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1716 2024-09-12T13:38:18.169 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:38:18.194 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1718 2024-09-12T13:38:18.205 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1720 2024-09-12T13:38:18.207 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:38:18.229 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:38:18.301 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:38:18.301 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:38:18.301 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:38:18.301 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:38:18.301 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:38:18.301 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:38:19.190 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:38:19.190 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:38:19.190 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:38:19.190 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:38:19.190 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:38:19.190 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:38:19.204 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1721 2024-09-12T13:38:19.221 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1719 2024-09-12T13:38:19.245 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:38:19.245 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:38:19.245 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:38:19.245 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:38:19.246 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:38:19.246 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:38:19.246 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:38:19.382 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:38:20.188 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:38:20.188 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:38:20.188 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:38:20.188 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:38:20.188 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:38:20.188 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:38:20.265 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:38:20.265 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:38:20.265 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:38:20.266 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:38:20.266 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:38:20.266 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:38:20.350 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1723 2024-09-12T13:38:20.367 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:38:21.174 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:38:21.174 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:38:21.174 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:38:21.174 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:38:21.174 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:38:21.174 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:38:21.210 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:38:21.210 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:38:21.210 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:38:21.210 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:38:21.210 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:38:21.210 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:38:21.212 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1722 2024-09-12T13:38:21.232 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:38:21.232 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:38:21.232 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:38:21.232 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:38:21.232 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:38:21.233 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:38:21.262 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:38:21.262+0000 7f67dd000640 -1 rbd::mirror::image_replayer::snapshot::Replayer: 0x56055c315800 handle_unlink_peer: failed to unlink local peer from remote image: (22) Invalid argument 2024-09-12T13:38:21.271 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:38:21.347 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1725 2024-09-12T13:38:21.362 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:38:22.168 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1724 2024-09-12T13:38:22.214 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:38:22.250 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:38:22.251 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:38:22.251 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:38:22.251 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:38:22.251 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:38:22.251 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:38:22.318 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1727 2024-09-12T13:38:22.363 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:38:22.385 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:38:22.385 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:38:22.386 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:38:22.386 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:38:22.386 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:38:22.386 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:38:23.119 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1726 2024-09-12T13:38:23.194 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:38:23.238 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1729 2024-09-12T13:38:23.264 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1728 2024-09-12T13:38:23.280 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:38:23.298 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:38:23.370 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:38:23.370 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:38:23.370 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:38:23.370 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:38:23.370 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:38:23.370 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:38:24.227 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1730 2024-09-12T13:38:24.276 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:38:24.277 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:38:24.277 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:38:24.277 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:38:24.277 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:38:24.277 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:38:24.287 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1731 2024-09-12T13:38:24.302 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:38:24.330 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:38:24.365 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:38:24.365 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:38:24.365 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:38:24.365 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:38:24.366 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:38:24.366 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:38:25.216 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:38:25.217 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:38:25.217 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:38:25.217 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:38:25.217 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:38:25.217 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:38:25.226 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1732 2024-09-12T13:38:25.252 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:38:25.255 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1733 2024-09-12T13:38:25.269 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:38:25.366 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:38:25.366 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:38:25.366 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:38:25.366 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:38:25.366 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:38:25.366 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:38:26.200 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:38:26.200 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:38:26.200 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:38:26.200 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:38:26.201 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:38:26.201 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:38:26.236 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1734 2024-09-12T13:38:26.280 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:38:26.283 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:38:26.283 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:38:26.283 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:38:26.283 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:38:26.283 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:38:26.283 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:38:26.288 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1735 2024-09-12T13:38:26.304 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:38:26.304 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:38:26.304 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:38:26.304 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:38:26.304 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:38:26.304 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:38:26.341 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:38:27.247 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1736 2024-09-12T13:38:27.262 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:38:27.295 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1737 2024-09-12T13:38:27.304 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:38:27.309 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:38:27.309 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:38:27.309 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:38:27.309 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:38:27.309 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:38:27.309 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:38:27.335 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:38:27.335 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:38:27.335 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:38:27.335 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:38:27.335 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:38:27.335 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:38:28.176 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1739 2024-09-12T13:38:28.187 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1738 2024-09-12T13:38:28.201 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:38:28.202 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:38:28.255 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:38:28.255 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:38:28.255 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:38:28.255 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:38:28.255 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:38:28.255 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:38:28.273 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:38:28.273 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:38:28.273 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:38:28.273 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:38:28.273 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:38:28.273 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:38:29.284 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:38:29.284 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:38:29.284 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:38:29.284 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:38:29.284 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:38:29.284 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:38:29.347 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:38:29.347 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:38:29.348 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:38:29.348 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:38:29.348 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:38:29.348 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:38:30.213 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1740 2024-09-12T13:38:30.227 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1741 2024-09-12T13:38:30.250 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1743 2024-09-12T13:38:30.265 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:38:30.265 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:38:30.265 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:38:30.265 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:38:30.266 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:38:30.266 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:38:30.266 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:38:30.290 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:38:30.301 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:38:30.306 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:38:30.306 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:38:30.306 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:38:30.306 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:38:30.307 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:38:30.307 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:38:30.351 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1745 2024-09-12T13:38:30.366 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:38:31.204 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:38:31.205 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:38:31.205 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:38:31.205 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:38:31.205 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:38:31.205 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:38:31.207 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:38:31.207 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:38:31.207 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:38:31.207 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:38:31.207 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:38:31.207 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:38:31.223 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1742 2024-09-12T13:38:31.280 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:38:31.306 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1744 2024-09-12T13:38:31.329 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:38:32.281 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1746 2024-09-12T13:38:32.321 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:38:32.325 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1747 2024-09-12T13:38:32.340 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:38:33.213 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1749 2024-09-12T13:38:33.221 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1748 2024-09-12T13:38:33.242 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:38:33.247 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:38:33.269 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:38:33.269 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:38:33.269 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:38:33.269 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:38:33.269 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:38:33.269 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:38:33.292 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:38:33.293 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:38:33.293 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:38:33.293 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:38:33.293 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:38:33.293 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:38:33.305 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:38:33.305 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:38:33.305 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:38:33.305 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:38:33.306 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:38:33.306 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:38:33.369 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:38:33.369 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:38:33.369 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:38:33.370 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:38:33.370 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:38:33.370 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:38:34.272 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1753 2024-09-12T13:38:34.281 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1752 2024-09-12T13:38:34.284 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:38:34.284 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:38:34.284 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:38:34.284 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:38:34.284 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:38:34.284 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:38:34.285 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1751 2024-09-12T13:38:34.296 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:38:34.304 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:38:34.305 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:38:34.330 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:38:34.331 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:38:34.331 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:38:34.331 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:38:34.331 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:38:34.331 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:38:35.290 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1754 2024-09-12T13:38:35.308 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:38:35.324 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:38:35.324 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:38:35.324 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:38:35.325 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:38:35.325 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:38:35.325 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:38:35.344 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:38:35.344 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:38:35.344 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:38:35.344 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:38:35.344 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:38:35.344 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:38:36.240 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1750 2024-09-12T13:38:36.248 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:38:36.248 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:38:36.248 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:38:36.248 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:38:36.248 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:38:36.248 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:38:36.250 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:38:36.250 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:38:36.250 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:38:36.250 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:38:36.251 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:38:36.251 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:38:36.289 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:38:37.240 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1755 2024-09-12T13:38:37.298 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:38:37.298 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:38:37.298 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:38:37.299 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:38:37.299 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:38:37.299 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:38:37.300 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:38:37.308 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:38:37.308 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:38:37.308 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:38:37.308 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:38:37.308 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:38:37.308 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:38:37.308 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:38:37.308 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:38:37.308 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:38:37.308 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:38:37.308 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:38:37.308 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:38:38.120 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1756 2024-09-12T13:38:38.135 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1757 2024-09-12T13:38:38.175 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:38:38.194 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:38:38.256 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1760 2024-09-12T13:38:38.279 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:38:38.311 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:38:38.311 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:38:38.311 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:38:38.312 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:38:38.312 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:38:38.312 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:38:39.121 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1758 2024-09-12T13:38:39.132 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1759 2024-09-12T13:38:39.172 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:38:39.189 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:38:39.292 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:38:39.292 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:38:39.293 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:38:39.293 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:38:39.293 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:38:39.293 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:38:40.115 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1762 2024-09-12T13:38:40.136 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1761 2024-09-12T13:38:40.174 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:38:40.179 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:38:40.249 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1764 2024-09-12T13:38:40.286 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:38:40.303 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:38:40.303 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:38:40.303 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:38:40.303 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:38:40.303 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:38:40.303 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:38:41.123 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1763 2024-09-12T13:38:41.158 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:38:41.177 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:38:41.177 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:38:41.177 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:38:41.178 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:38:41.178 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:38:41.178 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:38:41.197 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:38:41.197 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:38:41.197 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:38:41.197 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:38:41.197 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:38:41.197 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:38:41.235 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1765 2024-09-12T13:38:41.247 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:38:41.282 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:38:41.283 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:38:41.283 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:38:41.283 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:38:41.283 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:38:41.283 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:38:42.175 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:38:42.175 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:38:42.175 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:38:42.175 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:38:42.175 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:38:42.175 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:38:42.192 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:38:42.192 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:38:42.192 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:38:42.193 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:38:42.193 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:38:42.193 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:38:42.237 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1766 2024-09-12T13:38:42.252 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1767 2024-09-12T13:38:42.265 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:38:42.273 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:38:43.131 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1768 2024-09-12T13:38:43.168 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:38:43.177 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:38:43.178 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:38:43.178 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:38:43.178 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:38:43.178 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:38:43.178 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:38:43.178 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1769 2024-09-12T13:38:43.181 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:38:43.181 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:38:43.182 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:38:43.182 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:38:43.182 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:38:43.182 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:38:43.192 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1770 2024-09-12T13:38:43.201 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:38:43.203 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:38:43.289 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:38:43.290 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:38:43.290 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:38:43.290 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:38:43.290 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:38:43.290 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:38:44.161 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:38:44.161 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:38:44.161 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:38:44.161 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:38:44.161 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:38:44.161 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:38:44.210 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1772 2024-09-12T13:38:44.213 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1771 2024-09-12T13:38:44.219 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:38:44.224 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:38:44.250 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:38:44.250 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:38:44.250 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:38:44.250 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:38:44.250 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:38:44.250 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:38:45.244 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1774 2024-09-12T13:38:45.260 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:38:45.268 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:38:45.268 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:38:45.268 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:38:45.268 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:38:45.268 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:38:45.268 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:38:45.275 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:38:45.275 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:38:45.275 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:38:45.275 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:38:45.276 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:38:45.276 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:38:46.144 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1773 2024-09-12T13:38:46.169 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:38:46.169 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:38:46.169 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:38:46.169 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:38:46.169 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:38:46.170 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:38:46.202 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:38:46.203 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:38:46.203 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:38:46.203 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:38:46.204 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:38:46.204 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:38:46.204 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:38:46.205 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:38:46.206 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:38:46.206 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:38:46.206 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:38:46.206 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:38:46.206 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:38:47.166 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1775 2024-09-12T13:38:47.222 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:38:47.222 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:38:47.222 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:38:47.222 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:38:47.222 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:38:47.223 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:38:47.226 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:38:47.226 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:38:47.226 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:38:47.226 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:38:47.226 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:38:47.227 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:38:47.246 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:38:47.267 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:38:47.267+0000 7f67dc600640 -1 rbd::mirror::image_replayer::snapshot::Replayer: 0x560557a2c000 handle_unlink_peer: failed to unlink local peer from remote image: (22) Invalid argument 2024-09-12T13:38:48.127 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1778 2024-09-12T13:38:48.142 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1776 2024-09-12T13:38:48.163 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1777 2024-09-12T13:38:48.174 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:38:48.198 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:38:48.216 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:38:48.263 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:38:48.263 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:38:48.263 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:38:48.264 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:38:48.264 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:38:48.264 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:38:48.275 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:38:48.275+0000 7f67dd000640 -1 rbd::mirror::image_replayer::snapshot::Replayer: 0x56055b858800 handle_unlink_peer: failed to unlink local peer from remote image: (22) Invalid argument 2024-09-12T13:38:49.155 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1780 2024-09-12T13:38:49.159 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1779 2024-09-12T13:38:49.205 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:38:49.205 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:38:49.205 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:38:49.205 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:38:49.205 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:38:49.205 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:38:49.206 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:38:49.223 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:38:50.178 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1782 2024-09-12T13:38:50.213 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1781 2024-09-12T13:38:50.235 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:38:50.235+0000 7f67dd000640 -1 rbd::mirror::image_replayer::snapshot::Replayer: 0x56055c2e1400 handle_unlink_peer: failed to unlink local peer from remote image: (22) Invalid argument 2024-09-12T13:38:50.250 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:38:50.250 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:38:50.250 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:38:50.250 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:38:50.250 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:38:50.250 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:38:50.251 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:38:50.286 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:38:50.305 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1784 2024-09-12T13:38:50.351 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:38:51.165 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1783 2024-09-12T13:38:51.178 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:38:51.179 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:38:51.179 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:38:51.179 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:38:51.179 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:38:51.179 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:38:51.203 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:38:51.203 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:38:51.203 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:38:51.204 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:38:51.204 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:38:51.204 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:38:51.219 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:38:51.219 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:38:51.220 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:38:51.220 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:38:51.220 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:38:51.221 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:38:51.236 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:38:51.288 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1785 2024-09-12T13:38:51.315 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:38:52.204 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1787 2024-09-12T13:38:52.209 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:38:52.209 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:38:52.209 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:38:52.209 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:38:52.209 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:38:52.209 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:38:52.211 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1788 2024-09-12T13:38:52.224 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:38:52.226 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:38:52.226 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:38:52.226 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:38:52.226 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:38:52.227 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:38:52.227 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:38:52.239 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:38:53.187 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1790 2024-09-12T13:38:53.190 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1789 2024-09-12T13:38:53.208 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:38:53.225 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:38:53.256 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:38:53.256 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:38:53.256 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:38:53.256 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:38:53.256 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:38:53.256 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:38:53.289 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:38:53.289 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:38:53.289 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:38:53.289 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:38:53.289 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:38:53.289 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:38:53.354 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:38:53.354 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:38:53.354 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:38:53.354 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:38:53.355 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:38:53.355 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:38:54.237 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1791 2024-09-12T13:38:54.240 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:38:54.240 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:38:54.241 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:38:54.241 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:38:54.241 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:38:54.241 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:38:54.244 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1792 2024-09-12T13:38:54.256 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:38:54.259 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:38:54.317 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:38:54.317 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:38:54.317 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:38:54.318 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:38:54.318 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:38:54.318 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:38:55.194 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1786 2024-09-12T13:38:55.227 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:38:55.227 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:38:55.227 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:38:55.227 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:38:55.227 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:38:55.228 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:38:55.228 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:38:55.236 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:38:55.236+0000 7f67dd000640 -1 rbd::mirror::image_replayer::snapshot::Replayer: 0x56055b7dec00 handle_unlink_peer: failed to unlink local peer from remote image: (22) Invalid argument 2024-09-12T13:38:55.242 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:38:55.242 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:38:55.242 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:38:55.242 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:38:55.242 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:38:55.242 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:38:55.255 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1794 2024-09-12T13:38:55.273 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:38:56.195 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1793 2024-09-12T13:38:56.213 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:38:56.213 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:38:56.213 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:38:56.213 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:38:56.214 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:38:56.214 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:38:56.229 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:38:56.229 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:38:56.229 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:38:56.229 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:38:56.229 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:38:56.230 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:38:56.238 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:38:56.252 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1795 2024-09-12T13:38:56.267 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:38:57.259 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:38:57.259 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:38:57.259 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:38:57.260 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:38:57.260 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:38:57.260 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:38:57.262 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:38:57.263 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:38:57.263 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:38:57.263 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:38:57.263 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:38:57.263 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:38:57.267 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1799 2024-09-12T13:38:57.288 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:38:58.124 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1797 2024-09-12T13:38:58.129 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1796 2024-09-12T13:38:58.175 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:38:58.189 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:38:58.191 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1798 2024-09-12T13:38:58.208 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:38:58.231 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:38:58.231 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:38:58.231 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:38:58.231 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:38:58.231 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:38:58.231 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:38:58.276 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:38:58.277 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:38:58.277 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:38:58.277 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:38:58.277 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:38:58.277 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:38:59.242 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:38:59.242 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:38:59.242 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:38:59.242 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:38:59.242 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:38:59.242 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:38:59.245 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1800 2024-09-12T13:38:59.268 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:38:59.270 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:38:59.270 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:38:59.270 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:38:59.270 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:38:59.270 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:38:59.271 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:38:59.271 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1803 2024-09-12T13:38:59.302 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:39:00.197 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1801 2024-09-12T13:39:00.229 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:39:00.249 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1804 2024-09-12T13:39:00.266 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:39:00.291 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:39:00.292 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:39:00.292 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:39:00.292 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:39:00.292 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:39:00.292 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:39:01.180 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:39:01.180 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:39:01.180 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:39:01.181 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:39:01.181 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:39:01.181 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:39:01.193 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:39:01.194 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:39:01.194 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:39:01.194 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:39:01.194 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:39:01.194 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:39:01.211 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:39:01.211 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:39:01.211 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:39:01.212 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:39:01.212 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:39:01.212 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:39:01.275 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1806 2024-09-12T13:39:01.296 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:39:02.214 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1802 2024-09-12T13:39:02.217 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1805 2024-09-12T13:39:02.253 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:39:02.254 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:39:02.271 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:39:02.271 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:39:02.271 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:39:02.272 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:39:02.272 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:39:02.272 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:39:02.284 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1807 2024-09-12T13:39:02.289 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1808 2024-09-12T13:39:02.295 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:39:02.298 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:39:02.304 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:39:02.304 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:39:02.304 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:39:02.304 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:39:02.304 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:39:02.305 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:39:03.162 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1809 2024-09-12T13:39:03.170 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1811 2024-09-12T13:39:03.177 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:39:03.194 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:39:03.231 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:39:03.231 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:39:03.231 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:39:03.231 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:39:03.232 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:39:03.232 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:39:03.268 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:39:03.268 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:39:03.269 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:39:03.269 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:39:03.269 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:39:03.269 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:39:04.238 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1810 2024-09-12T13:39:04.254 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1813 2024-09-12T13:39:04.258 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:39:04.260 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1812 2024-09-12T13:39:04.271 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:39:04.275 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:39:04.298 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:39:04.298 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:39:04.298 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:39:04.298 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:39:04.298 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:39:04.299 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:39:05.240 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1814 2024-09-12T13:39:05.255 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:39:05.255 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:39:05.255 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:39:05.256 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:39:05.256 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:39:05.256 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:39:05.256 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:39:05.257 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:39:05.257 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:39:05.257 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:39:05.257 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:39:05.257 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:39:05.257 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:39:05.298 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:39:05.298 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:39:05.298 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:39:05.298 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:39:05.298 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:39:05.298 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:39:05.301 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:39:05.301 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:39:05.301 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:39:05.301 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:39:05.301 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:39:05.301 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:39:06.180 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:39:06.181 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:39:06.181 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:39:06.181 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:39:06.181 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:39:06.181 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:39:06.197 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:39:06.197 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:39:06.198 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:39:06.198 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:39:06.198 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:39:06.198 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:39:06.267 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1816 2024-09-12T13:39:06.280 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1815 2024-09-12T13:39:06.292 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:39:06.292 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:39:07.260 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:39:07.261 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:39:07.261 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:39:07.261 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:39:07.261 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:39:07.261 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:39:07.273 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:39:07.274 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:39:07.274 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:39:07.274 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:39:07.274 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:39:07.274 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:39:07.278 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:39:07.278 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:39:07.278 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:39:07.278 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:39:07.278 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:39:07.278 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:39:07.285 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1820 2024-09-12T13:39:07.302 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:39:08.138 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1817 2024-09-12T13:39:08.152 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1818 2024-09-12T13:39:08.195 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:39:08.208 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:39:08.258 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:39:08.258 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:39:08.258 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:39:08.258 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:39:08.259 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:39:08.259 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:39:08.279 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1822 2024-09-12T13:39:08.302 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:39:09.244 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1819 2024-09-12T13:39:09.286 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:39:09.295 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:39:09.295 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:39:09.295 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:39:09.295 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:39:09.295 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:39:09.295 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:39:09.296 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:39:09.296 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:39:09.296 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:39:09.296 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:39:09.296 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:39:09.296 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:39:10.225 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1823 2024-09-12T13:39:10.230 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1824 2024-09-12T13:39:10.247 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1821 2024-09-12T13:39:10.270 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:39:10.280 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:39:10.287 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:39:10.305 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:39:10.305 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:39:10.305 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:39:10.305 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:39:10.305 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:39:10.305 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:39:11.198 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:39:11.198 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:39:11.198 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:39:11.198 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:39:11.198 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:39:11.198 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:39:11.211 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:39:11.211 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:39:11.212 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:39:11.212 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:39:11.212 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:39:11.212 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:39:11.306 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:39:11.306 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:39:11.306 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:39:11.306 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:39:11.306 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:39:11.306 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:39:12.240 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1826 2024-09-12T13:39:12.252 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1825 2024-09-12T13:39:12.258 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1827 2024-09-12T13:39:12.288 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:39:12.288 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:39:12.288 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:39:12.289 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:39:12.289 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:39:12.289 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:39:12.291 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:39:12.297 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:39:12.309 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:39:12.332 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1830 2024-09-12T13:39:12.338 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1828 2024-09-12T13:39:12.343 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1829 2024-09-12T13:39:12.356 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:39:12.356 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:39:12.385 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:39:13.183 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1831 2024-09-12T13:39:13.193 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:39:13.273 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:39:13.273 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:39:13.273 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:39:13.273 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:39:13.273 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:39:13.273 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:39:13.282 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:39:13.282 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:39:13.283 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:39:13.283 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:39:13.283 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:39:13.283 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:39:13.289 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:39:13.289 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:39:13.290 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:39:13.290 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:39:13.290 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:39:13.290 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:39:14.273 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1833 2024-09-12T13:39:14.275 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1834 2024-09-12T13:39:14.293 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:39:14.295 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:39:15.238 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1832 2024-09-12T13:39:15.261 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:39:15.294 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:39:15.294 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:39:15.294 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:39:15.294 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:39:15.295 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:39:15.295 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:39:15.300 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:39:15.301 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:39:15.301 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:39:15.301 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:39:15.301 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:39:15.301 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:39:15.312 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:39:15.312 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:39:15.312 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:39:15.312 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:39:15.312 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:39:15.312 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:39:15.359 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:39:15.359 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:39:15.359 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:39:15.359 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:39:15.359 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:39:15.359 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:39:15.360 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:39:15.360 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:39:15.360 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:39:15.360 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:39:15.360 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:39:15.360 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:39:15.388 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:39:15.388 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:39:15.388 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:39:15.388 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:39:15.388 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:39:15.388 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:39:16.196 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:39:16.196 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:39:16.197 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:39:16.197 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:39:16.197 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:39:16.197 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:39:16.316 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1837 2024-09-12T13:39:16.330 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1835 2024-09-12T13:39:16.343 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:39:16.343 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:39:17.267 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1838 2024-09-12T13:39:17.275 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1836 2024-09-12T13:39:17.296 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:39:17.296 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:39:17.297 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:39:17.297 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:39:17.297 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:39:17.297 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:39:17.298 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:39:17.298 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:39:17.298 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:39:17.298 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:39:17.298 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:39:17.298 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:39:17.303 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:39:17.322 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:39:18.135 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1840 2024-09-12T13:39:18.140 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1839 2024-09-12T13:39:18.187 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:39:18.189 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1841 2024-09-12T13:39:18.190 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:39:18.211 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:39:18.264 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:39:18.264 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:39:18.264 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:39:18.264 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:39:18.265 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:39:18.265 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:39:19.154 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1843 2024-09-12T13:39:19.184 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1842 2024-09-12T13:39:19.220 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:39:19.231 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:39:19.347 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:39:19.347 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:39:19.347 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:39:19.347 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:39:19.347 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:39:19.347 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:39:19.347 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:39:19.347 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:39:19.347 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:39:19.347 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:39:19.348 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:39:19.348 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:39:20.259 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1844 2024-09-12T13:39:20.281 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:39:20.306 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:39:20.306 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:39:20.306 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:39:20.306 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:39:20.306 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:39:20.307 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:39:20.325 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:39:20.325 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:39:20.325 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:39:20.325 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:39:20.325 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:39:20.325 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:39:21.190 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:39:21.190 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:39:21.190 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:39:21.190 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:39:21.190 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:39:21.191 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:39:21.195 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:39:21.195 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:39:21.195 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:39:21.195 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:39:21.195 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:39:21.195 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:39:21.216 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:39:21.217 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:39:21.217 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:39:21.217 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:39:21.217 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:39:21.217 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:39:22.224 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:39:22.224 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:39:22.224 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:39:22.225 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:39:22.225 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:39:22.225 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:39:22.238 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:39:22.238 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:39:22.238 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:39:22.238 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:39:22.238 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:39:22.239 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:39:22.271 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1845 2024-09-12T13:39:22.288 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1846 2024-09-12T13:39:22.334 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:39:22.356 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:39:22.387 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1850 2024-09-12T13:39:22.406 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:39:22.408 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1849 2024-09-12T13:39:22.417 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:39:23.182 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1848 2024-09-12T13:39:23.202 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1847 2024-09-12T13:39:23.223 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:39:23.224 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1851 2024-09-12T13:39:23.235 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:39:23.241 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:39:23.283 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:39:23.283 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:39:23.283 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:39:23.284 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:39:23.284 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:39:23.284 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:39:24.292 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1853 2024-09-12T13:39:24.304 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1852 2024-09-12T13:39:24.319 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:39:24.328 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:39:25.336 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:39:25.337 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:39:25.337 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:39:25.337 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:39:25.337 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:39:25.337 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:39:25.359 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:39:25.359 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:39:25.359 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:39:25.359 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:39:25.359 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:39:25.359 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:39:25.408 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:39:25.409 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:39:25.409 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:39:25.409 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:39:25.409 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:39:25.409 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:39:25.420 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:39:25.420 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:39:25.420 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:39:25.420 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:39:25.420 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:39:25.420 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:39:26.226 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:39:26.226 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:39:26.226 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:39:26.227 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:39:26.227 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:39:26.227 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:39:26.238 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:39:26.238 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:39:26.238 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:39:26.239 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:39:26.239 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:39:26.239 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:39:26.244 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:39:26.244 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:39:26.244 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:39:26.244 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:39:26.244 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:39:26.244 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:39:26.343 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1856 2024-09-12T13:39:26.361 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1855 2024-09-12T13:39:26.370 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:39:26.371 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:39:27.325 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:39:27.325 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:39:27.325 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:39:27.325 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:39:27.325 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:39:27.325 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:39:27.332 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:39:27.332 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:39:27.332 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:39:27.332 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:39:27.332 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:39:27.333 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:39:28.131 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1857 2024-09-12T13:39:28.137 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1858 2024-09-12T13:39:28.184 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:39:28.196 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:39:28.245 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1861 2024-09-12T13:39:28.268 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:39:29.135 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1854 2024-09-12T13:39:29.180 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:39:29.185 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1862 2024-09-12T13:39:29.193 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1863 2024-09-12T13:39:29.224 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:39:29.243 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:39:29.373 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:39:29.374 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:39:29.374 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:39:29.374 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:39:29.374 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:39:29.374 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:39:29.375 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:39:29.375 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:39:29.375 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:39:29.375 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:39:29.375 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:39:29.375 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:39:31.150 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1860 2024-09-12T13:39:31.156 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1859 2024-09-12T13:39:31.189 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:39:31.189 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:39:31.189 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:39:31.189 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:39:31.189 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:39:31.189 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:39:31.192 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1865 2024-09-12T13:39:31.200 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:39:31.200 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:39:31.200 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:39:31.200 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:39:31.200 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:39:31.200 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:39:31.205 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:39:31.212 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1864 2024-09-12T13:39:31.213 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:39:31.219 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:39:31.228 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:39:31.272 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:39:31.272 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:39:31.272 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:39:31.272 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:39:31.272 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:39:31.273 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:39:32.179 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1867 2024-09-12T13:39:32.187 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1866 2024-09-12T13:39:32.187 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:39:32.188 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:39:32.188 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:39:32.188 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:39:32.188 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:39:32.188 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:39:32.211 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:39:32.211 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:39:32.228 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:39:32.228 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:39:32.228 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:39:32.228 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:39:32.228 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:39:32.228 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:39:32.246 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:39:32.247 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:39:32.247 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:39:32.247 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:39:32.247 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:39:32.247 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:39:33.176 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1868 2024-09-12T13:39:33.189 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:39:33.192 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1869 2024-09-12T13:39:33.211 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:39:34.183 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1870 2024-09-12T13:39:34.188 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1871 2024-09-12T13:39:34.211 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:39:34.211 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:39:34.212 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:39:34.212 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:39:34.212 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:39:34.212 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:39:34.215 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:39:34.217 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:39:34.217 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:39:34.218 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:39:34.218 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:39:34.218 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:39:34.218 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:39:34.223 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:39:34.223 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:39:34.223 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:39:34.224 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:39:34.224 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:39:34.224 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:39:34.227 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:39:34.235 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:39:34.235 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:39:34.235 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:39:34.235 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:39:34.235 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:39:34.235 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:39:35.184 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1875 2024-09-12T13:39:35.202 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:39:35.202 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1874 2024-09-12T13:39:35.211 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:39:35.214 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:39:35.215 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:39:35.215 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:39:35.215 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:39:35.215 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:39:35.215 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:39:35.215 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:39:35.215 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:39:35.215 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:39:35.215 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:39:35.215 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:39:35.215 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:39:36.190 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1873 2024-09-12T13:39:36.193 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:39:36.193 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:39:36.193 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:39:36.193 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:39:36.194 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:39:36.194 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:39:36.203 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1872 2024-09-12T13:39:36.215 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:39:36.215 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:39:36.215 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:39:36.215 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:39:36.215 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:39:36.215 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:39:36.240 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:39:36.241 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:39:37.206 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1877 2024-09-12T13:39:37.215 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1876 2024-09-12T13:39:37.221 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:39:37.221 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:39:37.221 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:39:37.221 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:39:37.221 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:39:37.221 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:39:37.232 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:39:37.233 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:39:37.233 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:39:37.233 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:39:37.233 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:39:37.233 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:39:37.243 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:39:37.248 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:39:38.187 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1879 2024-09-12T13:39:38.193 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1878 2024-09-12T13:39:38.207 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:39:38.207 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:39:38.207 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:39:38.208 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:39:38.208 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:39:38.208 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:39:38.215 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:39:38.215 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:39:38.215 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:39:38.215 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:39:38.215 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:39:38.215 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:39:38.216 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:39:38.224 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:39:39.198 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1880 2024-09-12T13:39:39.203 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1881 2024-09-12T13:39:39.228 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:39:39.239 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:39:39.246 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:39:39.246 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:39:39.247 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:39:39.247 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:39:39.247 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:39:39.247 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:39:39.247 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:39:39.247 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:39:39.247 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:39:39.247 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:39:39.247 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:39:39.247 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:39:40.205 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1882 2024-09-12T13:39:40.221 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1883 2024-09-12T13:39:40.232 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:39:40.232 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:39:40.248 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:39:40.248 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:39:40.248 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:39:40.248 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:39:40.248 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:39:40.248 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:39:40.252 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:39:40.252 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:39:40.253 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:39:40.253 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:39:40.253 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:39:40.253 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:39:41.212 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1884 2024-09-12T13:39:41.221 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:39:41.221 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:39:41.222 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:39:41.222 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:39:41.222 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:39:41.222 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:39:41.231 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:39:41.231 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:39:41.231 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:39:41.231 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:39:41.231 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:39:41.231 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:39:41.232 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1885 2024-09-12T13:39:41.248 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:39:41.253 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:39:42.217 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1886 2024-09-12T13:39:42.223 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1887 2024-09-12T13:39:42.233 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:39:42.233 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:39:42.233 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:39:42.233 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:39:42.233 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:39:42.233 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:39:42.242 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:39:42.243 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:39:42.243 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:39:42.243 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:39:42.243 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:39:42.243 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:39:42.245 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:39:42.255 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:39:43.177 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1888 2024-09-12T13:39:43.184 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1889 2024-09-12T13:39:43.209 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:39:43.223 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:39:43.237 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:39:43.237 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:39:43.237 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:39:43.237 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:39:43.237 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:39:43.237 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:39:43.237 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:39:43.237 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:39:43.237 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:39:43.237 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:39:43.238 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:39:43.238 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:39:44.237 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1891 2024-09-12T13:39:44.244 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1890 2024-09-12T13:39:44.252 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:39:44.252 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:39:44.253 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:39:44.253 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:39:44.253 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:39:44.253 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:39:44.257 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:39:44.257 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:39:44.257 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:39:44.257 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:39:44.257 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:39:44.257 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:39:44.431 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:39:44.442 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:39:45.249 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:39:45.249 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:39:45.249 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:39:45.249 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:39:45.249 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:39:45.249 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:39:45.261 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:39:45.261 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:39:45.261 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:39:45.261 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:39:45.261 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:39:45.261 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:39:46.196 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1892 2024-09-12T13:39:46.214 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:39:46.214 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:39:46.214 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:39:46.215 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:39:46.215 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:39:46.215 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:39:46.215 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1893 2024-09-12T13:39:46.228 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:39:46.228 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:39:46.228 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:39:46.228 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:39:46.228 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:39:46.228 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:39:46.248 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:39:46.269 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:39:47.210 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1895 2024-09-12T13:39:47.454 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1894 2024-09-12T13:39:47.455 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:39:47.456 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:39:47.456 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:39:47.456 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:39:47.456 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:39:47.456 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:39:47.456 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:39:47.456 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:39:47.457 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:39:47.457 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:39:47.457 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:39:47.457 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:39:47.457 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:39:47.457 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:39:48.163 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1896 2024-09-12T13:39:48.167 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1897 2024-09-12T13:39:48.209 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:39:48.215 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:39:48.216 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:39:48.217+0000 7f67dc600640 -1 rbd::mirror::image_replayer::snapshot::Replayer: 0x56055c10f400 handle_unlink_peer: failed to unlink local peer from remote image: (22) Invalid argument 2024-09-12T13:39:49.208 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1898 2024-09-12T13:39:49.213 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1899 2024-09-12T13:39:49.251 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:39:49.252 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:39:49.252 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:39:49.252 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:39:49.252 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:39:49.252 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:39:49.252 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:39:49.270 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:39:49.272 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:39:49.272 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:39:49.272 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:39:49.272 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:39:49.272 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:39:49.272 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:39:50.209 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1901 2024-09-12T13:39:50.226 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1900 2024-09-12T13:39:50.247 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:39:50.259 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:39:50.273 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1903 2024-09-12T13:39:50.274 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:39:50.274 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:39:50.274 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:39:50.274 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:39:50.275 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:39:50.275 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:39:50.277 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:39:50.277 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:39:50.277 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:39:50.278 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:39:50.278 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:39:50.278 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:39:50.282 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1902 2024-09-12T13:39:50.291 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:39:50.292 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:39:51.213 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:39:51.213 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:39:51.213 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:39:51.213 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:39:51.213 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:39:51.213 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:39:51.219 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:39:51.219 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:39:51.219 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:39:51.219 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:39:51.219 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:39:51.219 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:39:51.235 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1905 2024-09-12T13:39:51.239 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1904 2024-09-12T13:39:51.251 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:39:51.254 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:39:52.233 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1907 2024-09-12T13:39:52.239 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1906 2024-09-12T13:39:52.247 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:39:52.254 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:39:52.254 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:39:52.254 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:39:52.254 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:39:52.254 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:39:52.254 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:39:52.254 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:39:52.272 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:39:52.273 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:39:52.273 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:39:52.273 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:39:52.273 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:39:52.273 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:39:53.178 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1909 2024-09-12T13:39:53.179 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1908 2024-09-12T13:39:53.200 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:39:53.201 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:39:53.250 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:39:53.250 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:39:53.250 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:39:53.250 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:39:53.250 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:39:53.250 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:39:53.262 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:39:53.262 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:39:53.262 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:39:53.262 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:39:53.262 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:39:53.262 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:39:53.294 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:39:53.294 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:39:53.294 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:39:53.294 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:39:53.294 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:39:53.294 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:39:53.295 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:39:53.295 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:39:53.295 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:39:53.295 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:39:53.295 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:39:53.295 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:39:54.254 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:39:54.254 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:39:54.254 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:39:54.255 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:39:54.255 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:39:54.255 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:39:54.257 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:39:54.257 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:39:54.257 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:39:54.257 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:39:54.257 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:39:54.258 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:39:54.259 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1911 2024-09-12T13:39:54.261 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1910 2024-09-12T13:39:54.279 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:39:54.280 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:39:55.250 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:39:55.250 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:39:55.251 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:39:55.251 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:39:55.251 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:39:55.251 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:39:55.256 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:39:55.256 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:39:55.256 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:39:55.256 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:39:55.257 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:39:55.257 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:39:56.204 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:39:56.204 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:39:56.204 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:39:56.204 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:39:56.204 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:39:56.204 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:39:56.205 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:39:56.205 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:39:56.205 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:39:56.205 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:39:56.205 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:39:56.205 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:39:56.251 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1912 2024-09-12T13:39:56.256 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1913 2024-09-12T13:39:56.304 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:39:56.326 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:39:57.252 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1914 2024-09-12T13:39:57.256 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1915 2024-09-12T13:39:57.284 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:39:57.284 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:39:57.284 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:39:57.284 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:39:57.284 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:39:57.284 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:39:57.284 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:39:57.284 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:39:57.285 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:39:57.285 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:39:57.285 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:39:57.285 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:39:57.304 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:39:57.308 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:39:57.366 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1919 2024-09-12T13:39:57.423 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:39:58.150 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1917 2024-09-12T13:39:58.153 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1916 2024-09-12T13:39:58.191 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1918 2024-09-12T13:39:58.197 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:39:58.201 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:39:58.211 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:39:59.265 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:39:59.265+0000 7f67dd000640 -1 rbd::mirror::image_replayer::snapshot::Replayer: 0x56055c2e0800 handle_unlink_peer: failed to unlink local peer from remote image: (22) Invalid argument 2024-09-12T13:39:59.307 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:39:59.307 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:39:59.307 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:39:59.307 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:39:59.307 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:39:59.307 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:39:59.329 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:39:59.329 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:39:59.329 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:39:59.329 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:39:59.329 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:39:59.329 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:40:00.240 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1921 2024-09-12T13:40:00.249 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1920 2024-09-12T13:40:00.274 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:40:00.279 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:40:00.303 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1923 2024-09-12T13:40:00.306 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:40:00.306 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:40:00.306 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:40:00.306 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:40:00.306 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:40:00.306 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:40:00.310 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:40:00.310 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:40:00.310 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:40:00.310 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:40:00.310 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:40:00.310 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:40:00.324 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:40:00.333 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1922 2024-09-12T13:40:00.343 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:40:00.426 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:40:00.426 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:40:00.426 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:40:00.426 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:40:00.426 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:40:00.426 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:40:01.200 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:40:01.200 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:40:01.200 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:40:01.200 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:40:01.200 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:40:01.200 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:40:01.204 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:40:01.204 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:40:01.204 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:40:01.204 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:40:01.204 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:40:01.204 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:40:01.213 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:40:01.213 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:40:01.213 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:40:01.213 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:40:01.214 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:40:01.214 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:40:01.272 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1924 2024-09-12T13:40:01.277 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1925 2024-09-12T13:40:01.280 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1926 2024-09-12T13:40:01.291 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:40:01.300 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:40:01.301 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:40:02.287 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1927 2024-09-12T13:40:02.288 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1928 2024-09-12T13:40:02.303 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:40:02.304 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:40:03.170 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1929 2024-09-12T13:40:03.187 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:40:03.277 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:40:03.277 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:40:03.277 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:40:03.277 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:40:03.277 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:40:03.277 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:40:03.282 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:40:03.282 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:40:03.282 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:40:03.282 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:40:03.282 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:40:03.282 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:40:03.327 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:40:03.327 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:40:03.327 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:40:03.327 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:40:03.327 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:40:03.328 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:40:03.345 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:40:03.345 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:40:03.345 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:40:03.346 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:40:03.346 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:40:03.346 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:40:04.284 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1930 2024-09-12T13:40:04.294 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:40:04.294 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:40:04.294 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:40:04.295 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:40:04.295 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:40:04.295 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:40:04.297 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1931 2024-09-12T13:40:04.303 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:40:04.303 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:40:04.303 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:40:04.304 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:40:04.304 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:40:04.304 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:40:04.304 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:40:04.304 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:40:04.304 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:40:04.304 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:40:04.304 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:40:04.304 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:40:04.317 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:40:04.327 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:40:05.307 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:40:05.307 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:40:05.307 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:40:05.307 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:40:05.307 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:40:05.307 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:40:05.307 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:40:05.308 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:40:05.308 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:40:05.308 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:40:05.308 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:40:05.308 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:40:05.342 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1934 2024-09-12T13:40:05.353 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:40:06.191 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:40:06.192 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:40:06.192 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:40:06.192 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:40:06.192 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:40:06.193 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:40:06.266 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1932 2024-09-12T13:40:06.278 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1933 2024-09-12T13:40:06.316 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:40:06.325 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:40:06.333 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:40:06.333+0000 7f67dd000640 -1 rbd::mirror::image_replayer::snapshot::Replayer: 0x56055c17e800 handle_unlink_peer: failed to unlink local peer from remote image: (22) Invalid argument 2024-09-12T13:40:06.352 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:40:06.352+0000 7f67dd000640 -1 rbd::mirror::image_replayer::snapshot::Replayer: 0x5605562aa000 handle_unlink_peer: failed to unlink local peer from remote image: (22) Invalid argument 2024-09-12T13:40:07.273 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1935 2024-09-12T13:40:07.277 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1936 2024-09-12T13:40:07.316 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:40:07.320 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:40:07.320 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:40:07.320 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:40:07.321 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:40:07.321 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:40:07.321 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:40:07.329 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:40:07.329 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:40:07.330 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:40:07.330 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:40:07.330 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:40:07.330 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:40:07.339 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:40:08.142 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1937 2024-09-12T13:40:08.148 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1938 2024-09-12T13:40:08.189 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1939 2024-09-12T13:40:08.208 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:40:08.211 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:40:08.226 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:40:08.238 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1940 2024-09-12T13:40:08.259 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:40:08.356 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:40:08.356 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:40:08.356 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:40:08.356 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:40:08.356 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:40:08.356 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:40:09.319 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:40:09.319 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:40:09.319 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:40:09.320 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:40:09.320 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:40:09.320 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:40:09.328 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:40:09.328 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:40:09.328 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:40:09.329 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:40:09.329 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:40:09.329 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:40:10.278 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1942 2024-09-12T13:40:10.283 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1941 2024-09-12T13:40:10.319 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:40:10.319 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:40:10.319 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:40:10.319 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:40:10.320 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:40:10.320 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:40:10.320 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:40:10.321 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:40:10.337 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1944 2024-09-12T13:40:10.342 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:40:10.342 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:40:10.342 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:40:10.342 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:40:10.342 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:40:10.342 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:40:10.355 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:40:10.437 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1943 2024-09-12T13:40:10.448 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:40:11.213 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:40:11.213 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:40:11.213 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:40:11.213 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:40:11.213 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:40:11.213 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:40:11.215 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:40:11.215 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:40:11.215 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:40:11.215 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:40:11.215 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:40:11.215 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:40:11.229 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:40:11.230 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:40:11.230 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:40:11.230 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:40:11.230 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:40:11.230 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:40:11.261 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:40:11.261 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:40:11.261 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:40:11.261 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:40:11.262 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:40:11.262 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:40:11.300 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1946 2024-09-12T13:40:11.304 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1945 2024-09-12T13:40:11.320 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:40:11.326 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:40:12.316 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1950 2024-09-12T13:40:12.333 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:40:12.335 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1947 2024-09-12T13:40:12.345 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:40:12.520 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1948 2024-09-12T13:40:12.533 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:40:13.154 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1949 2024-09-12T13:40:13.169 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:40:13.323 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:40:13.323 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:40:13.323 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:40:13.323 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:40:13.323 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:40:13.323 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:40:13.324 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:40:13.324 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:40:13.324 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:40:13.324 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:40:13.324 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:40:13.324 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:40:13.357 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:40:13.357 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:40:13.358 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:40:13.358 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:40:13.358 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:40:13.358 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:40:13.451 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:40:13.451 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:40:13.451 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:40:13.451 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:40:13.452 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:40:13.452 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:40:14.319 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1952 2024-09-12T13:40:14.323 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:40:14.323 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:40:14.324 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:40:14.324 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:40:14.324 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:40:14.324 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:40:14.328 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:40:14.328 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:40:14.328 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:40:14.328 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:40:14.329 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:40:14.329 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:40:14.339 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:40:15.295 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1951 2024-09-12T13:40:15.336 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:40:15.336 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:40:15.337 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:40:15.337 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:40:15.337 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:40:15.337 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:40:15.347 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:40:15.348 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:40:15.348 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:40:15.348 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:40:15.348 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:40:15.348 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:40:15.356 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:40:15.381 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1955 2024-09-12T13:40:15.384 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1956 2024-09-12T13:40:15.401 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:40:15.402 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:40:15.536 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:40:15.536 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:40:15.536 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:40:15.536 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:40:15.537 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:40:15.537 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:40:16.172 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${SNAP_PIDS[@]} 2024-09-12T13:40:16.172 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 94667 2024-09-12T13:40:16.302 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1954 2024-09-12T13:40:16.315 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1953 2024-09-12T13:40:16.345 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:40:16.346 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:40:16.399 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1957 2024-09-12T13:40:16.411 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:40:17.341 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${SNAP_PIDS[@]} 2024-09-12T13:40:17.341 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 94799 2024-09-12T13:40:18.179 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1958 2024-09-12T13:40:18.214 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1959 2024-09-12T13:40:18.214 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:40:18.215 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:40:18.405 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:40:18.405 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:40:18.405 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:40:18.406 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:40:18.406 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:40:18.406 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:40:19.348 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:40:19.348 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:40:19.348 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:40:19.349 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:40:19.349 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:40:19.349 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:40:19.349 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${SNAP_PIDS[@]} 2024-09-12T13:40:19.349 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 94931 2024-09-12T13:40:19.349 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${SNAP_PIDS[@]} 2024-09-12T13:40:19.349 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 95083 2024-09-12T13:40:20.331 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1960 2024-09-12T13:40:20.347 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:40:20.354 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1961 2024-09-12T13:40:20.368 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:40:21.198 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${SNAP_PIDS[@]} 2024-09-12T13:40:21.198 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 95236 2024-09-12T13:40:21.198 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${SNAP_PIDS[@]} 2024-09-12T13:40:21.198 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 95389 2024-09-12T13:40:21.198 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${SNAP_PIDS[@]} 2024-09-12T13:40:21.198 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 95542 2024-09-12T13:40:21.198 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:40:21.198 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:40:21.198 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:40:21.198 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:40:21.199 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:40:21.199 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:40:22.333 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1962 2024-09-12T13:40:22.347 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:40:23.371 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${SNAP_PIDS[@]} 2024-09-12T13:40:23.372 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 95715 2024-09-12T13:40:23.372 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${SNAP_PIDS[@]} 2024-09-12T13:40:23.372 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 95889 2024-09-12T13:40:25.350 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ (( RET != 0 )) 2024-09-12T13:40:25.350 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${WORKLOAD_PIDS[@]} 2024-09-12T13:40:25.350 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 94536 2024-09-12T13:42:51.913 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ ret=124 2024-09-12T13:42:51.913 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ (( ret != 124 )) 2024-09-12T13:42:51.913 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${WORKLOAD_PIDS[@]} 2024-09-12T13:42:51.913 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 94668 2024-09-12T13:42:52.834 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ ret=124 2024-09-12T13:42:52.834 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ (( ret != 124 )) 2024-09-12T13:42:52.835 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${WORKLOAD_PIDS[@]} 2024-09-12T13:42:52.835 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 94800 2024-09-12T13:42:53.820 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ ret=124 2024-09-12T13:42:53.820 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ (( ret != 124 )) 2024-09-12T13:42:53.820 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${WORKLOAD_PIDS[@]} 2024-09-12T13:42:53.820 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 94932 2024-09-12T13:42:54.887 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ ret=124 2024-09-12T13:42:54.887 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ (( ret != 124 )) 2024-09-12T13:42:54.887 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${WORKLOAD_PIDS[@]} 2024-09-12T13:42:54.887 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 95084 2024-09-12T13:42:56.024 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ ret=124 2024-09-12T13:42:56.025 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ (( ret != 124 )) 2024-09-12T13:42:56.025 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${WORKLOAD_PIDS[@]} 2024-09-12T13:42:56.025 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 95237 2024-09-12T13:42:57.143 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ ret=124 2024-09-12T13:42:57.143 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ (( ret != 124 )) 2024-09-12T13:42:57.143 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${WORKLOAD_PIDS[@]} 2024-09-12T13:42:57.143 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 95390 2024-09-12T13:42:58.078 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ ret=124 2024-09-12T13:42:58.079 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ (( ret != 124 )) 2024-09-12T13:42:58.079 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${WORKLOAD_PIDS[@]} 2024-09-12T13:42:58.079 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 95543 2024-09-12T13:42:58.863 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ ret=124 2024-09-12T13:42:58.863 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ (( ret != 124 )) 2024-09-12T13:42:58.863 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${WORKLOAD_PIDS[@]} 2024-09-12T13:42:58.864 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 95716 2024-09-12T13:43:00.020 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ ret=124 2024-09-12T13:43:00.021 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ (( ret != 124 )) 2024-09-12T13:43:00.021 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${WORKLOAD_PIDS[@]} 2024-09-12T13:43:00.021 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 95890 2024-09-12T13:43:01.012 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ ret=124 2024-09-12T13:43:01.012 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ (( ret != 124 )) 2024-09-12T13:43:01.013 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ (( RET != 0 )) 2024-09-12T13:43:01.013 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:43:01.014 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ COMPARE_PIDS+=($!) 2024-09-12T13:43:01.014 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:43:01.014 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ compare_demoted_promoted_image 1 2024-09-12T13:43:01.015 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local dev=/dev/rbd0 2024-09-12T13:43:01.015 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local img=image-primary1 2024-09-12T13:43:01.015 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mntpt=test-primary1 2024-09-12T13:43:01.015 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local demote_md5 promote_md5 2024-09-12T13:43:01.015 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo umount test-primary1 2024-09-12T13:43:01.015 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ COMPARE_PIDS+=($!) 2024-09-12T13:43:01.015 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:43:01.015 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ compare_demoted_promoted_image 2 2024-09-12T13:43:01.015 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local dev=/dev/rbd1 2024-09-12T13:43:01.015 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local img=image-primary2 2024-09-12T13:43:01.015 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mntpt=test-primary2 2024-09-12T13:43:01.015 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local demote_md5 promote_md5 2024-09-12T13:43:01.015 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo umount test-primary2 2024-09-12T13:43:01.016 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ COMPARE_PIDS+=($!) 2024-09-12T13:43:01.016 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:43:01.016 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ compare_demoted_promoted_image 3 2024-09-12T13:43:01.018 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local dev=/dev/rbd2 2024-09-12T13:43:01.018 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local img=image-primary3 2024-09-12T13:43:01.018 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mntpt=test-primary3 2024-09-12T13:43:01.018 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local demote_md5 promote_md5 2024-09-12T13:43:01.018 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo umount test-primary3 2024-09-12T13:43:01.018 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ COMPARE_PIDS+=($!) 2024-09-12T13:43:01.018 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:43:01.018 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ compare_demoted_promoted_image 4 2024-09-12T13:43:01.018 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local dev=/dev/rbd3 2024-09-12T13:43:01.018 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local img=image-primary4 2024-09-12T13:43:01.018 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mntpt=test-primary4 2024-09-12T13:43:01.018 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ COMPARE_PIDS+=($!) 2024-09-12T13:43:01.019 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local demote_md5 promote_md5 2024-09-12T13:43:01.019 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo umount test-primary4 2024-09-12T13:43:01.019 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:43:01.019 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ compare_demoted_promoted_image 5 2024-09-12T13:43:01.019 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local dev=/dev/rbd4 2024-09-12T13:43:01.019 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local img=image-primary5 2024-09-12T13:43:01.019 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mntpt=test-primary5 2024-09-12T13:43:01.019 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local demote_md5 promote_md5 2024-09-12T13:43:01.019 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo umount test-primary5 2024-09-12T13:43:01.019 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ COMPARE_PIDS+=($!) 2024-09-12T13:43:01.019 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:43:01.019 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ compare_demoted_promoted_image 6 2024-09-12T13:43:01.021 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local dev=/dev/rbd5 2024-09-12T13:43:01.021 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ COMPARE_PIDS+=($!) 2024-09-12T13:43:01.022 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local img=image-primary6 2024-09-12T13:43:01.022 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mntpt=test-primary6 2024-09-12T13:43:01.022 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local demote_md5 promote_md5 2024-09-12T13:43:01.022 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo umount test-primary6 2024-09-12T13:43:01.022 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:43:01.022 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ compare_demoted_promoted_image 7 2024-09-12T13:43:01.022 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local dev=/dev/rbd6 2024-09-12T13:43:01.022 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local img=image-primary7 2024-09-12T13:43:01.022 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mntpt=test-primary7 2024-09-12T13:43:01.022 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local demote_md5 promote_md5 2024-09-12T13:43:01.022 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo umount test-primary7 2024-09-12T13:43:01.022 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ COMPARE_PIDS+=($!) 2024-09-12T13:43:01.022 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:43:01.023 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ compare_demoted_promoted_image 8 2024-09-12T13:43:01.023 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local dev=/dev/rbd7 2024-09-12T13:43:01.023 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ COMPARE_PIDS+=($!) 2024-09-12T13:43:01.023 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local img=image-primary8 2024-09-12T13:43:01.023 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mntpt=test-primary8 2024-09-12T13:43:01.023 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local demote_md5 promote_md5 2024-09-12T13:43:01.023 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo umount test-primary8 2024-09-12T13:43:01.024 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ compare_demoted_promoted_image 9 2024-09-12T13:43:01.024 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:43:01.024 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ COMPARE_PIDS+=($!) 2024-09-12T13:43:01.024 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local dev=/dev/rbd8 2024-09-12T13:43:01.024 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local img=image-primary9 2024-09-12T13:43:01.025 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mntpt=test-primary9 2024-09-12T13:43:01.025 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local demote_md5 promote_md5 2024-09-12T13:43:01.025 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo umount test-primary9 2024-09-12T13:43:01.025 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${COMPARE_PIDS[@]} 2024-09-12T13:43:01.025 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 101846 2024-09-12T13:43:01.027 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ compare_demoted_promoted_image 10 2024-09-12T13:43:01.028 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local dev=/dev/rbd9 2024-09-12T13:43:01.028 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local img=image-primary10 2024-09-12T13:43:01.028 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mntpt=test-primary10 2024-09-12T13:43:01.028 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local demote_md5 promote_md5 2024-09-12T13:43:01.028 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo umount test-primary10 2024-09-12T13:43:01.642 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo md5sum /dev/rbd1 2024-09-12T13:43:01.646 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ awk '{print $1}' 2024-09-12T13:43:01.665 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo md5sum /dev/rbd0 2024-09-12T13:43:01.668 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ awk '{print $1}' 2024-09-12T13:43:01.677 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo md5sum /dev/rbd4 2024-09-12T13:43:01.684 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ awk '{print $1}' 2024-09-12T13:43:01.705 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo md5sum /dev/rbd2 2024-09-12T13:43:01.708 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ awk '{print $1}' 2024-09-12T13:43:01.805 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo md5sum /dev/rbd7 2024-09-12T13:43:01.806 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ awk '{print $1}' 2024-09-12T13:43:01.871 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo md5sum /dev/rbd8 2024-09-12T13:43:01.876 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ awk '{print $1}' 2024-09-12T13:43:01.877 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo md5sum /dev/rbd9 2024-09-12T13:43:01.877 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ awk '{print $1}' 2024-09-12T13:43:01.925 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo md5sum /dev/rbd6 2024-09-12T13:43:01.928 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ awk '{print $1}' 2024-09-12T13:43:01.933 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ awk '{print $1}' 2024-09-12T13:43:01.936 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo md5sum /dev/rbd3 2024-09-12T13:43:01.992 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ awk '{print $1}' 2024-09-12T13:43:01.992 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo md5sum /dev/rbd5 2024-09-12T13:43:52.202 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ demote_md5=eeb66717e02189ca91f864aa17d93a8f 2024-09-12T13:43:52.203 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo rbd --cluster cluster1 device unmap -t krbd mirror/image-primary9 2024-09-12T13:43:52.862 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ demote_image cluster1 mirror image-primary9 2024-09-12T13:43:52.862 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:43:52.862 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:43:52.862 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:43:52.862 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 mirror image demote mirror/image-primary9 2024-09-12T13:43:54.323 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ demote_md5=2b6769f814133ccdb1f87119664c5c38 2024-09-12T13:43:54.323 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo rbd --cluster cluster1 device unmap -t krbd mirror/image-primary2 2024-09-12T13:43:54.537 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ demote_image cluster1 mirror image-primary2 2024-09-12T13:43:54.537 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:43:54.538 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:43:54.538 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:43:54.538 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 mirror image demote mirror/image-primary2 2024-09-12T13:43:55.240 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Image demoted to non-primary 2024-09-12T13:43:55.274 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary9 up+unknown 2024-09-12T13:43:55.274 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:43:55.274 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:43:55.274 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:43:55.274 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:43:55.275 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:43:55.275 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:43:55.275 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:43:55.275 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:43:55.423 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ demote_md5=2bfe718220eddfa00fb7325b61ba4af3 2024-09-12T13:43:55.423 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo rbd --cluster cluster1 device unmap -t krbd mirror/image-primary6 2024-09-12T13:43:55.660 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ demote_image cluster1 mirror image-primary6 2024-09-12T13:43:55.660 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:43:55.660 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:43:55.660 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:43:55.660 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 mirror image demote mirror/image-primary6 2024-09-12T13:43:55.866 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ demote_md5=e4e69be8d6a6fb00c74d3e090d096b6e 2024-09-12T13:43:55.866 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo rbd --cluster cluster1 device unmap -t krbd mirror/image-primary7 2024-09-12T13:43:56.207 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ demote_image cluster1 mirror image-primary7 2024-09-12T13:43:56.208 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:43:56.208 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:43:56.208 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:43:56.208 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 mirror image demote mirror/image-primary7 2024-09-12T13:43:56.251 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Image demoted to non-primary 2024-09-12T13:43:56.301 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary9 up+unknown '' '' 2024-09-12T13:43:56.301 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:43:56.301 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:43:56.302 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:43:56.302 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:43:56.302 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:43:56.302 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:43:56.302 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:43:56.325 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:43:56.325 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary9 2024-09-12T13:43:56.342 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary2 up+unknown 2024-09-12T13:43:56.343 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:43:56.343 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:43:56.343 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:43:56.343 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:43:56.344 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:43:56.344 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:43:56.344 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:43:56.344 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:43:56.638 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary9: 2024-09-12T13:43:56.638 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: b98c3fed-4807-46cf-b9ef-4cbefc529d10 2024-09-12T13:43:56.638 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:43:56.638 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:43:56.638 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:43:56.638 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:43:47' 2024-09-12T13:43:56.639 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:43:56.648 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:43:56.648 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:43:56.648 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 2 2024-09-12T13:43:57.189 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ demote_md5=78e3674d0ba59fb198419c279aaa3a35 2024-09-12T13:43:57.190 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo rbd --cluster cluster1 device unmap -t krbd mirror/image-primary8 2024-09-12T13:43:57.230 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ demote_md5=ab12518a14de3ef37d42312f3d912429 2024-09-12T13:43:57.230 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo rbd --cluster cluster1 device unmap -t krbd mirror/image-primary3 2024-09-12T13:43:57.246 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ demote_md5=d538fc510fb812e420e71721db55e508 2024-09-12T13:43:57.246 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo rbd --cluster cluster1 device unmap -t krbd mirror/image-primary4 2024-09-12T13:43:57.342 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ demote_md5=9ea431d0695a08b4407dbabbb30171c8 2024-09-12T13:43:57.342 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo rbd --cluster cluster1 device unmap -t krbd mirror/image-primary5 2024-09-12T13:43:57.376 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary2 up+unknown '' '' 2024-09-12T13:43:57.377 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:43:57.377 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:43:57.377 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:43:57.377 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:43:57.377 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:43:57.377 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:43:57.378 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:43:57.383 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:43:57.383 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary2 2024-09-12T13:43:57.453 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ demote_image cluster1 mirror image-primary8 2024-09-12T13:43:57.454 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:43:57.454 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:43:57.454 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:43:57.454 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 mirror image demote mirror/image-primary8 2024-09-12T13:43:57.484 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ demote_image cluster1 mirror image-primary4 2024-09-12T13:43:57.484 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:43:57.485 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:43:57.485 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:43:57.485 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 mirror image demote mirror/image-primary4 2024-09-12T13:43:57.514 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ demote_image cluster1 mirror image-primary3 2024-09-12T13:43:57.515 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:43:57.515 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:43:57.515 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:43:57.515 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 mirror image demote mirror/image-primary3 2024-09-12T13:43:57.542 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ demote_image cluster1 mirror image-primary5 2024-09-12T13:43:57.542 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:43:57.542 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:43:57.542 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:43:57.542 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 mirror image demote mirror/image-primary5 2024-09-12T13:43:57.566 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary2: 2024-09-12T13:43:57.566 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 603c11d9-89c3-4bd2-8c39-0d4ed9ed3093 2024-09-12T13:43:57.566 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:43:57.567 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:43:57.567 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:43:57.567 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:43:47' 2024-09-12T13:43:57.567 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:43:57.572 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:43:57.572 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:43:57.572 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 2 2024-09-12T13:43:57.776 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ demote_md5=0bb624ca6718db88f78b6d0e3552396c 2024-09-12T13:43:57.776 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo rbd --cluster cluster1 device unmap -t krbd mirror/image-primary10 2024-09-12T13:43:57.924 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ demote_image cluster1 mirror image-primary10 2024-09-12T13:43:57.924 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:43:57.924 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:43:57.925 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:43:57.925 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 mirror image demote mirror/image-primary10 2024-09-12T13:43:58.266 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Image demoted to non-primary 2024-09-12T13:43:58.304 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary5 up+unknown 2024-09-12T13:43:58.305 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:43:58.305 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:43:58.305 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:43:58.305 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:43:58.305 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:43:58.305 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:43:58.305 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:43:58.305 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:43:58.311 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Image demoted to non-primary 2024-09-12T13:43:58.335 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary3 up+unknown 2024-09-12T13:43:58.335 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:43:58.335 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:43:58.335 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:43:58.335 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:43:58.335 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:43:58.335 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:43:58.335 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:43:58.336 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:43:58.653 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary9 up+unknown '' '' 2024-09-12T13:43:58.653 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:43:58.653 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:43:58.653 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:43:58.653 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:43:58.653 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:43:58.653 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:43:58.653 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:43:58.654 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:43:58.654 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary9 2024-09-12T13:43:58.721 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary9: 2024-09-12T13:43:58.721 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: b98c3fed-4807-46cf-b9ef-4cbefc529d10 2024-09-12T13:43:58.721 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:43:58.721 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:43:58.721 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:43:58.722 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:43:47' 2024-09-12T13:43:58.722 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:43:58.724 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:43:58.724 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:43:58.724 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 4 2024-09-12T13:43:59.080 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ demote_md5=4ef5fd590827cc3f057aaa2cbbc80493 2024-09-12T13:43:59.080 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo rbd --cluster cluster1 device unmap -t krbd mirror/image-primary1 2024-09-12T13:43:59.192 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ demote_image cluster1 mirror image-primary1 2024-09-12T13:43:59.192 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:43:59.193 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:43:59.193 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:43:59.193 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 mirror image demote mirror/image-primary1 2024-09-12T13:43:59.309 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary5 up+unknown '' '' 2024-09-12T13:43:59.309 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:43:59.309 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:43:59.309 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:43:59.310 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:43:59.310 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:43:59.310 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:43:59.310 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:43:59.315 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:43:59.315 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary5 2024-09-12T13:43:59.340 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary3 up+unknown '' '' 2024-09-12T13:43:59.340 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:43:59.341 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:43:59.341 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:43:59.342 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:43:59.342 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:43:59.342 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:43:59.342 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:43:59.344 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:43:59.344 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary3 2024-09-12T13:43:59.364 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Image demoted to non-primary 2024-09-12T13:43:59.366 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Image demoted to non-primary 2024-09-12T13:43:59.408 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary8 up+unknown 2024-09-12T13:43:59.409 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:43:59.409 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:43:59.409 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:43:59.409 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:43:59.409 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:43:59.409 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:43:59.409 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:43:59.409 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:43:59.420 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary10 up+unknown 2024-09-12T13:43:59.420 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:43:59.420 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:43:59.420 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:43:59.420 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:43:59.420 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:43:59.420 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:43:59.421 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:43:59.421 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:43:59.569 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary3: 2024-09-12T13:43:59.569 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 6eed8ffb-1fae-4e8b-9794-cf7af72eafbc 2024-09-12T13:43:59.569 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:43:59.569 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:43:59.569 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:43:59.570 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:43:47' 2024-09-12T13:43:59.570 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:43:59.576 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:43:59.577 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:43:59.577 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 2 2024-09-12T13:43:59.582 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary2 up+unknown '' '' 2024-09-12T13:43:59.582 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:43:59.582 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:43:59.583 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:43:59.583 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:43:59.583 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:43:59.583 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:43:59.583 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:43:59.584 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary5: 2024-09-12T13:43:59.584 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 19fef5ab-79b5-4712-8250-75fffa450f0a 2024-09-12T13:43:59.584 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:43:59.585 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:43:59.585 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:43:59.585 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:43:47' 2024-09-12T13:43:59.586 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:43:59.586 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:43:59.587 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary2 2024-09-12T13:43:59.598 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:43:59.598 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:43:59.598 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 2 2024-09-12T13:43:59.774 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary2: 2024-09-12T13:43:59.774 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 603c11d9-89c3-4bd2-8c39-0d4ed9ed3093 2024-09-12T13:43:59.775 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:43:59.776 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:43:59.776 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:43:59.776 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:43:47' 2024-09-12T13:43:59.777 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:43:59.780 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:43:59.780 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:43:59.780 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 4 2024-09-12T13:44:00.282 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Image demoted to non-primary 2024-09-12T13:44:00.282 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Image demoted to non-primary 2024-09-12T13:44:00.301 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary6 up+unknown 2024-09-12T13:44:00.301 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:44:00.301 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:44:00.302 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:44:00.302 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:44:00.302 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:44:00.302 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:44:00.302 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:44:00.302 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:44:00.302 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary4 up+unknown 2024-09-12T13:44:00.302 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:44:00.302 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:44:00.302 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:44:00.302 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:44:00.302 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:44:00.302 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:44:00.302 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:44:00.303 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:44:00.412 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary8 up+unknown '' '' 2024-09-12T13:44:00.413 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:44:00.413 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:44:00.413 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:44:00.413 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:44:00.413 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:44:00.413 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:44:00.413 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:44:00.414 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:44:00.414 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary8 2024-09-12T13:44:00.423 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary10 up+unknown '' '' 2024-09-12T13:44:00.423 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:44:00.424 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:44:00.424 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:44:00.424 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:44:00.424 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:44:00.424 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:44:00.424 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:44:00.424 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:44:00.425 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary10 2024-09-12T13:44:00.485 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary8: 2024-09-12T13:44:00.485 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 11646481-daec-461e-b7dc-ed4d9a9fd543 2024-09-12T13:44:00.485 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:44:00.485 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:44:00.485 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:44:00.485 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:43:47' 2024-09-12T13:44:00.485 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:44:00.488 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:44:00.488 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:44:00.488 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 2 2024-09-12T13:44:00.494 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary10: 2024-09-12T13:44:00.494 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 2e48042f-ba7f-45c0-9287-a6bae29dd722 2024-09-12T13:44:00.494 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:44:00.494 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:44:00.495 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:44:00.495 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:43:47' 2024-09-12T13:44:00.495 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:44:00.497 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:44:00.497 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:44:00.497 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 2 2024-09-12T13:44:01.286 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Image demoted to non-primary 2024-09-12T13:44:01.294 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Image demoted to non-primary 2024-09-12T13:44:01.304 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary6 up+unknown '' '' 2024-09-12T13:44:01.304 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:44:01.304 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:44:01.304 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:44:01.304 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:44:01.304 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:44:01.304 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:44:01.304 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:44:01.305 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:44:01.305 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary6 2024-09-12T13:44:01.309 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary4 up+unknown '' '' 2024-09-12T13:44:01.310 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary1 up+unknown 2024-09-12T13:44:01.310 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:44:01.310 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:44:01.310 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:44:01.310 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:44:01.310 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:44:01.310 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:44:01.310 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:44:01.310 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:44:01.310 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:44:01.310 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:44:01.310 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:44:01.311 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:44:01.311 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:44:01.311 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:44:01.311 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:44:01.311 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:44:01.311 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary4 2024-09-12T13:44:01.321 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary7 up+unknown 2024-09-12T13:44:01.323 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:44:01.323 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:44:01.324 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:44:01.324 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:44:01.324 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:44:01.324 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:44:01.324 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:44:01.324 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:44:01.408 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary6: 2024-09-12T13:44:01.409 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 85abb8e7-1989-4d4a-a874-9bc560cf4e0d 2024-09-12T13:44:01.409 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:44:01.409 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:44:01.409 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:44:01.409 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:43:47' 2024-09-12T13:44:01.409 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:44:01.411 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:44:01.411 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:44:01.411 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 2 2024-09-12T13:44:01.413 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary4: 2024-09-12T13:44:01.413 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 747f0ef0-de52-4d30-a780-d09d65c61a3b 2024-09-12T13:44:01.413 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:44:01.413 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:44:01.413 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:44:01.413 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:43:47' 2024-09-12T13:44:01.413 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:44:01.416 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:44:01.416 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:44:01.416 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 2 2024-09-12T13:44:01.581 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary3 up+unknown '' '' 2024-09-12T13:44:01.581 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:44:01.581 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:44:01.581 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:44:01.582 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:44:01.582 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:44:01.582 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:44:01.582 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:44:01.582 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:44:01.582 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary3 2024-09-12T13:44:01.604 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary5 up+unknown '' '' 2024-09-12T13:44:01.604 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:44:01.604 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:44:01.604 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:44:01.604 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:44:01.604 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:44:01.604 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:44:01.604 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:44:01.605 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:44:01.605 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary5 2024-09-12T13:44:01.653 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary3: 2024-09-12T13:44:01.654 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 6eed8ffb-1fae-4e8b-9794-cf7af72eafbc 2024-09-12T13:44:01.654 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:44:01.654 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:44:01.654 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:44:01.654 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:43:47' 2024-09-12T13:44:01.654 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:44:01.656 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:44:01.656 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:44:01.656 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 4 2024-09-12T13:44:01.676 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary5: 2024-09-12T13:44:01.676 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 19fef5ab-79b5-4712-8250-75fffa450f0a 2024-09-12T13:44:01.676 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:44:01.676 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:44:01.676 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:44:01.676 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:43:47' 2024-09-12T13:44:01.676 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:44:01.678 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:44:01.679 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:44:01.679 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 4 2024-09-12T13:44:02.313 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary1 up+unknown '' '' 2024-09-12T13:44:02.313 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:44:02.313 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:44:02.313 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:44:02.313 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:44:02.313 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:44:02.314 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:44:02.314 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:44:02.314 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:44:02.314 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary1 2024-09-12T13:44:02.324 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary7 up+unknown '' '' 2024-09-12T13:44:02.324 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:44:02.324 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:44:02.324 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:44:02.324 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:44:02.324 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:44:02.324 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:44:02.325 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:44:02.325 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:44:02.325 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary7 2024-09-12T13:44:02.390 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary1: 2024-09-12T13:44:02.390 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 99b2aeba-005c-4935-8fc2-c4175a9a85c1 2024-09-12T13:44:02.390 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:44:02.390 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:44:02.391 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:44:02.391 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:43:47' 2024-09-12T13:44:02.391 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:44:02.393 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:44:02.393 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:44:02.393 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 2 2024-09-12T13:44:02.398 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary7: 2024-09-12T13:44:02.399 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 6ca46845-7741-492c-99fb-642d3ab53fab 2024-09-12T13:44:02.399 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:44:02.399 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:44:02.399 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:44:02.399 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:43:47' 2024-09-12T13:44:02.399 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:44:02.401 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:44:02.401 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:44:02.401 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 2 2024-09-12T13:44:02.491 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary8 up+unknown '' '' 2024-09-12T13:44:02.491 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:44:02.491 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:44:02.491 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:44:02.491 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:44:02.491 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:44:02.491 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:44:02.491 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:44:02.492 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:44:02.492 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary8 2024-09-12T13:44:02.499 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary10 up+unknown '' '' 2024-09-12T13:44:02.500 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:44:02.500 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:44:02.500 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:44:02.500 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:44:02.500 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:44:02.500 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:44:02.500 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:44:02.500 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:44:02.501 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary10 2024-09-12T13:44:02.567 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary8: 2024-09-12T13:44:02.567 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 11646481-daec-461e-b7dc-ed4d9a9fd543 2024-09-12T13:44:02.567 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:44:02.567 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:44:02.567 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:44:02.567 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:43:47' 2024-09-12T13:44:02.567 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:44:02.570 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:44:02.570 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:44:02.570 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 4 2024-09-12T13:44:02.573 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary10: 2024-09-12T13:44:02.573 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 2e48042f-ba7f-45c0-9287-a6bae29dd722 2024-09-12T13:44:02.573 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:44:02.573 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:44:02.573 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:44:02.574 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:43:47' 2024-09-12T13:44:02.574 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:44:02.576 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:44:02.576 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:44:02.576 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 4 2024-09-12T13:44:02.727 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary9 up+unknown '' '' 2024-09-12T13:44:02.728 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:44:02.728 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:44:02.728 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:44:02.728 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:44:02.728 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:44:02.728 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:44:02.728 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:44:02.728 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:44:02.728 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary9 2024-09-12T13:44:02.805 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary9: 2024-09-12T13:44:02.805 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: b98c3fed-4807-46cf-b9ef-4cbefc529d10 2024-09-12T13:44:02.805 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:44:02.806 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:44:02.806 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:44:02.806 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:43:47' 2024-09-12T13:44:02.806 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:44:02.808 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:44:02.808 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:44:02.808 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 8 2024-09-12T13:44:03.414 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary6 up+unknown '' '' 2024-09-12T13:44:03.414 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:44:03.414 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:44:03.414 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:44:03.415 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:44:03.415 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:44:03.415 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:44:03.415 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:44:03.415 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:44:03.415 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary6 2024-09-12T13:44:03.418 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary4 up+unknown '' '' 2024-09-12T13:44:03.419 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:44:03.419 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:44:03.419 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:44:03.419 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:44:03.419 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:44:03.419 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:44:03.419 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:44:03.419 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:44:03.419 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary4 2024-09-12T13:44:03.510 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary6: 2024-09-12T13:44:03.510 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 85abb8e7-1989-4d4a-a874-9bc560cf4e0d 2024-09-12T13:44:03.510 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:44:03.511 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:44:03.511 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:44:03.511 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:43:47' 2024-09-12T13:44:03.511 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:44:03.511 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary4: 2024-09-12T13:44:03.511 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 747f0ef0-de52-4d30-a780-d09d65c61a3b 2024-09-12T13:44:03.511 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:44:03.511 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:44:03.511 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:44:03.511 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:43:47' 2024-09-12T13:44:03.511 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:44:03.513 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:44:03.513 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:44:03.513 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 4 2024-09-12T13:44:03.513 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:44:03.513 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:44:03.513 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 4 2024-09-12T13:44:03.783 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary2 up+unknown '' '' 2024-09-12T13:44:03.783 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:44:03.784 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:44:03.784 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:44:03.784 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:44:03.784 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:44:03.784 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:44:03.784 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:44:03.784 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:44:03.784 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary2 2024-09-12T13:44:03.863 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary2: 2024-09-12T13:44:03.863 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 603c11d9-89c3-4bd2-8c39-0d4ed9ed3093 2024-09-12T13:44:03.863 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:44:03.863 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:44:03.863 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:44:03.863 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:43:47' 2024-09-12T13:44:03.863 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:44:03.866 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:44:03.866 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:44:03.866 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 8 2024-09-12T13:44:04.395 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary1 up+unknown '' '' 2024-09-12T13:44:04.395 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:44:04.396 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:44:04.396 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:44:04.396 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:44:04.396 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:44:04.396 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:44:04.396 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:44:04.396 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:44:04.396 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary1 2024-09-12T13:44:04.404 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary7 up+unknown '' '' 2024-09-12T13:44:04.404 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:44:04.404 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:44:04.404 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:44:04.404 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:44:04.404 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:44:04.404 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:44:04.404 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:44:04.405 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:44:04.405 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary7 2024-09-12T13:44:04.472 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary1: 2024-09-12T13:44:04.472 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 99b2aeba-005c-4935-8fc2-c4175a9a85c1 2024-09-12T13:44:04.472 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:44:04.473 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:44:04.473 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:44:04.473 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:43:47' 2024-09-12T13:44:04.473 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:44:04.475 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:44:04.475 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:44:04.475 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 4 2024-09-12T13:44:04.478 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary7: 2024-09-12T13:44:04.478 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 6ca46845-7741-492c-99fb-642d3ab53fab 2024-09-12T13:44:04.478 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:44:04.478 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:44:04.478 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:44:04.478 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:43:47' 2024-09-12T13:44:04.478 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:44:04.480 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:44:04.481 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:44:04.481 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 4 2024-09-12T13:44:05.659 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary3 up+unknown '' '' 2024-09-12T13:44:05.659 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:44:05.659 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:44:05.659 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:44:05.659 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:44:05.660 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:44:05.660 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:44:05.660 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:44:05.660 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:44:05.660 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary3 2024-09-12T13:44:05.681 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary5 up+unknown '' '' 2024-09-12T13:44:05.681 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:44:05.681 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:44:05.681 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:44:05.681 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:44:05.682 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:44:05.682 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:44:05.682 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:44:05.682 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:44:05.682 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary5 2024-09-12T13:44:05.732 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary3: 2024-09-12T13:44:05.732 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 6eed8ffb-1fae-4e8b-9794-cf7af72eafbc 2024-09-12T13:44:05.732 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:44:05.733 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:44:05.733 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:44:05.733 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:43:47' 2024-09-12T13:44:05.733 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:44:05.735 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:44:05.735 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:44:05.735 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 8 2024-09-12T13:44:05.757 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary5: 2024-09-12T13:44:05.757 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 19fef5ab-79b5-4712-8250-75fffa450f0a 2024-09-12T13:44:05.757 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:44:05.757 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:44:05.757 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:44:05.757 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:43:47' 2024-09-12T13:44:05.757 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:44:05.760 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:44:05.760 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:44:05.760 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 8 2024-09-12T13:44:06.572 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary8 up+unknown '' '' 2024-09-12T13:44:06.572 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:44:06.572 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:44:06.572 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:44:06.573 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:44:06.573 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:44:06.573 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:44:06.573 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:44:06.573 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:44:06.573 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary8 2024-09-12T13:44:06.578 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary10 up+unknown '' '' 2024-09-12T13:44:06.578 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:44:06.578 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:44:06.578 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:44:06.578 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:44:06.579 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:44:06.579 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:44:06.579 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:44:06.579 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:44:06.579 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary10 2024-09-12T13:44:06.650 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary8: 2024-09-12T13:44:06.651 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 11646481-daec-461e-b7dc-ed4d9a9fd543 2024-09-12T13:44:06.651 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:44:06.651 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:44:06.651 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:44:06.651 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:43:47' 2024-09-12T13:44:06.651 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:44:06.653 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:44:06.653 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:44:06.654 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 8 2024-09-12T13:44:06.658 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary10: 2024-09-12T13:44:06.658 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 2e48042f-ba7f-45c0-9287-a6bae29dd722 2024-09-12T13:44:06.658 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:44:06.658 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:44:06.658 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:44:06.658 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:43:47' 2024-09-12T13:44:06.658 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:44:06.660 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:44:06.661 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:44:06.661 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 8 2024-09-12T13:44:07.516 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary6 up+unknown '' '' 2024-09-12T13:44:07.516 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:44:07.521 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:44:07.521 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:44:07.521 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:44:07.521 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:44:07.521 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:44:07.521 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:44:07.522 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary4 up+unknown '' '' 2024-09-12T13:44:07.522 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:44:07.522 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:44:07.522 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:44:07.522 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:44:07.522 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:44:07.522 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:44:07.523 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:44:07.523 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:44:07.523 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary6 2024-09-12T13:44:07.523 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:44:07.523 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary4 2024-09-12T13:44:07.606 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary6: 2024-09-12T13:44:07.606 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 85abb8e7-1989-4d4a-a874-9bc560cf4e0d 2024-09-12T13:44:07.606 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:44:07.606 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:44:07.606 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:44:07.606 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:43:47' 2024-09-12T13:44:07.606 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:44:07.606 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary4: 2024-09-12T13:44:07.606 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 747f0ef0-de52-4d30-a780-d09d65c61a3b 2024-09-12T13:44:07.607 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:44:07.607 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:44:07.607 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:44:07.607 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:43:47' 2024-09-12T13:44:07.607 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:44:07.607 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:44:07.607 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:44:07.607 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 8 2024-09-12T13:44:07.607 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:44:07.607 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:44:07.608 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 8 2024-09-12T13:44:08.479 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary1 up+unknown '' '' 2024-09-12T13:44:08.479 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:44:08.479 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:44:08.479 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:44:08.479 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:44:08.479 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:44:08.480 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:44:08.480 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:44:08.480 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:44:08.480 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary1 2024-09-12T13:44:08.483 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary7 up+unknown '' '' 2024-09-12T13:44:08.483 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:44:08.483 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:44:08.483 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:44:08.483 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:44:08.484 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:44:08.484 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:44:08.484 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:44:08.484 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:44:08.484 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary7 2024-09-12T13:44:08.573 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary1: 2024-09-12T13:44:08.573 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 99b2aeba-005c-4935-8fc2-c4175a9a85c1 2024-09-12T13:44:08.573 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:44:08.573 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:44:08.573 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:44:08.573 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:43:47' 2024-09-12T13:44:08.573 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:44:08.574 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary7: 2024-09-12T13:44:08.574 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 6ca46845-7741-492c-99fb-642d3ab53fab 2024-09-12T13:44:08.575 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:44:08.575 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:44:08.575 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:44:08.575 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:43:47' 2024-09-12T13:44:08.575 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:44:08.575 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:44:08.576 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:44:08.576 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 8 2024-09-12T13:44:08.579 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:44:08.579 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:44:08.580 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 8 2024-09-12T13:44:10.811 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary9 up+unknown '' '' 2024-09-12T13:44:10.811 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:44:10.811 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:44:10.811 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:44:10.811 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:44:10.811 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:44:10.811 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:44:10.811 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:44:10.812 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:44:10.812 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary9 2024-09-12T13:44:10.895 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary9: 2024-09-12T13:44:10.895 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: b98c3fed-4807-46cf-b9ef-4cbefc529d10 2024-09-12T13:44:10.896 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:44:10.896 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:44:10.896 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:44:10.896 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:43:47' 2024-09-12T13:44:10.896 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:44:10.900 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:44:10.900 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:44:10.900 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 8 2024-09-12T13:44:11.869 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary2 up+unknown '' '' 2024-09-12T13:44:11.869 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:44:11.869 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:44:11.869 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:44:11.869 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:44:11.869 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:44:11.869 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:44:11.869 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:44:11.870 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:44:11.870 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary2 2024-09-12T13:44:11.952 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary2: 2024-09-12T13:44:11.952 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 603c11d9-89c3-4bd2-8c39-0d4ed9ed3093 2024-09-12T13:44:11.952 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:44:11.952 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:44:11.952 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:44:11.952 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:43:47' 2024-09-12T13:44:11.952 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:44:11.955 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:44:11.955 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:44:11.955 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 8 2024-09-12T13:44:13.738 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary3 up+unknown '' '' 2024-09-12T13:44:13.738 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:44:13.738 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:44:13.738 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:44:13.738 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:44:13.738 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:44:13.739 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:44:13.739 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:44:13.739 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:44:13.739 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary3 2024-09-12T13:44:13.762 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary5 up+unknown '' '' 2024-09-12T13:44:13.762 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:44:13.762 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:44:13.762 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:44:13.762 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:44:13.763 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:44:13.763 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:44:13.763 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:44:13.763 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:44:13.763 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary5 2024-09-12T13:44:13.820 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary3: 2024-09-12T13:44:13.820 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 6eed8ffb-1fae-4e8b-9794-cf7af72eafbc 2024-09-12T13:44:13.820 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:44:13.820 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:44:13.820 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:44:13.820 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:43:47' 2024-09-12T13:44:13.820 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:44:13.823 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:44:13.823 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:44:13.823 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 8 2024-09-12T13:44:13.839 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary5: 2024-09-12T13:44:13.839 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 19fef5ab-79b5-4712-8250-75fffa450f0a 2024-09-12T13:44:13.839 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:44:13.839 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:44:13.840 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:44:13.840 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:43:47' 2024-09-12T13:44:13.840 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:44:13.842 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:44:13.842 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:44:13.842 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 8 2024-09-12T13:44:14.656 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary8 up+unknown '' '' 2024-09-12T13:44:14.657 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:44:14.657 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:44:14.657 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:44:14.657 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:44:14.657 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:44:14.657 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:44:14.657 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:44:14.657 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:44:14.657 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary8 2024-09-12T13:44:14.663 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary10 up+unknown '' '' 2024-09-12T13:44:14.663 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:44:14.663 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:44:14.663 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:44:14.663 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:44:14.663 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:44:14.663 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:44:14.664 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:44:14.664 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:44:14.664 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary10 2024-09-12T13:44:14.739 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary8: 2024-09-12T13:44:14.739 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 11646481-daec-461e-b7dc-ed4d9a9fd543 2024-09-12T13:44:14.739 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:44:14.739 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:44:14.739 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:44:14.739 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:43:47' 2024-09-12T13:44:14.739 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:44:14.742 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:44:14.742 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:44:14.742 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 8 2024-09-12T13:44:14.745 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary10: 2024-09-12T13:44:14.745 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 2e48042f-ba7f-45c0-9287-a6bae29dd722 2024-09-12T13:44:14.745 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:44:14.745 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:44:14.745 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:44:14.745 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:43:47' 2024-09-12T13:44:14.745 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:44:14.747 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:44:14.748 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:44:14.748 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 8 2024-09-12T13:44:15.602 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary6 up+unknown '' '' 2024-09-12T13:44:15.603 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:44:15.603 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:44:15.603 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:44:15.603 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:44:15.603 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:44:15.603 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:44:15.603 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:44:15.603 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary4 up+unknown '' '' 2024-09-12T13:44:15.603 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:44:15.603 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:44:15.603 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:44:15.603 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:44:15.603 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:44:15.604 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:44:15.604 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:44:15.604 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:44:15.604 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary6 2024-09-12T13:44:15.604 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:44:15.604 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary4 2024-09-12T13:44:15.685 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary4: 2024-09-12T13:44:15.685 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 747f0ef0-de52-4d30-a780-d09d65c61a3b 2024-09-12T13:44:15.685 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:44:15.685 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:44:15.685 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:44:15.685 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:43:47' 2024-09-12T13:44:15.685 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:44:15.686 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary6: 2024-09-12T13:44:15.686 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 85abb8e7-1989-4d4a-a874-9bc560cf4e0d 2024-09-12T13:44:15.686 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:44:15.686 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:44:15.686 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:44:15.686 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:43:47' 2024-09-12T13:44:15.686 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:44:15.688 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:44:15.688 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:44:15.688 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:44:15.689 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 8 2024-09-12T13:44:15.689 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:44:15.689 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 8 2024-09-12T13:44:16.578 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary1 up+unknown '' '' 2024-09-12T13:44:16.579 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:44:16.579 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:44:16.579 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:44:16.579 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:44:16.579 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:44:16.579 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:44:16.579 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:44:16.580 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:44:16.580 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary1 2024-09-12T13:44:16.580 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary7 up+unknown '' '' 2024-09-12T13:44:16.580 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:44:16.580 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:44:16.580 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:44:16.580 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:44:16.580 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:44:16.581 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:44:16.581 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:44:16.581 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:44:16.581 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary7 2024-09-12T13:44:16.662 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary7: 2024-09-12T13:44:16.662 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 6ca46845-7741-492c-99fb-642d3ab53fab 2024-09-12T13:44:16.662 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:44:16.662 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:44:16.663 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:44:16.663 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:43:47' 2024-09-12T13:44:16.663 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:44:16.663 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary1: 2024-09-12T13:44:16.663 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 99b2aeba-005c-4935-8fc2-c4175a9a85c1 2024-09-12T13:44:16.663 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:44:16.663 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:44:16.664 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:44:16.664 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:43:47' 2024-09-12T13:44:16.664 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:44:16.665 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:44:16.665 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:44:16.665 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 8 2024-09-12T13:44:16.666 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:44:16.666 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:44:16.666 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 8 2024-09-12T13:44:18.903 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary9 up+unknown '' '' 2024-09-12T13:44:18.903 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:44:18.903 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:44:18.903 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:44:18.903 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:44:18.904 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:44:18.904 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:44:18.904 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:44:18.904 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:44:18.904 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary9 2024-09-12T13:44:18.986 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary9: 2024-09-12T13:44:18.986 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: b98c3fed-4807-46cf-b9ef-4cbefc529d10 2024-09-12T13:44:18.987 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+unknown 2024-09-12T13:44:18.987 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: remote image is not primary 2024-09-12T13:44:18.987 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:44:18.987 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:44:17' 2024-09-12T13:44:18.987 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:44:18.989 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+unknown 2024-09-12T13:44:18.989 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:44:18.992 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: remote image is not primary 2024-09-12T13:44:18.992 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:44:18.993 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:44:18.993 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:44:18.995 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:44:18.995 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:44:18.998 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:44:18.998 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:44:18.999 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:44:19.003 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:44:17' 2024-09-12T13:44:19.003 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary9 up+unknown '2024-09-12 13:44:17' 2024-09-12T13:44:19.003 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:44:19.004 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:44:19.004 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:44:19.004 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:44:19.004 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:44:17' 2024-09-12T13:44:19.004 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:44:19.004 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:44:19.004 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-09-12T13:44:19.114 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041934232mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary199b2aeba-005c-4935-8fc2-c4175a9a85c1up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:44:17image-primary102e48042f-ba7f-45c0-9287-a6bae29dd722up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:44:17image-primary2603c11d9-89c3-4bd2-8c39-0d4ed9ed3093up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:44:17image-primary36eed8ffb-1fae-4e8b-9794-cf7af72eafbcup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:44:17image-primary4747f0ef0-de52-4d30-a780-d09d65c61a3bup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:44:17image-primary519fef5ab-79b5-4712-8250-75fffa450f0aup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:44:17image-primary685abb8e7-1989-4d4a-a874-9bc560cf4e0dup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:44:17image-primary76ca46845-7741-492c-99fb-642d3ab53fabup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:44:17image-primary811646481-daec-461e-b7dc-ed4d9a9fd543up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:44:17image-primary9b98c3fed-4807-46cf-b9ef-4cbefc529d10up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:44:17' 2024-09-12T13:44:19.114 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:44:19.115 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary9'\'']/last_update' 2024-09-12T13:44:19.138 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:44:17' 2024-09-12T13:44:19.139 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary9'\'']/state' 2024-09-12T13:44:19.142 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+unknown 2024-09-12T13:44:19.143 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:44:19.143 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+unknown 2024-09-12T13:44:19.145 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:44:19.146 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:44:19.146 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:44:19.146 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster2 mirror image-primary9 up+unknown 2024-09-12T13:44:19.146 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:44:19.147 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:44:19.147 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:44:19.147 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:44:19.147 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:44:19.147 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:44:19.147 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:44:19.147 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:44:19.957 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary2 up+unknown '' '' 2024-09-12T13:44:19.958 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:44:19.958 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:44:19.958 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:44:19.958 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:44:19.958 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:44:19.958 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:44:19.958 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:44:19.958 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:44:19.959 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary2 2024-09-12T13:44:20.044 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary2: 2024-09-12T13:44:20.044 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 603c11d9-89c3-4bd2-8c39-0d4ed9ed3093 2024-09-12T13:44:20.045 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+unknown 2024-09-12T13:44:20.045 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: remote image is not primary 2024-09-12T13:44:20.045 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:44:20.045 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:44:17' 2024-09-12T13:44:20.045 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:44:20.047 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+unknown 2024-09-12T13:44:20.047 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:44:20.050 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: remote image is not primary 2024-09-12T13:44:20.050 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:44:20.051 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:44:20.051 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:44:20.053 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:44:20.053 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:44:20.056 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:44:20.056 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:44:20.057 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:44:20.061 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:44:17' 2024-09-12T13:44:20.061 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary2 up+unknown '2024-09-12 13:44:17' 2024-09-12T13:44:20.062 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:44:20.062 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:44:20.062 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:44:20.062 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:44:20.062 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:44:17' 2024-09-12T13:44:20.062 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:44:20.062 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:44:20.062 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-09-12T13:44:20.149 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster2 mirror image-primary9 up+unknown '' '' 2024-09-12T13:44:20.149 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:44:20.149 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:44:20.149 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:44:20.149 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:44:20.149 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:44:20.149 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:44:20.149 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:44:20.150 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:44:20.150 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster2 mirror image status mirror/image-primary9 2024-09-12T13:44:20.170 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041934232mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary199b2aeba-005c-4935-8fc2-c4175a9a85c1up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:44:17image-primary102e48042f-ba7f-45c0-9287-a6bae29dd722up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:44:17image-primary2603c11d9-89c3-4bd2-8c39-0d4ed9ed3093up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:44:17image-primary36eed8ffb-1fae-4e8b-9794-cf7af72eafbcup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:44:17image-primary4747f0ef0-de52-4d30-a780-d09d65c61a3bup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:44:17image-primary519fef5ab-79b5-4712-8250-75fffa450f0aup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:44:17image-primary685abb8e7-1989-4d4a-a874-9bc560cf4e0dup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:44:17image-primary76ca46845-7741-492c-99fb-642d3ab53fabup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:44:17image-primary811646481-daec-461e-b7dc-ed4d9a9fd543up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:44:17image-primary9b98c3fed-4807-46cf-b9ef-4cbefc529d10up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:44:17' 2024-09-12T13:44:20.170 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:44:20.171 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary2'\'']/last_update' 2024-09-12T13:44:20.175 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:44:17' 2024-09-12T13:44:20.176 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary2'\'']/state' 2024-09-12T13:44:20.179 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+unknown 2024-09-12T13:44:20.180 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:44:20.180 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+unknown 2024-09-12T13:44:20.182 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:44:20.183 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:44:20.183 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:44:20.183 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster2 mirror image-primary2 up+unknown 2024-09-12T13:44:20.183 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:44:20.183 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:44:20.183 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:44:20.183 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:44:20.184 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:44:20.184 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:44:20.184 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:44:20.184 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:44:20.214 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary9: 2024-09-12T13:44:20.215 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: b98c3fed-4807-46cf-b9ef-4cbefc529d10 2024-09-12T13:44:20.215 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+unknown 2024-09-12T13:44:20.215 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: remote image is not primary 2024-09-12T13:44:20.215 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:44:20.215 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:44:17' 2024-09-12T13:44:20.215 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:44:20.217 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+unknown 2024-09-12T13:44:20.217 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:44:20.220 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: remote image is not primary 2024-09-12T13:44:20.220 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:44:20.221 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:44:20.221 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:44:20.223 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:44:20.223 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:44:20.226 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:44:20.226 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:44:20.227 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:44:20.230 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:44:17' 2024-09-12T13:44:20.231 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster2 mirror image-primary9 up+unknown '2024-09-12 13:44:17' 2024-09-12T13:44:20.231 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:44:20.231 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:44:20.231 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:44:20.231 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:44:20.231 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:44:17' 2024-09-12T13:44:20.231 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:44:20.231 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:44:20.231 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2024-09-12T13:44:20.330 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041924231mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary199b2aeba-005c-4935-8fc2-c4175a9a85c1up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:44:17image-primary102e48042f-ba7f-45c0-9287-a6bae29dd722up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:44:17image-primary2603c11d9-89c3-4bd2-8c39-0d4ed9ed3093up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:44:17image-primary36eed8ffb-1fae-4e8b-9794-cf7af72eafbcup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:44:17image-primary4747f0ef0-de52-4d30-a780-d09d65c61a3bup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:44:17image-primary519fef5ab-79b5-4712-8250-75fffa450f0aup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:44:17image-primary685abb8e7-1989-4d4a-a874-9bc560cf4e0dup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:44:17image-primary76ca46845-7741-492c-99fb-642d3ab53fabup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:44:17image-primary811646481-daec-461e-b7dc-ed4d9a9fd543up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:44:17image-primary9b98c3fed-4807-46cf-b9ef-4cbefc529d10up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:44:17' 2024-09-12T13:44:20.331 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:44:20.331 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary9'\'']/last_update' 2024-09-12T13:44:20.335 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:44:17' 2024-09-12T13:44:20.335 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary9'\'']/state' 2024-09-12T13:44:20.339 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+unknown 2024-09-12T13:44:20.340 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:44:20.340 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+unknown 2024-09-12T13:44:20.342 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:44:20.343 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:44:20.343 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:44:20.343 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ promote_image cluster2 mirror image-primary9 2024-09-12T13:44:20.344 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:44:20.344 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:44:20.344 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:44:20.344 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local force= 2024-09-12T13:44:20.344 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 mirror image promote mirror/image-primary9 2024-09-12T13:44:21.186 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster2 mirror image-primary2 up+unknown '' '' 2024-09-12T13:44:21.186 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:44:21.186 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:44:21.186 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:44:21.186 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:44:21.186 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:44:21.186 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:44:21.186 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:44:21.187 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:44:21.187 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster2 mirror image status mirror/image-primary2 2024-09-12T13:44:21.253 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary2: 2024-09-12T13:44:21.253 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 603c11d9-89c3-4bd2-8c39-0d4ed9ed3093 2024-09-12T13:44:21.253 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+unknown 2024-09-12T13:44:21.253 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: remote image is not primary 2024-09-12T13:44:21.254 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:44:21.254 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:44:17' 2024-09-12T13:44:21.254 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:44:21.255 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+unknown 2024-09-12T13:44:21.256 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:44:21.258 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: remote image is not primary 2024-09-12T13:44:21.258 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:44:21.259 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:44:21.259 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:44:21.261 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:44:21.262 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:44:21.264 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:44:21.264 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:44:21.265 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:44:21.269 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:44:17' 2024-09-12T13:44:21.269 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster2 mirror image-primary2 up+unknown '2024-09-12 13:44:17' 2024-09-12T13:44:21.269 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:44:21.269 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:44:21.269 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:44:21.269 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:44:21.269 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:44:17' 2024-09-12T13:44:21.269 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:44:21.270 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:44:21.270 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2024-09-12T13:44:21.368 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041924231mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary199b2aeba-005c-4935-8fc2-c4175a9a85c1up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:44:17image-primary102e48042f-ba7f-45c0-9287-a6bae29dd722up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:44:17image-primary2603c11d9-89c3-4bd2-8c39-0d4ed9ed3093up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:44:17image-primary36eed8ffb-1fae-4e8b-9794-cf7af72eafbcup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:44:17image-primary4747f0ef0-de52-4d30-a780-d09d65c61a3bup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:44:17image-primary519fef5ab-79b5-4712-8250-75fffa450f0aup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:44:17image-primary685abb8e7-1989-4d4a-a874-9bc560cf4e0dup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:44:17image-primary76ca46845-7741-492c-99fb-642d3ab53fabup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:44:17image-primary811646481-daec-461e-b7dc-ed4d9a9fd543up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:44:17image-primary9b98c3fed-4807-46cf-b9ef-4cbefc529d10up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:44:17' 2024-09-12T13:44:21.368 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:44:21.369 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary2'\'']/last_update' 2024-09-12T13:44:21.372 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:44:17' 2024-09-12T13:44:21.373 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary2'\'']/state' 2024-09-12T13:44:21.377 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+unknown 2024-09-12T13:44:21.377 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:44:21.378 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+unknown 2024-09-12T13:44:21.380 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:44:21.381 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:44:21.381 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:44:21.381 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ promote_image cluster2 mirror image-primary2 2024-09-12T13:44:21.381 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:44:21.381 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:44:21.381 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:44:21.381 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local force= 2024-09-12T13:44:21.381 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 mirror image promote mirror/image-primary2 2024-09-12T13:44:21.417 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Image promoted to primary 2024-09-12T13:44:21.429 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \n\b\d ]] 2024-09-12T13:44:21.429 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \k\r\b\d ]] 2024-09-12T13:44:21.430 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo rbd --cluster cluster2 device map -t krbd mirror/image-primary9 2024-09-12T13:44:21.559 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ dev=/dev/rbd0 2024-09-12T13:44:21.560 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo md5sum /dev/rbd0 2024-09-12T13:44:21.561 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ awk '{print $1}' 2024-09-12T13:44:21.826 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary3 up+unknown '' '' 2024-09-12T13:44:21.826 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:44:21.826 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:44:21.826 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:44:21.826 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:44:21.826 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:44:21.826 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:44:21.826 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:44:21.827 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:44:21.827 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary3 2024-09-12T13:44:21.845 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary5 up+unknown '' '' 2024-09-12T13:44:21.845 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:44:21.845 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:44:21.845 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:44:21.845 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:44:21.845 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:44:21.845 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:44:21.845 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:44:21.846 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:44:21.846 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary5 2024-09-12T13:44:21.906 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary3: 2024-09-12T13:44:21.906 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 6eed8ffb-1fae-4e8b-9794-cf7af72eafbc 2024-09-12T13:44:21.906 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+unknown 2024-09-12T13:44:21.906 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: remote image is not primary 2024-09-12T13:44:21.906 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:44:21.906 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:44:17' 2024-09-12T13:44:21.906 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:44:21.908 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+unknown 2024-09-12T13:44:21.909 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:44:21.911 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: remote image is not primary 2024-09-12T13:44:21.912 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:44:21.912 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:44:21.912 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:44:21.914 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:44:21.915 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:44:21.917 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:44:21.918 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:44:21.918 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:44:21.922 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:44:17' 2024-09-12T13:44:21.922 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary3 up+unknown '2024-09-12 13:44:17' 2024-09-12T13:44:21.923 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:44:21.923 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:44:21.923 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:44:21.923 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:44:21.923 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:44:17' 2024-09-12T13:44:21.923 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:44:21.923 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:44:21.923 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-09-12T13:44:21.924 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary5: 2024-09-12T13:44:21.924 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 19fef5ab-79b5-4712-8250-75fffa450f0a 2024-09-12T13:44:21.924 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+unknown 2024-09-12T13:44:21.924 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: remote image is not primary 2024-09-12T13:44:21.924 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:44:21.924 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:44:17' 2024-09-12T13:44:21.924 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:44:21.927 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+unknown 2024-09-12T13:44:21.927 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:44:21.930 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: remote image is not primary 2024-09-12T13:44:21.930 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:44:21.931 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:44:21.931 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:44:21.933 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:44:21.934 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:44:21.936 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:44:21.937 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:44:21.938 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:44:21.941 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:44:17' 2024-09-12T13:44:21.941 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary5 up+unknown '2024-09-12 13:44:17' 2024-09-12T13:44:21.942 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:44:21.942 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:44:21.942 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:44:21.942 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:44:21.942 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:44:17' 2024-09-12T13:44:21.942 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:44:21.942 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:44:21.942 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-09-12T13:44:22.045 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041934232mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary199b2aeba-005c-4935-8fc2-c4175a9a85c1up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:44:17image-primary102e48042f-ba7f-45c0-9287-a6bae29dd722up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:44:17image-primary2603c11d9-89c3-4bd2-8c39-0d4ed9ed3093up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:44:17image-primary36eed8ffb-1fae-4e8b-9794-cf7af72eafbcup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:44:17image-primary4747f0ef0-de52-4d30-a780-d09d65c61a3bup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:44:17image-primary519fef5ab-79b5-4712-8250-75fffa450f0aup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:44:17image-primary685abb8e7-1989-4d4a-a874-9bc560cf4e0dup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:44:17image-primary76ca46845-7741-492c-99fb-642d3ab53fabup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:44:17image-primary811646481-daec-461e-b7dc-ed4d9a9fd543up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:44:17image-primary9b98c3fed-4807-46cf-b9ef-4cbefc529d10up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:44:17' 2024-09-12T13:44:22.045 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:44:22.046 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary3'\'']/last_update' 2024-09-12T13:44:22.050 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:44:17' 2024-09-12T13:44:22.051 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary3'\'']/state' 2024-09-12T13:44:22.055 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+unknown 2024-09-12T13:44:22.056 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:44:22.056 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+unknown 2024-09-12T13:44:22.058 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:44:22.059 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:44:22.059 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:44:22.059 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster2 mirror image-primary3 up+unknown 2024-09-12T13:44:22.059 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:44:22.059 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:44:22.059 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:44:22.059 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:44:22.059 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:44:22.059 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:44:22.060 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:44:22.060 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:44:22.067 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041934232mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary199b2aeba-005c-4935-8fc2-c4175a9a85c1up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:44:17image-primary102e48042f-ba7f-45c0-9287-a6bae29dd722up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:44:17image-primary2603c11d9-89c3-4bd2-8c39-0d4ed9ed3093up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:44:17image-primary36eed8ffb-1fae-4e8b-9794-cf7af72eafbcup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:44:17image-primary4747f0ef0-de52-4d30-a780-d09d65c61a3bup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:44:17image-primary519fef5ab-79b5-4712-8250-75fffa450f0aup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:44:17image-primary685abb8e7-1989-4d4a-a874-9bc560cf4e0dup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:44:17image-primary76ca46845-7741-492c-99fb-642d3ab53fabup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:44:17image-primary811646481-daec-461e-b7dc-ed4d9a9fd543up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:44:17image-primary9b98c3fed-4807-46cf-b9ef-4cbefc529d10up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:44:17' 2024-09-12T13:44:22.067 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:44:22.068 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary5'\'']/last_update' 2024-09-12T13:44:22.072 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:44:17' 2024-09-12T13:44:22.073 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary5'\'']/state' 2024-09-12T13:44:22.076 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+unknown 2024-09-12T13:44:22.077 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:44:22.078 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+unknown 2024-09-12T13:44:22.080 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:44:22.081 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:44:22.081 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:44:22.081 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster2 mirror image-primary5 up+unknown 2024-09-12T13:44:22.081 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:44:22.081 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:44:22.081 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:44:22.081 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:44:22.081 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:44:22.081 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:44:22.081 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:44:22.081 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:44:22.423 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Image promoted to primary 2024-09-12T13:44:22.434 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \n\b\d ]] 2024-09-12T13:44:22.434 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \k\r\b\d ]] 2024-09-12T13:44:22.435 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo rbd --cluster cluster2 device map -t krbd mirror/image-primary2 2024-09-12T13:44:22.553 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ dev=/dev/rbd1 2024-09-12T13:44:22.554 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo md5sum /dev/rbd1 2024-09-12T13:44:22.554 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ awk '{print $1}' 2024-09-12T13:44:22.744 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary8 up+unknown '' '' 2024-09-12T13:44:22.744 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:44:22.745 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:44:22.745 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:44:22.745 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:44:22.745 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:44:22.745 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:44:22.745 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:44:22.745 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:44:22.745 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary8 2024-09-12T13:44:22.750 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary10 up+unknown '' '' 2024-09-12T13:44:22.750 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:44:22.750 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:44:22.750 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:44:22.750 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:44:22.751 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:44:22.751 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:44:22.751 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:44:22.751 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:44:22.751 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary10 2024-09-12T13:44:22.822 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary8: 2024-09-12T13:44:22.822 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 11646481-daec-461e-b7dc-ed4d9a9fd543 2024-09-12T13:44:22.822 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+unknown 2024-09-12T13:44:22.823 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: remote image is not primary 2024-09-12T13:44:22.823 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:44:22.823 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:44:17' 2024-09-12T13:44:22.823 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:44:22.825 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+unknown 2024-09-12T13:44:22.825 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:44:22.827 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: remote image is not primary 2024-09-12T13:44:22.828 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary10: 2024-09-12T13:44:22.828 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 2e48042f-ba7f-45c0-9287-a6bae29dd722 2024-09-12T13:44:22.828 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+unknown 2024-09-12T13:44:22.828 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: remote image is not primary 2024-09-12T13:44:22.828 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:44:22.828 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:44:17' 2024-09-12T13:44:22.828 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:44:22.828 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:44:22.829 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:44:22.829 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:44:22.831 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+unknown 2024-09-12T13:44:22.831 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:44:22.831 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:44:22.832 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:44:22.834 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: remote image is not primary 2024-09-12T13:44:22.834 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:44:22.835 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:44:22.835 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:44:22.835 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:44:22.835 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:44:22.836 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:44:22.838 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:44:22.838 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:44:22.839 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:44:17' 2024-09-12T13:44:22.839 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary8 up+unknown '2024-09-12 13:44:17' 2024-09-12T13:44:22.839 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:44:22.840 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:44:22.840 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:44:22.840 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:44:22.840 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:44:17' 2024-09-12T13:44:22.840 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:44:22.840 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:44:22.841 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-09-12T13:44:22.841 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:44:22.841 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:44:22.842 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:44:22.847 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:44:17' 2024-09-12T13:44:22.847 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary10 up+unknown '2024-09-12 13:44:17' 2024-09-12T13:44:22.847 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:44:22.847 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:44:22.847 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:44:22.847 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:44:22.847 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:44:17' 2024-09-12T13:44:22.847 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:44:22.848 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:44:22.848 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-09-12T13:44:22.983 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041934232mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary199b2aeba-005c-4935-8fc2-c4175a9a85c1up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:44:17image-primary102e48042f-ba7f-45c0-9287-a6bae29dd722up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:44:17image-primary2603c11d9-89c3-4bd2-8c39-0d4ed9ed3093up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:44:17image-primary36eed8ffb-1fae-4e8b-9794-cf7af72eafbcup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:44:17image-primary4747f0ef0-de52-4d30-a780-d09d65c61a3bup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:44:17image-primary519fef5ab-79b5-4712-8250-75fffa450f0aup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:44:17image-primary685abb8e7-1989-4d4a-a874-9bc560cf4e0dup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:44:17image-primary76ca46845-7741-492c-99fb-642d3ab53fabup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:44:17image-primary811646481-daec-461e-b7dc-ed4d9a9fd543up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:44:17image-primary9b98c3fed-4807-46cf-b9ef-4cbefc529d10up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:44:17' 2024-09-12T13:44:23.023 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:44:23.026 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary10'\'']/last_update' 2024-09-12T13:44:23.026 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041934232mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary199b2aeba-005c-4935-8fc2-c4175a9a85c1up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:44:17image-primary102e48042f-ba7f-45c0-9287-a6bae29dd722up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:44:17image-primary2603c11d9-89c3-4bd2-8c39-0d4ed9ed3093up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:44:17image-primary36eed8ffb-1fae-4e8b-9794-cf7af72eafbcup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:44:17image-primary4747f0ef0-de52-4d30-a780-d09d65c61a3bup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:44:17image-primary519fef5ab-79b5-4712-8250-75fffa450f0aup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:44:17image-primary685abb8e7-1989-4d4a-a874-9bc560cf4e0dup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:44:17image-primary76ca46845-7741-492c-99fb-642d3ab53fabup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:44:17image-primary811646481-daec-461e-b7dc-ed4d9a9fd543up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:44:17image-primary9b98c3fed-4807-46cf-b9ef-4cbefc529d10up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:44:17' 2024-09-12T13:44:23.026 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:44:23.026 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary8'\'']/last_update' 2024-09-12T13:44:23.026 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:44:17' 2024-09-12T13:44:23.026 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary10'\'']/state' 2024-09-12T13:44:23.026 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:44:17' 2024-09-12T13:44:23.026 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary8'\'']/state' 2024-09-12T13:44:23.026 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+unknown 2024-09-12T13:44:23.026 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:44:23.026 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+unknown 2024-09-12T13:44:23.027 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+unknown 2024-09-12T13:44:23.027 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:44:23.027 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:44:23.028 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:44:23.028 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:44:23.028 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+unknown 2024-09-12T13:44:23.028 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster2 mirror image-primary10 up+unknown 2024-09-12T13:44:23.028 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:44:23.028 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:44:23.028 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:44:23.028 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:44:23.028 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:44:23.028 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:44:23.028 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:44:23.028 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:44:23.029 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:44:23.030 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:44:23.030 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:44:23.030 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster2 mirror image-primary8 up+unknown 2024-09-12T13:44:23.030 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:44:23.030 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:44:23.030 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:44:23.030 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:44:23.030 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:44:23.030 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:44:23.030 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:44:23.030 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:44:23.061 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster2 mirror image-primary3 up+unknown '' '' 2024-09-12T13:44:23.062 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:44:23.062 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:44:23.062 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:44:23.062 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:44:23.062 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:44:23.062 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:44:23.062 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:44:23.062 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:44:23.062 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster2 mirror image status mirror/image-primary3 2024-09-12T13:44:23.083 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster2 mirror image-primary5 up+unknown '' '' 2024-09-12T13:44:23.083 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:44:23.083 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:44:23.083 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:44:23.084 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:44:23.084 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:44:23.084 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:44:23.084 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:44:23.084 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:44:23.084 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster2 mirror image status mirror/image-primary5 2024-09-12T13:44:23.136 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary3: 2024-09-12T13:44:23.137 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 6eed8ffb-1fae-4e8b-9794-cf7af72eafbc 2024-09-12T13:44:23.137 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+unknown 2024-09-12T13:44:23.137 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: remote image is not primary 2024-09-12T13:44:23.137 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:44:23.137 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:44:17' 2024-09-12T13:44:23.137 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:44:23.139 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+unknown 2024-09-12T13:44:23.140 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:44:23.142 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: remote image is not primary 2024-09-12T13:44:23.143 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:44:23.143 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:44:23.144 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:44:23.146 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:44:23.147 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:44:23.149 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:44:23.150 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:44:23.151 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:44:23.156 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:44:17' 2024-09-12T13:44:23.156 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster2 mirror image-primary3 up+unknown '2024-09-12 13:44:17' 2024-09-12T13:44:23.156 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:44:23.156 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:44:23.157 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:44:23.157 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:44:23.157 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:44:17' 2024-09-12T13:44:23.157 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:44:23.157 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:44:23.157 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2024-09-12T13:44:23.159 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary5: 2024-09-12T13:44:23.159 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 19fef5ab-79b5-4712-8250-75fffa450f0a 2024-09-12T13:44:23.159 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+unknown 2024-09-12T13:44:23.159 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: remote image is not primary 2024-09-12T13:44:23.159 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:44:23.159 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:44:17' 2024-09-12T13:44:23.159 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:44:23.161 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+unknown 2024-09-12T13:44:23.162 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:44:23.164 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: remote image is not primary 2024-09-12T13:44:23.165 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:44:23.166 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:44:23.166 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:44:23.168 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:44:23.169 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:44:23.172 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:44:23.172 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:44:23.173 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:44:23.178 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:44:17' 2024-09-12T13:44:23.178 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster2 mirror image-primary5 up+unknown '2024-09-12 13:44:17' 2024-09-12T13:44:23.178 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:44:23.178 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:44:23.178 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:44:23.178 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:44:23.178 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:44:17' 2024-09-12T13:44:23.178 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:44:23.179 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:44:23.179 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2024-09-12T13:44:23.291 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041924231mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary199b2aeba-005c-4935-8fc2-c4175a9a85c1up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:44:17image-primary102e48042f-ba7f-45c0-9287-a6bae29dd722up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:44:17image-primary2603c11d9-89c3-4bd2-8c39-0d4ed9ed3093up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:44:17image-primary36eed8ffb-1fae-4e8b-9794-cf7af72eafbcup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:44:17image-primary4747f0ef0-de52-4d30-a780-d09d65c61a3bup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:44:17image-primary519fef5ab-79b5-4712-8250-75fffa450f0aup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:44:17image-primary685abb8e7-1989-4d4a-a874-9bc560cf4e0dup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:44:17image-primary76ca46845-7741-492c-99fb-642d3ab53fabup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:44:17image-primary811646481-daec-461e-b7dc-ed4d9a9fd543up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:44:17image-primary9b98c3fed-4807-46cf-b9ef-4cbefc529d10up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:44:17' 2024-09-12T13:44:23.291 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:44:23.293 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary3'\'']/last_update' 2024-09-12T13:44:23.300 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:44:17' 2024-09-12T13:44:23.301 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary3'\'']/state' 2024-09-12T13:44:23.312 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+unknown 2024-09-12T13:44:23.313 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:44:23.317 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+unknown 2024-09-12T13:44:23.319 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:44:23.321 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:44:23.321 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:44:23.321 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ promote_image cluster2 mirror image-primary3 2024-09-12T13:44:23.322 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:44:23.322 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:44:23.322 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:44:23.322 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local force= 2024-09-12T13:44:23.322 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 mirror image promote mirror/image-primary3 2024-09-12T13:44:23.339 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041924231mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary199b2aeba-005c-4935-8fc2-c4175a9a85c1up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:44:17image-primary102e48042f-ba7f-45c0-9287-a6bae29dd722up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:44:17image-primary2603c11d9-89c3-4bd2-8c39-0d4ed9ed3093up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:44:17image-primary36eed8ffb-1fae-4e8b-9794-cf7af72eafbcup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:44:17image-primary4747f0ef0-de52-4d30-a780-d09d65c61a3bup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:44:17image-primary519fef5ab-79b5-4712-8250-75fffa450f0aup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:44:17image-primary685abb8e7-1989-4d4a-a874-9bc560cf4e0dup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:44:17image-primary76ca46845-7741-492c-99fb-642d3ab53fabup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:44:17image-primary811646481-daec-461e-b7dc-ed4d9a9fd543up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:44:17image-primary9b98c3fed-4807-46cf-b9ef-4cbefc529d10up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:44:17' 2024-09-12T13:44:23.340 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:44:23.340 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary5'\'']/last_update' 2024-09-12T13:44:23.344 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:44:17' 2024-09-12T13:44:23.345 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary5'\'']/state' 2024-09-12T13:44:23.349 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+unknown 2024-09-12T13:44:23.350 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:44:23.351 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+unknown 2024-09-12T13:44:23.353 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:44:23.354 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:44:23.354 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:44:23.354 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ promote_image cluster2 mirror image-primary5 2024-09-12T13:44:23.354 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:44:23.354 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:44:23.354 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:44:23.354 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local force= 2024-09-12T13:44:23.354 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 mirror image promote mirror/image-primary5 2024-09-12T13:44:23.691 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary4 up+unknown '' '' 2024-09-12T13:44:23.691 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:44:23.691 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:44:23.691 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:44:23.691 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:44:23.691 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:44:23.691 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:44:23.691 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:44:23.692 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary6 up+unknown '' '' 2024-09-12T13:44:23.692 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:44:23.692 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:44:23.692 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:44:23.692 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:44:23.692 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:44:23.692 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:44:23.692 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:44:23.692 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:44:23.693 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary4 2024-09-12T13:44:23.693 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:44:23.693 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary6 2024-09-12T13:44:23.764 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary6: 2024-09-12T13:44:23.765 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 85abb8e7-1989-4d4a-a874-9bc560cf4e0d 2024-09-12T13:44:23.765 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+unknown 2024-09-12T13:44:23.765 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: remote image is not primary 2024-09-12T13:44:23.765 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:44:23.765 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:44:17' 2024-09-12T13:44:23.765 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:44:23.767 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+unknown 2024-09-12T13:44:23.767 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:44:23.770 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: remote image is not primary 2024-09-12T13:44:23.770 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:44:23.770 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:44:23.771 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:44:23.773 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:44:23.773 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:44:23.776 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:44:23.776 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:44:23.777 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:44:23.781 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:44:17' 2024-09-12T13:44:23.781 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary6 up+unknown '2024-09-12 13:44:17' 2024-09-12T13:44:23.781 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:44:23.781 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:44:23.781 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:44:23.781 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:44:23.781 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:44:17' 2024-09-12T13:44:23.781 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:44:23.782 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:44:23.782 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-09-12T13:44:23.788 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary4: 2024-09-12T13:44:23.788 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 747f0ef0-de52-4d30-a780-d09d65c61a3b 2024-09-12T13:44:23.788 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+unknown 2024-09-12T13:44:23.788 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: remote image is not primary 2024-09-12T13:44:23.788 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:44:23.788 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:44:17' 2024-09-12T13:44:23.788 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:44:23.791 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+unknown 2024-09-12T13:44:23.791 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:44:23.793 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: remote image is not primary 2024-09-12T13:44:23.794 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:44:23.795 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:44:23.795 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:44:23.797 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:44:23.798 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:44:23.800 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:44:23.801 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:44:23.802 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:44:23.806 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:44:17' 2024-09-12T13:44:23.806 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary4 up+unknown '2024-09-12 13:44:17' 2024-09-12T13:44:23.806 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:44:23.807 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:44:23.807 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:44:23.807 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:44:23.807 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:44:17' 2024-09-12T13:44:23.807 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:44:23.807 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:44:23.807 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-09-12T13:44:23.898 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041934232mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary199b2aeba-005c-4935-8fc2-c4175a9a85c1up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:44:17image-primary102e48042f-ba7f-45c0-9287-a6bae29dd722up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:44:17image-primary2603c11d9-89c3-4bd2-8c39-0d4ed9ed3093up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:44:17image-primary36eed8ffb-1fae-4e8b-9794-cf7af72eafbcup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:44:17image-primary4747f0ef0-de52-4d30-a780-d09d65c61a3bup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:44:17image-primary519fef5ab-79b5-4712-8250-75fffa450f0aup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:44:17image-primary685abb8e7-1989-4d4a-a874-9bc560cf4e0dup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:44:17image-primary76ca46845-7741-492c-99fb-642d3ab53fabup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:44:17image-primary811646481-daec-461e-b7dc-ed4d9a9fd543up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:44:17image-primary9b98c3fed-4807-46cf-b9ef-4cbefc529d10up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:44:17' 2024-09-12T13:44:23.898 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:44:23.898 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary6'\'']/last_update' 2024-09-12T13:44:23.903 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:44:17' 2024-09-12T13:44:23.904 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary6'\'']/state' 2024-09-12T13:44:23.909 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+unknown 2024-09-12T13:44:23.909 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:44:23.910 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+unknown 2024-09-12T13:44:23.913 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:44:23.915 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:44:23.915 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:44:23.915 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster2 mirror image-primary6 up+unknown 2024-09-12T13:44:23.915 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:44:23.915 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:44:23.915 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:44:23.915 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:44:23.915 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:44:23.915 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:44:23.915 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:44:23.916 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:44:23.937 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041934232mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary199b2aeba-005c-4935-8fc2-c4175a9a85c1up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:44:17image-primary102e48042f-ba7f-45c0-9287-a6bae29dd722up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:44:17image-primary2603c11d9-89c3-4bd2-8c39-0d4ed9ed3093up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:44:17image-primary36eed8ffb-1fae-4e8b-9794-cf7af72eafbcup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:44:17image-primary4747f0ef0-de52-4d30-a780-d09d65c61a3bup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:44:17image-primary519fef5ab-79b5-4712-8250-75fffa450f0aup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:44:17image-primary685abb8e7-1989-4d4a-a874-9bc560cf4e0dup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:44:17image-primary76ca46845-7741-492c-99fb-642d3ab53fabup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:44:17image-primary811646481-daec-461e-b7dc-ed4d9a9fd543up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:44:17image-primary9b98c3fed-4807-46cf-b9ef-4cbefc529d10up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:44:17' 2024-09-12T13:44:23.937 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:44:23.938 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary4'\'']/last_update' 2024-09-12T13:44:23.941 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:44:17' 2024-09-12T13:44:23.942 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary4'\'']/state' 2024-09-12T13:44:23.946 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+unknown 2024-09-12T13:44:23.946 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:44:23.947 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+unknown 2024-09-12T13:44:23.949 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:44:23.950 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:44:23.950 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:44:23.950 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster2 mirror image-primary4 up+unknown 2024-09-12T13:44:23.950 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:44:23.950 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:44:23.950 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:44:23.950 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:44:23.950 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:44:23.950 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:44:23.950 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:44:23.950 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:44:24.000 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster2 mirror image-primary10 up+unknown '' '' 2024-09-12T13:44:24.000 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:44:24.000 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:44:24.000 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:44:24.000 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:44:24.000 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:44:24.000 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:44:24.000 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:44:24.001 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:44:24.001 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster2 mirror image status mirror/image-primary10 2024-09-12T13:44:24.001 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster2 mirror image-primary8 up+unknown '' '' 2024-09-12T13:44:24.001 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:44:24.001 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:44:24.001 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:44:24.001 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:44:24.001 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:44:24.002 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:44:24.002 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:44:24.002 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:44:24.002 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster2 mirror image status mirror/image-primary8 2024-09-12T13:44:24.079 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary10: 2024-09-12T13:44:24.079 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 2e48042f-ba7f-45c0-9287-a6bae29dd722 2024-09-12T13:44:24.079 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+unknown 2024-09-12T13:44:24.079 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: remote image is not primary 2024-09-12T13:44:24.079 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:44:24.079 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary8: 2024-09-12T13:44:24.079 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 11646481-daec-461e-b7dc-ed4d9a9fd543 2024-09-12T13:44:24.080 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+unknown 2024-09-12T13:44:24.080 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: remote image is not primary 2024-09-12T13:44:24.080 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:44:24.080 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:44:17' 2024-09-12T13:44:24.080 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:44:17' 2024-09-12T13:44:24.080 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:44:24.080 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:44:24.083 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+unknown 2024-09-12T13:44:24.083 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+unknown 2024-09-12T13:44:24.083 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:44:24.083 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:44:24.088 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: remote image is not primary 2024-09-12T13:44:24.088 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: remote image is not primary 2024-09-12T13:44:24.088 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:44:24.088 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:44:24.089 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:44:24.089 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:44:24.090 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:44:24.090 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:44:24.091 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:44:24.091 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:44:24.092 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:44:24.092 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:44:24.094 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:44:24.094 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:44:24.095 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:44:24.095 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:44:24.095 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:44:24.096 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:44:24.100 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:44:17' 2024-09-12T13:44:24.100 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster2 mirror image-primary10 up+unknown '2024-09-12 13:44:17' 2024-09-12T13:44:24.100 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:44:24.100 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:44:24.101 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:44:24.101 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:44:24.101 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:44:17' 2024-09-12T13:44:24.101 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:44:24.101 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:44:17' 2024-09-12T13:44:24.101 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster2 mirror image-primary8 up+unknown '2024-09-12 13:44:17' 2024-09-12T13:44:24.101 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:44:24.101 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2024-09-12T13:44:24.101 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:44:24.101 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:44:24.101 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:44:24.101 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:44:24.101 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:44:17' 2024-09-12T13:44:24.102 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:44:24.102 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:44:24.102 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2024-09-12T13:44:24.250 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041924231mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary199b2aeba-005c-4935-8fc2-c4175a9a85c1up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:44:17image-primary102e48042f-ba7f-45c0-9287-a6bae29dd722up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:44:17image-primary2603c11d9-89c3-4bd2-8c39-0d4ed9ed3093up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:44:17image-primary36eed8ffb-1fae-4e8b-9794-cf7af72eafbcup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:44:17image-primary4747f0ef0-de52-4d30-a780-d09d65c61a3bup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:44:17image-primary519fef5ab-79b5-4712-8250-75fffa450f0aup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:44:17image-primary685abb8e7-1989-4d4a-a874-9bc560cf4e0dup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:44:17image-primary76ca46845-7741-492c-99fb-642d3ab53fabup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:44:17image-primary811646481-daec-461e-b7dc-ed4d9a9fd543up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:44:17image-primary9b98c3fed-4807-46cf-b9ef-4cbefc529d10up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:44:17' 2024-09-12T13:44:24.250 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:44:24.250 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary10'\'']/last_update' 2024-09-12T13:44:24.255 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:44:17' 2024-09-12T13:44:24.256 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary10'\'']/state' 2024-09-12T13:44:24.259 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041924231mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary199b2aeba-005c-4935-8fc2-c4175a9a85c1up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:44:17image-primary102e48042f-ba7f-45c0-9287-a6bae29dd722up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:44:17image-primary2603c11d9-89c3-4bd2-8c39-0d4ed9ed3093up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:44:17image-primary36eed8ffb-1fae-4e8b-9794-cf7af72eafbcup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:44:17image-primary4747f0ef0-de52-4d30-a780-d09d65c61a3bup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:44:17image-primary519fef5ab-79b5-4712-8250-75fffa450f0aup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:44:17image-primary685abb8e7-1989-4d4a-a874-9bc560cf4e0dup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:44:17image-primary76ca46845-7741-492c-99fb-642d3ab53fabup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:44:17image-primary811646481-daec-461e-b7dc-ed4d9a9fd543up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:44:17image-primary9b98c3fed-4807-46cf-b9ef-4cbefc529d10up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:44:17' 2024-09-12T13:44:24.259 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:44:24.260 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary8'\'']/last_update' 2024-09-12T13:44:24.260 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+unknown 2024-09-12T13:44:24.261 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:44:24.261 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+unknown 2024-09-12T13:44:24.264 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:44:24.264 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:44:17' 2024-09-12T13:44:24.265 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:44:24.265 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:44:24.265 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ promote_image cluster2 mirror image-primary10 2024-09-12T13:44:24.265 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:44:24.265 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:44:24.265 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:44:24.265 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local force= 2024-09-12T13:44:24.265 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 mirror image promote mirror/image-primary10 2024-09-12T13:44:24.265 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary8'\'']/state' 2024-09-12T13:44:24.269 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+unknown 2024-09-12T13:44:24.269 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:44:24.270 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+unknown 2024-09-12T13:44:24.272 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:44:24.273 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:44:24.273 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:44:24.273 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ promote_image cluster2 mirror image-primary8 2024-09-12T13:44:24.273 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:44:24.274 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:44:24.274 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:44:24.274 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local force= 2024-09-12T13:44:24.274 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 mirror image promote mirror/image-primary8 2024-09-12T13:44:24.450 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Image promoted to primary 2024-09-12T13:44:24.452 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Image promoted to primary 2024-09-12T13:44:24.452 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Image promoted to primary 2024-09-12T13:44:24.452 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Image promoted to primary 2024-09-12T13:44:24.472 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \n\b\d ]] 2024-09-12T13:44:24.472 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \k\r\b\d ]] 2024-09-12T13:44:24.472 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \n\b\d ]] 2024-09-12T13:44:24.472 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \k\r\b\d ]] 2024-09-12T13:44:24.472 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \n\b\d ]] 2024-09-12T13:44:24.473 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \k\r\b\d ]] 2024-09-12T13:44:24.473 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo rbd --cluster cluster2 device map -t krbd mirror/image-primary3 2024-09-12T13:44:24.473 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo rbd --cluster cluster2 device map -t krbd mirror/image-primary8 2024-09-12T13:44:24.474 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \n\b\d ]] 2024-09-12T13:44:24.474 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \k\r\b\d ]] 2024-09-12T13:44:24.474 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo rbd --cluster cluster2 device map -t krbd mirror/image-primary10 2024-09-12T13:44:24.474 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo rbd --cluster cluster2 device map -t krbd mirror/image-primary5 2024-09-12T13:44:24.660 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ dev=/dev/rbd2 2024-09-12T13:44:24.663 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ awk '{print $1}' 2024-09-12T13:44:24.667 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo md5sum /dev/rbd2 2024-09-12T13:44:24.669 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary1 up+unknown '' '' 2024-09-12T13:44:24.670 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:44:24.670 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:44:24.670 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:44:24.670 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:44:24.670 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:44:24.670 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:44:24.670 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:44:24.671 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:44:24.671 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary1 2024-09-12T13:44:24.675 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary7 up+unknown '' '' 2024-09-12T13:44:24.675 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:44:24.675 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:44:24.675 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:44:24.675 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:44:24.675 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:44:24.675 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:44:24.675 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:44:24.679 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:44:24.680 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary7 2024-09-12T13:44:24.699 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ dev=/dev/rbd3 2024-09-12T13:44:24.701 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo md5sum /dev/rbd3 2024-09-12T13:44:24.702 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ awk '{print $1}' 2024-09-12T13:44:24.702 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ dev=/dev/rbd4 2024-09-12T13:44:24.705 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo md5sum /dev/rbd4 2024-09-12T13:44:24.706 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ awk '{print $1}' 2024-09-12T13:44:24.715 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ dev=/dev/rbd5 2024-09-12T13:44:24.718 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo md5sum /dev/rbd5 2024-09-12T13:44:24.718 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ awk '{print $1}' 2024-09-12T13:44:24.808 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary1: 2024-09-12T13:44:24.808 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 99b2aeba-005c-4935-8fc2-c4175a9a85c1 2024-09-12T13:44:24.808 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+unknown 2024-09-12T13:44:24.808 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: remote image is not primary 2024-09-12T13:44:24.809 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:44:24.809 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:44:17' 2024-09-12T13:44:24.809 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:44:24.813 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+unknown 2024-09-12T13:44:24.814 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:44:24.819 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: remote image is not primary 2024-09-12T13:44:24.820 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:44:24.822 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:44:24.822 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:44:24.826 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:44:24.827 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:44:24.828 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary7: 2024-09-12T13:44:24.828 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 6ca46845-7741-492c-99fb-642d3ab53fab 2024-09-12T13:44:24.828 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+unknown 2024-09-12T13:44:24.828 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: remote image is not primary 2024-09-12T13:44:24.828 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:44:24.828 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:44:17' 2024-09-12T13:44:24.828 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:44:24.833 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:44:24.833 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:44:24.834 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+unknown 2024-09-12T13:44:24.835 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:44:24.835 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:44:24.839 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: remote image is not primary 2024-09-12T13:44:24.840 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:44:24.842 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:44:17' 2024-09-12T13:44:24.842 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary1 up+unknown '2024-09-12 13:44:17' 2024-09-12T13:44:24.842 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:44:24.842 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:44:24.842 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:44:24.842 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:44:24.843 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:44:17' 2024-09-12T13:44:24.843 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:44:24.843 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:44:24.843 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:44:24.844 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:44:24.844 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-09-12T13:44:24.847 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:44:24.848 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:44:24.853 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:44:24.854 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:44:24.855 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:44:24.862 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:44:17' 2024-09-12T13:44:24.862 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary7 up+unknown '2024-09-12 13:44:17' 2024-09-12T13:44:24.863 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:44:24.863 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:44:24.863 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:44:24.863 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:44:24.863 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:44:17' 2024-09-12T13:44:24.864 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:44:24.865 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:44:24.865 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-09-12T13:44:24.920 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster2 mirror image-primary6 up+unknown '' '' 2024-09-12T13:44:24.920 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:44:24.920 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:44:24.920 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:44:24.921 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:44:24.921 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:44:24.921 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:44:24.921 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:44:24.924 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:44:24.924 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster2 mirror image status mirror/image-primary6 2024-09-12T13:44:24.954 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster2 mirror image-primary4 up+unknown '' '' 2024-09-12T13:44:24.954 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:44:24.954 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:44:24.954 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:44:24.954 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:44:24.954 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:44:24.954 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:44:24.954 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:44:24.955 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:44:24.955 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster2 mirror image status mirror/image-primary4 2024-09-12T13:44:25.078 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041934232mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary199b2aeba-005c-4935-8fc2-c4175a9a85c1up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:44:17image-primary102e48042f-ba7f-45c0-9287-a6bae29dd722up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:44:17image-primary2603c11d9-89c3-4bd2-8c39-0d4ed9ed3093up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:44:17image-primary36eed8ffb-1fae-4e8b-9794-cf7af72eafbcup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:44:17image-primary4747f0ef0-de52-4d30-a780-d09d65c61a3bup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:44:17image-primary519fef5ab-79b5-4712-8250-75fffa450f0aup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:44:17image-primary685abb8e7-1989-4d4a-a874-9bc560cf4e0dup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:44:17image-primary76ca46845-7741-492c-99fb-642d3ab53fabup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:44:17image-primary811646481-daec-461e-b7dc-ed4d9a9fd543up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:44:17image-primary9b98c3fed-4807-46cf-b9ef-4cbefc529d10up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:44:17' 2024-09-12T13:44:25.078 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:44:25.079 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary1'\'']/last_update' 2024-09-12T13:44:25.085 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:44:17' 2024-09-12T13:44:25.087 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary1'\'']/state' 2024-09-12T13:44:25.094 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+unknown 2024-09-12T13:44:25.097 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:44:25.098 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+unknown 2024-09-12T13:44:25.099 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041934232mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary199b2aeba-005c-4935-8fc2-c4175a9a85c1up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:44:17image-primary102e48042f-ba7f-45c0-9287-a6bae29dd722up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:44:17image-primary2603c11d9-89c3-4bd2-8c39-0d4ed9ed3093up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:44:17image-primary36eed8ffb-1fae-4e8b-9794-cf7af72eafbcup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:44:17image-primary4747f0ef0-de52-4d30-a780-d09d65c61a3bup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:44:17image-primary519fef5ab-79b5-4712-8250-75fffa450f0aup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:44:17image-primary685abb8e7-1989-4d4a-a874-9bc560cf4e0dup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:44:17image-primary76ca46845-7741-492c-99fb-642d3ab53fabup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:44:17image-primary811646481-daec-461e-b7dc-ed4d9a9fd543up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:44:17image-primary9b98c3fed-4807-46cf-b9ef-4cbefc529d10up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:44:17' 2024-09-12T13:44:25.099 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:44:25.101 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary7'\'']/last_update' 2024-09-12T13:44:25.105 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:44:25.106 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:44:25.106 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:44:25.106 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster2 mirror image-primary1 up+unknown 2024-09-12T13:44:25.107 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:44:25.108 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:44:25.108 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:44:25.108 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:44:25.108 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:44:25.108 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:44:25.108 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:44:25.108 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:44:25.109 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:44:17' 2024-09-12T13:44:25.110 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary7'\'']/state' 2024-09-12T13:44:25.118 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+unknown 2024-09-12T13:44:25.120 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:44:25.120 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+unknown 2024-09-12T13:44:25.121 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary6: 2024-09-12T13:44:25.121 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 85abb8e7-1989-4d4a-a874-9bc560cf4e0d 2024-09-12T13:44:25.121 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+unknown 2024-09-12T13:44:25.121 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: remote image is not primary 2024-09-12T13:44:25.121 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:44:25.121 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:44:17' 2024-09-12T13:44:25.121 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:44:25.125 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:44:25.125 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+unknown 2024-09-12T13:44:25.126 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:44:25.126 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:44:25.126 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster2 mirror image-primary7 up+unknown 2024-09-12T13:44:25.126 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:44:25.126 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:44:25.126 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:44:25.127 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:44:25.127 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:44:25.127 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:44:25.127 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:44:25.127 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:44:25.127 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:44:25.130 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: remote image is not primary 2024-09-12T13:44:25.131 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:44:25.133 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary4: 2024-09-12T13:44:25.133 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 747f0ef0-de52-4d30-a780-d09d65c61a3b 2024-09-12T13:44:25.133 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+unknown 2024-09-12T13:44:25.133 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: remote image is not primary 2024-09-12T13:44:25.133 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:44:25.134 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:44:17' 2024-09-12T13:44:25.134 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:44:25.134 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:44:25.134 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:44:25.138 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:44:25.138 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+unknown 2024-09-12T13:44:25.138 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:44:25.138 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:44:25.143 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: remote image is not primary 2024-09-12T13:44:25.143 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:44:25.144 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:44:25.144 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:44:25.145 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:44:25.145 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:44:25.147 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:44:25.150 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:44:25.151 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:44:25.152 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:44:17' 2024-09-12T13:44:25.152 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster2 mirror image-primary6 up+unknown '2024-09-12 13:44:17' 2024-09-12T13:44:25.152 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:44:25.152 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:44:25.152 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:44:25.153 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:44:25.153 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:44:17' 2024-09-12T13:44:25.153 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:44:25.154 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:44:25.154 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2024-09-12T13:44:25.156 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:44:25.156 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:44:25.158 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:44:25.165 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:44:17' 2024-09-12T13:44:25.166 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster2 mirror image-primary4 up+unknown '2024-09-12 13:44:17' 2024-09-12T13:44:25.166 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:44:25.166 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:44:25.166 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:44:25.166 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:44:25.166 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:44:17' 2024-09-12T13:44:25.166 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:44:25.167 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:44:25.167 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2024-09-12T13:44:25.420 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041924231mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary199b2aeba-005c-4935-8fc2-c4175a9a85c1up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:44:17image-primary102e48042f-ba7f-45c0-9287-a6bae29dd722up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:44:17image-primary2603c11d9-89c3-4bd2-8c39-0d4ed9ed3093up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:44:17image-primary36eed8ffb-1fae-4e8b-9794-cf7af72eafbcup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:44:17image-primary4747f0ef0-de52-4d30-a780-d09d65c61a3bup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:44:17image-primary519fef5ab-79b5-4712-8250-75fffa450f0aup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:44:17image-primary685abb8e7-1989-4d4a-a874-9bc560cf4e0dup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:44:17image-primary76ca46845-7741-492c-99fb-642d3ab53fabup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:44:17image-primary811646481-daec-461e-b7dc-ed4d9a9fd543up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:44:17image-primary9b98c3fed-4807-46cf-b9ef-4cbefc529d10up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:44:17' 2024-09-12T13:44:25.420 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:44:25.421 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary4'\'']/last_update' 2024-09-12T13:44:25.426 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041924231mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary199b2aeba-005c-4935-8fc2-c4175a9a85c1up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:44:17image-primary102e48042f-ba7f-45c0-9287-a6bae29dd722up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:44:17image-primary2603c11d9-89c3-4bd2-8c39-0d4ed9ed3093up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:44:17image-primary36eed8ffb-1fae-4e8b-9794-cf7af72eafbcup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:44:17image-primary4747f0ef0-de52-4d30-a780-d09d65c61a3bup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:44:17image-primary519fef5ab-79b5-4712-8250-75fffa450f0aup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:44:17image-primary685abb8e7-1989-4d4a-a874-9bc560cf4e0dup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:44:17image-primary76ca46845-7741-492c-99fb-642d3ab53fabup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:44:17image-primary811646481-daec-461e-b7dc-ed4d9a9fd543up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:44:17image-primary9b98c3fed-4807-46cf-b9ef-4cbefc529d10up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:44:17' 2024-09-12T13:44:25.427 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:44:25.428 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary6'\'']/last_update' 2024-09-12T13:44:25.428 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:44:17' 2024-09-12T13:44:25.430 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary4'\'']/state' 2024-09-12T13:44:25.436 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:44:17' 2024-09-12T13:44:25.437 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+unknown 2024-09-12T13:44:25.438 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary6'\'']/state' 2024-09-12T13:44:25.439 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:44:25.440 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+unknown 2024-09-12T13:44:25.444 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:44:25.446 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:44:25.446 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:44:25.446 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ promote_image cluster2 mirror image-primary4 2024-09-12T13:44:25.446 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:44:25.447 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:44:25.447 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:44:25.447 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local force= 2024-09-12T13:44:25.447 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+unknown 2024-09-12T13:44:25.447 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 mirror image promote mirror/image-primary4 2024-09-12T13:44:25.448 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:44:25.448 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+unknown 2024-09-12T13:44:25.451 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:44:25.453 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:44:25.453 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:44:25.453 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ promote_image cluster2 mirror image-primary6 2024-09-12T13:44:25.453 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:44:25.453 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:44:25.453 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:44:25.453 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local force= 2024-09-12T13:44:25.454 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 mirror image promote mirror/image-primary6 2024-09-12T13:44:26.115 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster2 mirror image-primary1 up+unknown '' '' 2024-09-12T13:44:26.115 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:44:26.115 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:44:26.115 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:44:26.115 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:44:26.115 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:44:26.115 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:44:26.116 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:44:26.116 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:44:26.116 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster2 mirror image status mirror/image-primary1 2024-09-12T13:44:26.135 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster2 mirror image-primary7 up+unknown '' '' 2024-09-12T13:44:26.135 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:44:26.135 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:44:26.135 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:44:26.136 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:44:26.136 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:44:26.136 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:44:26.136 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:44:26.137 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:44:26.137 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster2 mirror image status mirror/image-primary7 2024-09-12T13:44:26.259 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary1: 2024-09-12T13:44:26.259 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 99b2aeba-005c-4935-8fc2-c4175a9a85c1 2024-09-12T13:44:26.259 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+unknown 2024-09-12T13:44:26.259 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: remote image is not primary 2024-09-12T13:44:26.259 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:44:26.259 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:44:17' 2024-09-12T13:44:26.259 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:44:26.264 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+unknown 2024-09-12T13:44:26.264 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:44:26.269 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: remote image is not primary 2024-09-12T13:44:26.271 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:44:26.271 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:44:26.272 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:44:26.275 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:44:26.276 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:44:26.280 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:44:26.281 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:44:26.282 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary7: 2024-09-12T13:44:26.282 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 6ca46845-7741-492c-99fb-642d3ab53fab 2024-09-12T13:44:26.282 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+unknown 2024-09-12T13:44:26.282 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: remote image is not primary 2024-09-12T13:44:26.282 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:44:26.282 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:44:17' 2024-09-12T13:44:26.282 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:44:26.282 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:44:26.287 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+unknown 2024-09-12T13:44:26.287 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:44:26.290 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:44:17' 2024-09-12T13:44:26.290 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster2 mirror image-primary1 up+unknown '2024-09-12 13:44:17' 2024-09-12T13:44:26.290 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:44:26.290 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:44:26.290 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:44:26.290 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:44:26.290 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:44:17' 2024-09-12T13:44:26.290 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:44:26.291 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:44:26.293 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2024-09-12T13:44:26.293 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: remote image is not primary 2024-09-12T13:44:26.293 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:44:26.295 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:44:26.296 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:44:26.300 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:44:26.301 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:44:26.306 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:44:26.307 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:44:26.308 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:44:26.315 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:44:17' 2024-09-12T13:44:26.315 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster2 mirror image-primary7 up+unknown '2024-09-12 13:44:17' 2024-09-12T13:44:26.316 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:44:26.316 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:44:26.316 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:44:26.316 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:44:26.316 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:44:17' 2024-09-12T13:44:26.317 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:44:26.317 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:44:26.317 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2024-09-12T13:44:26.515 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Image promoted to primary 2024-09-12T13:44:26.516 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Image promoted to primary 2024-09-12T13:44:26.577 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \n\b\d ]] 2024-09-12T13:44:26.577 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \k\r\b\d ]] 2024-09-12T13:44:26.578 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \n\b\d ]] 2024-09-12T13:44:26.579 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo rbd --cluster cluster2 device map -t krbd mirror/image-primary6 2024-09-12T13:44:26.579 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \k\r\b\d ]] 2024-09-12T13:44:26.581 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041924231mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary199b2aeba-005c-4935-8fc2-c4175a9a85c1up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:44:17image-primary102e48042f-ba7f-45c0-9287-a6bae29dd722up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:44:17image-primary2603c11d9-89c3-4bd2-8c39-0d4ed9ed3093up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:44:17image-primary36eed8ffb-1fae-4e8b-9794-cf7af72eafbcup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:44:17image-primary4747f0ef0-de52-4d30-a780-d09d65c61a3bup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:44:17image-primary519fef5ab-79b5-4712-8250-75fffa450f0aup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:44:17image-primary685abb8e7-1989-4d4a-a874-9bc560cf4e0dup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:44:17image-primary76ca46845-7741-492c-99fb-642d3ab53fabup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:44:17image-primary811646481-daec-461e-b7dc-ed4d9a9fd543up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:44:17image-primary9b98c3fed-4807-46cf-b9ef-4cbefc529d10up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:44:17' 2024-09-12T13:44:26.584 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:44:26.584 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary1'\'']/last_update' 2024-09-12T13:44:26.597 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:44:17' 2024-09-12T13:44:26.599 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary1'\'']/state' 2024-09-12T13:44:26.609 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+unknown 2024-09-12T13:44:26.611 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:44:26.613 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+unknown 2024-09-12T13:44:26.620 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:44:26.621 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:44:26.621 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:44:26.621 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ promote_image cluster2 mirror image-primary1 2024-09-12T13:44:26.621 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:44:26.621 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:44:26.621 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:44:26.622 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local force= 2024-09-12T13:44:26.622 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 mirror image promote mirror/image-primary1 2024-09-12T13:44:26.633 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041924231mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary199b2aeba-005c-4935-8fc2-c4175a9a85c1up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:44:17image-primary102e48042f-ba7f-45c0-9287-a6bae29dd722up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:44:17image-primary2603c11d9-89c3-4bd2-8c39-0d4ed9ed3093up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:44:17image-primary36eed8ffb-1fae-4e8b-9794-cf7af72eafbcup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:44:17image-primary4747f0ef0-de52-4d30-a780-d09d65c61a3bup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:44:17image-primary519fef5ab-79b5-4712-8250-75fffa450f0aup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:44:17image-primary685abb8e7-1989-4d4a-a874-9bc560cf4e0dup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:44:17image-primary76ca46845-7741-492c-99fb-642d3ab53fabup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:44:17image-primary811646481-daec-461e-b7dc-ed4d9a9fd543up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:44:17image-primary9b98c3fed-4807-46cf-b9ef-4cbefc529d10up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:44:17' 2024-09-12T13:44:26.633 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:44:26.635 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary7'\'']/last_update' 2024-09-12T13:44:26.644 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:44:17' 2024-09-12T13:44:26.646 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary7'\'']/state' 2024-09-12T13:44:26.653 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+unknown 2024-09-12T13:44:26.655 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:44:26.657 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+unknown 2024-09-12T13:44:26.660 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:44:26.661 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:44:26.662 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:44:26.662 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ promote_image cluster2 mirror image-primary7 2024-09-12T13:44:26.662 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:44:26.662 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:44:26.662 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:44:26.662 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local force= 2024-09-12T13:44:26.662 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 mirror image promote mirror/image-primary7 2024-09-12T13:44:26.866 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ dev=/dev/rbd6 2024-09-12T13:44:26.868 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo md5sum /dev/rbd6 2024-09-12T13:44:26.869 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ awk '{print $1}' 2024-09-12T13:44:26.939 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ dev=/dev/rbd7 2024-09-12T13:44:26.942 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo md5sum /dev/rbd7 2024-09-12T13:44:26.944 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ awk '{print $1}' 2024-09-12T13:44:27.486 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Image promoted to primary 2024-09-12T13:44:27.486 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Image promoted to primary 2024-09-12T13:44:27.531 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \n\b\d ]] 2024-09-12T13:44:27.532 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \k\r\b\d ]] 2024-09-12T13:44:27.534 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \n\b\d ]] 2024-09-12T13:44:27.534 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \k\r\b\d ]] 2024-09-12T13:44:27.535 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo rbd --cluster cluster2 device map -t krbd mirror/image-primary1 2024-09-12T13:44:27.537 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo rbd --cluster cluster2 device map -t krbd mirror/image-primary7 2024-09-12T13:44:27.946 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ dev=/dev/rbd9 2024-09-12T13:44:27.953 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ dev=/dev/rbd8 2024-09-12T13:44:27.954 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ awk '{print $1}' 2024-09-12T13:44:27.954 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo md5sum /dev/rbd9 2024-09-12T13:44:27.959 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo md5sum /dev/rbd8 2024-09-12T13:44:27.960 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ awk '{print $1}' 2024-09-12T13:45:09.292 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ promote_md5=eeb66717e02189ca91f864aa17d93a8f 2024-09-12T13:45:09.296 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo rbd --cluster cluster2 device unmap -t krbd /dev/rbd0 2024-09-12T13:45:09.924 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ eeb66717e02189ca91f864aa17d93a8f != \e\e\b\6\6\7\1\7\e\0\2\1\8\9\c\a\9\1\f\8\6\4\a\a\1\7\d\9\3\a\8\f ]] 2024-09-12T13:45:15.730 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ promote_md5=2b6769f814133ccdb1f87119664c5c38 2024-09-12T13:45:15.731 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo rbd --cluster cluster2 device unmap -t krbd /dev/rbd1 2024-09-12T13:45:15.964 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ 2b6769f814133ccdb1f87119664c5c38 != \2\b\6\7\6\9\f\8\1\4\1\3\3\c\c\d\b\1\f\8\7\1\1\9\6\6\4\c\5\c\3\8 ]] 2024-09-12T13:45:17.445 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ promote_md5=78e3674d0ba59fb198419c279aaa3a35 2024-09-12T13:45:17.445 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo rbd --cluster cluster2 device unmap -t krbd /dev/rbd5 2024-09-12T13:45:17.624 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ 78e3674d0ba59fb198419c279aaa3a35 != \7\8\e\3\6\7\4\d\0\b\a\5\9\f\b\1\9\8\4\1\9\c\2\7\9\a\a\a\3\a\3\5 ]] 2024-09-12T13:45:18.200 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ promote_md5=ab12518a14de3ef37d42312f3d912429 2024-09-12T13:45:18.200 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo rbd --cluster cluster2 device unmap -t krbd /dev/rbd4 2024-09-12T13:45:18.441 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ ab12518a14de3ef37d42312f3d912429 != \a\b\1\2\5\1\8\a\1\4\d\e\3\e\f\3\7\d\4\2\3\1\2\f\3\d\9\1\2\4\2\9 ]] 2024-09-12T13:45:19.259 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ promote_md5=0bb624ca6718db88f78b6d0e3552396c 2024-09-12T13:45:19.259 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo rbd --cluster cluster2 device unmap -t krbd /dev/rbd2 2024-09-12T13:45:19.270 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ promote_md5=9ea431d0695a08b4407dbabbb30171c8 2024-09-12T13:45:19.270 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo rbd --cluster cluster2 device unmap -t krbd /dev/rbd3 2024-09-12T13:45:19.392 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ 9ea431d0695a08b4407dbabbb30171c8 != \9\e\a\4\3\1\d\0\6\9\5\a\0\8\b\4\4\0\7\d\b\a\b\b\b\3\0\1\7\1\c\8 ]] 2024-09-12T13:45:19.451 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ 0bb624ca6718db88f78b6d0e3552396c != \0\b\b\6\2\4\c\a\6\7\1\8\d\b\8\8\f\7\8\b\6\d\0\e\3\5\5\2\3\9\6\c ]] 2024-09-12T13:45:19.670 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ promote_md5=d538fc510fb812e420e71721db55e508 2024-09-12T13:45:19.670 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo rbd --cluster cluster2 device unmap -t krbd /dev/rbd7 2024-09-12T13:45:19.778 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ d538fc510fb812e420e71721db55e508 != \d\5\3\8\f\c\5\1\0\f\b\8\1\2\e\4\2\0\e\7\1\7\2\1\d\b\5\5\e\5\0\8 ]] 2024-09-12T13:45:19.883 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ promote_md5=2bfe718220eddfa00fb7325b61ba4af3 2024-09-12T13:45:19.883 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo rbd --cluster cluster2 device unmap -t krbd /dev/rbd6 2024-09-12T13:45:20.044 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ 2bfe718220eddfa00fb7325b61ba4af3 != \2\b\f\e\7\1\8\2\2\0\e\d\d\f\a\0\0\f\b\7\3\2\5\b\6\1\b\a\4\a\f\3 ]] 2024-09-12T13:45:20.833 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ promote_md5=e4e69be8d6a6fb00c74d3e090d096b6e 2024-09-12T13:45:20.833 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo rbd --cluster cluster2 device unmap -t krbd /dev/rbd9 2024-09-12T13:45:20.923 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ e4e69be8d6a6fb00c74d3e090d096b6e != \e\4\e\6\9\b\e\8\d\6\a\6\f\b\0\0\c\7\4\d\3\e\0\9\0\d\0\9\6\b\6\e ]] 2024-09-12T13:45:22.021 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ promote_md5=4ef5fd590827cc3f057aaa2cbbc80493 2024-09-12T13:45:22.021 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo rbd --cluster cluster2 device unmap -t krbd /dev/rbd8 2024-09-12T13:45:22.107 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ 4ef5fd590827cc3f057aaa2cbbc80493 != \4\e\f\5\f\d\5\9\0\8\2\7\c\c\3\f\0\5\7\a\a\a\2\c\b\b\c\8\0\4\9\3 ]] 2024-09-12T13:45:22.107 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${COMPARE_PIDS[@]} 2024-09-12T13:45:22.107 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 101847 2024-09-12T13:45:22.107 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${COMPARE_PIDS[@]} 2024-09-12T13:45:22.107 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 101849 2024-09-12T13:45:22.107 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${COMPARE_PIDS[@]} 2024-09-12T13:45:22.107 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 101851 2024-09-12T13:45:22.107 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${COMPARE_PIDS[@]} 2024-09-12T13:45:22.108 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 101853 2024-09-12T13:45:22.108 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${COMPARE_PIDS[@]} 2024-09-12T13:45:22.108 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 101855 2024-09-12T13:45:22.108 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${COMPARE_PIDS[@]} 2024-09-12T13:45:22.108 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 101857 2024-09-12T13:45:22.108 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${COMPARE_PIDS[@]} 2024-09-12T13:45:22.108 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 101859 2024-09-12T13:45:22.108 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${COMPARE_PIDS[@]} 2024-09-12T13:45:22.108 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 101861 2024-09-12T13:45:22.108 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${COMPARE_PIDS[@]} 2024-09-12T13:45:22.108 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 101863 2024-09-12T13:45:22.108 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ (( RET != 0 )) 2024-09-12T13:45:22.108 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:45:22.109 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ IMG=image-primary1 2024-09-12T13:45:22.109 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_replaying_status_in_pool_dir cluster1 mirror image-primary1 2024-09-12T13:45:22.109 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:45:22.109 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:45:22.109 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:45:22.109 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' snapshot = journal ']' 2024-09-12T13:45:22.109 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary1 up+replaying 2024-09-12T13:45:22.109 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:45:22.109 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:45:22.109 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:45:22.109 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:45:22.109 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:45:22.109 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:45:22.110 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:45:22.110 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:45:23.229 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary1 up+replaying '' '' 2024-09-12T13:45:23.229 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:45:23.229 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:45:23.229 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:45:23.230 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:45:23.230 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:45:23.230 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:45:23.230 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:45:23.230 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:45:23.230 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary1 2024-09-12T13:45:23.313 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary1: 2024-09-12T13:45:23.313 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 99b2aeba-005c-4935-8fc2-c4175a9a85c1 2024-09-12T13:45:23.313 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+replaying 2024-09-12T13:45:23.313 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: replaying, {"bytes_per_second":3334553.6,"bytes_per_snapshot":66691072.0,"last_snapshot_bytes":66691072,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726148667,"remote_snapshot_timestamp":1726148667,"replay_state":"idle"} 2024-09-12T13:45:23.313 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:45:23.313 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:45:17' 2024-09-12T13:45:23.313 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+replaying' 2024-09-12T13:45:23.316 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+replaying 2024-09-12T13:45:23.316 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:45:23.318 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: replaying, {"bytes_per_second":3334553.6,"bytes_per_snapshot":66691072.0,"last_snapshot_bytes":66691072,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726148667,"remote_snapshot_timestamp":1726148667,"replay_state":"idle"} 2024-09-12T13:45:23.319 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:45:23.319 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+replaying 2024-09-12T13:45:23.320 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:45:23.321 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+replaying 2024-09-12T13:45:23.322 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:45:23.325 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:45:23.325 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:45:23.326 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:45:23.330 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:45:17' 2024-09-12T13:45:23.330 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary1 up+replaying '2024-09-12 13:45:17' 2024-09-12T13:45:23.330 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:45:23.330 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:45:23.330 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:45:23.330 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:45:23.330 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:45:17' 2024-09-12T13:45:23.330 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:45:23.331 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:45:23.331 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-09-12T13:45:23.440 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041934232mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary199b2aeba-005c-4935-8fc2-c4175a9a85c1up+replayingreplaying, {"bytes_per_second":3334553.6,"bytes_per_snapshot":66691072.0,"last_snapshot_bytes":66691072,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726148667,"remote_snapshot_timestamp":1726148667,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:45:17image-primary102e48042f-ba7f-45c0-9287-a6bae29dd722up+replayingreplaying, {"bytes_per_second":3311411.2,"bytes_per_snapshot":66228224.0,"last_snapshot_bytes":66228224,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726148664,"remote_snapshot_timestamp":1726148664,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:45:17image-primary2603c11d9-89c3-4bd2-8c39-0d4ed9ed3093up+replayingreplaying, {"bytes_per_second":3304448.0,"bytes_per_snapshot":66088960.0,"last_snapshot_bytes":66088960,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726148662,"remote_snapshot_timestamp":1726148662,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:45:17image-primary36eed8ffb-1fae-4e8b-9794-cf7af72eafbcup+replayingreplaying, {"bytes_per_second":3231539.2,"bytes_per_snapshot":64630784.0,"last_snapshot_bytes":64630784,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726148664,"remote_snapshot_timestamp":1726148664,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:45:17image-primary4747f0ef0-de52-4d30-a780-d09d65c61a3bup+replayingreplaying, {"bytes_per_second":3382067.2,"bytes_per_snapshot":67641344.0,"last_snapshot_bytes":67641344,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726148666,"remote_snapshot_timestamp":1726148666,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:45:17image-primary519fef5ab-79b5-4712-8250-75fffa450f0aup+replayingreplaying, {"bytes_per_second":3340083.2,"bytes_per_snapshot":66801664.0,"last_snapshot_bytes":66801664,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726148664,"remote_snapshot_timestamp":1726148664,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:45:17image-primary685abb8e7-1989-4d4a-a874-9bc560cf4e0dup+replayingreplaying, {"bytes_per_second":3262873.6,"bytes_per_snapshot":65257472.0,"last_snapshot_bytes":65257472,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726148666,"remote_snapshot_timestamp":1726148666,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:45:17image-primary76ca46845-7741-492c-99fb-642d3ab53fabup+replayingreplaying, {"bytes_per_second":3228876.8,"bytes_per_snapshot":64577536.0,"last_snapshot_bytes":64577536,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726148667,"remote_snapshot_timestamp":1726148667,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:45:17image-primary811646481-daec-461e-b7dc-ed4d9a9fd543up+replayingreplaying, {"bytes_per_second":3386982.4,"bytes_per_snapshot":67739648.0,"last_snapshot_bytes":67739648,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726148664,"remote_snapshot_timestamp":1726148664,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:45:17image-primary9b98c3fed-4807-46cf-b9ef-4cbefc529d10up+replayingreplaying, {"bytes_per_second":3216179.2,"bytes_per_snapshot":64323584.0,"last_snapshot_bytes":64323584,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726148661,"remote_snapshot_timestamp":1726148661,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:45:17' 2024-09-12T13:45:23.441 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:45:23.441 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary1'\'']/last_update' 2024-09-12T13:45:23.537 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:45:17' 2024-09-12T13:45:23.538 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary1'\'']/state' 2024-09-12T13:45:23.541 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+replaying 2024-09-12T13:45:23.542 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+replaying 2024-09-12T13:45:23.542 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+replaying 2024-09-12T13:45:23.544 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+replaying 2024-09-12T13:45:23.545 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:45:23.545 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:45:23.545 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ remove_image cluster2 mirror image-primary1 2024-09-12T13:45:23.545 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:45:23.545 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:45:23.545 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:45:23.546 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 snap purge mirror/image-primary1 2024-09-12T13:45:23.606 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 rm mirror/image-primary1 2024-09-12T13:45:23.749 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:45:23.749+0000 7f67e5200640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x5605571a0000 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-09-12T13:45:23.751 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:45:23.749+0000 7f67ed400640 -1 rbd::mirror::ImageReplayer: 0x5605561ac500 [3/99b2aeba-005c-4935-8fc2-c4175a9a85c1] operator(): start failed: (85) Interrupted system call should be restarted 2024-09-12T13:45:25.571 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-09-12T13:45:25.577 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_image_removal cluster1 mirror image-primary1 2024-09-12T13:45:25.577 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:45:25.577 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:45:25.577 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:45:25.577 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:45:25.578 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 ls mirror 2024-09-12T13:45:25.578 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep -wq image-primary1 2024-09-12T13:45:25.628 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:45:25.628 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rm -rf test-primary1 2024-09-12T13:45:25.830 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:45:25.830 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ IMG=image-primary2 2024-09-12T13:45:25.830 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_replaying_status_in_pool_dir cluster1 mirror image-primary2 2024-09-12T13:45:25.830 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:45:25.830 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:45:25.830 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:45:25.830 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' snapshot = journal ']' 2024-09-12T13:45:25.831 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary2 up+replaying 2024-09-12T13:45:25.831 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:45:25.831 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:45:25.831 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:45:25.831 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:45:25.831 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:45:25.831 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:45:25.831 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:45:25.831 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:45:26.832 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary2 up+replaying '' '' 2024-09-12T13:45:26.833 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:45:26.833 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:45:26.833 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:45:26.833 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:45:26.833 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:45:26.833 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:45:26.833 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:45:26.834 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:45:26.834 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary2 2024-09-12T13:45:26.916 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary2: 2024-09-12T13:45:26.916 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 603c11d9-89c3-4bd2-8c39-0d4ed9ed3093 2024-09-12T13:45:26.916 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+replaying 2024-09-12T13:45:26.916 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: replaying, {"bytes_per_second":3304448.0,"bytes_per_snapshot":66088960.0,"last_snapshot_bytes":66088960,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726148662,"remote_snapshot_timestamp":1726148662,"replay_state":"idle"} 2024-09-12T13:45:26.916 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:45:26.916 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:45:17' 2024-09-12T13:45:26.916 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+replaying' 2024-09-12T13:45:26.918 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+replaying 2024-09-12T13:45:26.918 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:45:26.921 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: replaying, {"bytes_per_second":3304448.0,"bytes_per_snapshot":66088960.0,"last_snapshot_bytes":66088960,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726148662,"remote_snapshot_timestamp":1726148662,"replay_state":"idle"} 2024-09-12T13:45:26.921 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:45:26.922 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+replaying 2024-09-12T13:45:26.922 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:45:26.924 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+replaying 2024-09-12T13:45:26.925 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:45:26.927 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:45:26.928 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:45:26.928 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:45:26.932 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:45:17' 2024-09-12T13:45:26.932 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary2 up+replaying '2024-09-12 13:45:17' 2024-09-12T13:45:26.933 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:45:26.933 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:45:26.933 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:45:26.933 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:45:26.933 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:45:17' 2024-09-12T13:45:26.933 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:45:26.933 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:45:26.933 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-09-12T13:45:27.041 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING941934232mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary102e48042f-ba7f-45c0-9287-a6bae29dd722up+replayingreplaying, {"bytes_per_second":3311411.2,"bytes_per_snapshot":66228224.0,"last_snapshot_bytes":66228224,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726148664,"remote_snapshot_timestamp":1726148664,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:45:17image-primary2603c11d9-89c3-4bd2-8c39-0d4ed9ed3093up+replayingreplaying, {"bytes_per_second":3304448.0,"bytes_per_snapshot":66088960.0,"last_snapshot_bytes":66088960,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726148662,"remote_snapshot_timestamp":1726148662,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:45:17image-primary36eed8ffb-1fae-4e8b-9794-cf7af72eafbcup+replayingreplaying, {"bytes_per_second":3231539.2,"bytes_per_snapshot":64630784.0,"last_snapshot_bytes":64630784,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726148664,"remote_snapshot_timestamp":1726148664,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:45:17image-primary4747f0ef0-de52-4d30-a780-d09d65c61a3bup+replayingreplaying, {"bytes_per_second":3382067.2,"bytes_per_snapshot":67641344.0,"last_snapshot_bytes":67641344,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726148666,"remote_snapshot_timestamp":1726148666,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:45:17image-primary519fef5ab-79b5-4712-8250-75fffa450f0aup+replayingreplaying, {"bytes_per_second":3340083.2,"bytes_per_snapshot":66801664.0,"last_snapshot_bytes":66801664,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726148664,"remote_snapshot_timestamp":1726148664,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:45:17image-primary685abb8e7-1989-4d4a-a874-9bc560cf4e0dup+replayingreplaying, {"bytes_per_second":3262873.6,"bytes_per_snapshot":65257472.0,"last_snapshot_bytes":65257472,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726148666,"remote_snapshot_timestamp":1726148666,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:45:17image-primary76ca46845-7741-492c-99fb-642d3ab53fabup+replayingreplaying, {"bytes_per_second":3228876.8,"bytes_per_snapshot":64577536.0,"last_snapshot_bytes":64577536,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726148667,"remote_snapshot_timestamp":1726148667,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:45:17image-primary811646481-daec-461e-b7dc-ed4d9a9fd543up+replayingreplaying, {"bytes_per_second":3386982.4,"bytes_per_snapshot":67739648.0,"last_snapshot_bytes":67739648,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726148664,"remote_snapshot_timestamp":1726148664,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:45:17image-primary9b98c3fed-4807-46cf-b9ef-4cbefc529d10up+replayingreplaying, {"bytes_per_second":3216179.2,"bytes_per_snapshot":64323584.0,"last_snapshot_bytes":64323584,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726148661,"remote_snapshot_timestamp":1726148661,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:45:17' 2024-09-12T13:45:27.041 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:45:27.042 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary2'\'']/last_update' 2024-09-12T13:45:27.046 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:45:17' 2024-09-12T13:45:27.046 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary2'\'']/state' 2024-09-12T13:45:27.050 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+replaying 2024-09-12T13:45:27.051 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+replaying 2024-09-12T13:45:27.051 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+replaying 2024-09-12T13:45:27.053 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+replaying 2024-09-12T13:45:27.056 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:45:27.056 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:45:27.056 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ remove_image cluster2 mirror image-primary2 2024-09-12T13:45:27.056 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:45:27.056 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:45:27.056 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:45:27.056 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 snap purge mirror/image-primary2 2024-09-12T13:45:27.114 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 rm mirror/image-primary2 2024-09-12T13:45:27.255 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:45:27.255+0000 7f67e5200640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x56055b865a00 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-09-12T13:45:27.255 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:45:27.255+0000 7f67ed400640 -1 rbd::mirror::ImageReplayer: 0x5605561acf00 [3/603c11d9-89c3-4bd2-8c39-0d4ed9ed3093] operator(): start failed: (85) Interrupted system call should be restarted 2024-09-12T13:45:28.596 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-09-12T13:45:28.602 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_image_removal cluster1 mirror image-primary2 2024-09-12T13:45:28.602 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:45:28.602 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:45:28.602 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:45:28.602 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:45:28.603 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 ls mirror 2024-09-12T13:45:28.603 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep -wq image-primary2 2024-09-12T13:45:28.652 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:45:28.652 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rm -rf test-primary2 2024-09-12T13:45:28.655 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:45:28.655 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ IMG=image-primary3 2024-09-12T13:45:28.655 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_replaying_status_in_pool_dir cluster1 mirror image-primary3 2024-09-12T13:45:28.655 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:45:28.655 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:45:28.655 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:45:28.655 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' snapshot = journal ']' 2024-09-12T13:45:28.655 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary3 up+replaying 2024-09-12T13:45:28.655 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:45:28.655 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:45:28.655 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:45:28.656 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:45:28.656 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:45:28.656 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:45:28.656 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:45:28.656 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:45:29.657 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary3 up+replaying '' '' 2024-09-12T13:45:29.657 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:45:29.657 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:45:29.657 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:45:29.657 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:45:29.658 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:45:29.658 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:45:29.658 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:45:29.658 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:45:29.658 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary3 2024-09-12T13:45:29.741 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary3: 2024-09-12T13:45:29.742 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 6eed8ffb-1fae-4e8b-9794-cf7af72eafbc 2024-09-12T13:45:29.742 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+replaying 2024-09-12T13:45:29.742 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: replaying, {"bytes_per_second":3231539.2,"bytes_per_snapshot":64630784.0,"last_snapshot_bytes":64630784,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726148664,"remote_snapshot_timestamp":1726148664,"replay_state":"idle"} 2024-09-12T13:45:29.742 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:45:29.742 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:45:17' 2024-09-12T13:45:29.742 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+replaying' 2024-09-12T13:45:29.744 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+replaying 2024-09-12T13:45:29.744 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:45:29.747 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: replaying, {"bytes_per_second":3231539.2,"bytes_per_snapshot":64630784.0,"last_snapshot_bytes":64630784,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726148664,"remote_snapshot_timestamp":1726148664,"replay_state":"idle"} 2024-09-12T13:45:29.747 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:45:29.748 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+replaying 2024-09-12T13:45:29.748 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:45:29.751 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+replaying 2024-09-12T13:45:29.751 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:45:29.754 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:45:29.754 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:45:29.755 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:45:29.759 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:45:17' 2024-09-12T13:45:29.759 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary3 up+replaying '2024-09-12 13:45:17' 2024-09-12T13:45:29.759 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:45:29.759 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:45:29.759 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:45:29.759 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:45:29.759 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:45:17' 2024-09-12T13:45:29.759 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:45:29.760 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:45:29.760 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-09-12T13:45:29.864 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING841934232mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary102e48042f-ba7f-45c0-9287-a6bae29dd722up+replayingreplaying, {"bytes_per_second":3311411.2,"bytes_per_snapshot":66228224.0,"last_snapshot_bytes":66228224,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726148664,"remote_snapshot_timestamp":1726148664,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:45:17image-primary36eed8ffb-1fae-4e8b-9794-cf7af72eafbcup+replayingreplaying, {"bytes_per_second":3231539.2,"bytes_per_snapshot":64630784.0,"last_snapshot_bytes":64630784,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726148664,"remote_snapshot_timestamp":1726148664,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:45:17image-primary4747f0ef0-de52-4d30-a780-d09d65c61a3bup+replayingreplaying, {"bytes_per_second":3382067.2,"bytes_per_snapshot":67641344.0,"last_snapshot_bytes":67641344,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726148666,"remote_snapshot_timestamp":1726148666,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:45:17image-primary519fef5ab-79b5-4712-8250-75fffa450f0aup+replayingreplaying, {"bytes_per_second":3340083.2,"bytes_per_snapshot":66801664.0,"last_snapshot_bytes":66801664,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726148664,"remote_snapshot_timestamp":1726148664,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:45:17image-primary685abb8e7-1989-4d4a-a874-9bc560cf4e0dup+replayingreplaying, {"bytes_per_second":3262873.6,"bytes_per_snapshot":65257472.0,"last_snapshot_bytes":65257472,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726148666,"remote_snapshot_timestamp":1726148666,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:45:17image-primary76ca46845-7741-492c-99fb-642d3ab53fabup+replayingreplaying, {"bytes_per_second":3228876.8,"bytes_per_snapshot":64577536.0,"last_snapshot_bytes":64577536,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726148667,"remote_snapshot_timestamp":1726148667,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:45:17image-primary811646481-daec-461e-b7dc-ed4d9a9fd543up+replayingreplaying, {"bytes_per_second":3386982.4,"bytes_per_snapshot":67739648.0,"last_snapshot_bytes":67739648,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726148664,"remote_snapshot_timestamp":1726148664,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:45:17image-primary9b98c3fed-4807-46cf-b9ef-4cbefc529d10up+replayingreplaying, {"bytes_per_second":3216179.2,"bytes_per_snapshot":64323584.0,"last_snapshot_bytes":64323584,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726148661,"remote_snapshot_timestamp":1726148661,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:45:17' 2024-09-12T13:45:29.864 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:45:29.865 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary3'\'']/last_update' 2024-09-12T13:45:29.868 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:45:17' 2024-09-12T13:45:29.869 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary3'\'']/state' 2024-09-12T13:45:29.873 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+replaying 2024-09-12T13:45:29.874 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+replaying 2024-09-12T13:45:29.874 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+replaying 2024-09-12T13:45:29.876 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+replaying 2024-09-12T13:45:29.877 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:45:29.877 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:45:29.877 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ remove_image cluster2 mirror image-primary3 2024-09-12T13:45:29.877 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:45:29.877 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:45:29.877 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:45:29.877 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 snap purge mirror/image-primary3 2024-09-12T13:45:29.937 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 rm mirror/image-primary3 2024-09-12T13:45:30.084 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:45:30.085+0000 7f67e5200640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x56055b973520 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-09-12T13:45:30.085 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:45:30.085+0000 7f67ed400640 -1 rbd::mirror::ImageReplayer: 0x5605561ad400 [3/6eed8ffb-1fae-4e8b-9794-cf7af72eafbc] operator(): start failed: (85) Interrupted system call should be restarted 2024-09-12T13:45:31.321 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-09-12T13:45:31.326 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_image_removal cluster1 mirror image-primary3 2024-09-12T13:45:31.326 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:45:31.326 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:45:31.326 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:45:31.327 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:45:31.327 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 ls mirror 2024-09-12T13:45:31.327 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep -wq image-primary3 2024-09-12T13:45:31.383 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:45:31.383 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rm -rf test-primary3 2024-09-12T13:45:31.386 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:45:31.386 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ IMG=image-primary4 2024-09-12T13:45:31.386 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_replaying_status_in_pool_dir cluster1 mirror image-primary4 2024-09-12T13:45:31.386 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:45:31.386 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:45:31.386 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:45:31.387 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' snapshot = journal ']' 2024-09-12T13:45:31.387 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary4 up+replaying 2024-09-12T13:45:31.387 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:45:31.387 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:45:31.387 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:45:31.387 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:45:31.387 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:45:31.387 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:45:31.387 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:45:31.387 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:45:32.388 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary4 up+replaying '' '' 2024-09-12T13:45:32.388 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:45:32.388 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:45:32.388 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:45:32.389 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:45:32.389 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:45:32.389 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:45:32.389 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:45:32.389 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:45:32.390 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary4 2024-09-12T13:45:32.462 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary4: 2024-09-12T13:45:32.462 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 747f0ef0-de52-4d30-a780-d09d65c61a3b 2024-09-12T13:45:32.462 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+replaying 2024-09-12T13:45:32.463 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: replaying, {"bytes_per_second":3382067.2,"bytes_per_snapshot":67641344.0,"last_snapshot_bytes":67641344,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726148666,"remote_snapshot_timestamp":1726148666,"replay_state":"idle"} 2024-09-12T13:45:32.463 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:45:32.463 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:45:17' 2024-09-12T13:45:32.463 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+replaying' 2024-09-12T13:45:32.465 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+replaying 2024-09-12T13:45:32.465 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:45:32.467 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: replaying, {"bytes_per_second":3382067.2,"bytes_per_snapshot":67641344.0,"last_snapshot_bytes":67641344,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726148666,"remote_snapshot_timestamp":1726148666,"replay_state":"idle"} 2024-09-12T13:45:32.468 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:45:32.468 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+replaying 2024-09-12T13:45:32.469 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:45:32.470 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+replaying 2024-09-12T13:45:32.471 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:45:32.473 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:45:32.474 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:45:32.474 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:45:32.478 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:45:17' 2024-09-12T13:45:32.478 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary4 up+replaying '2024-09-12 13:45:17' 2024-09-12T13:45:32.478 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:45:32.478 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:45:32.479 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:45:32.479 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:45:32.479 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:45:17' 2024-09-12T13:45:32.479 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:45:32.479 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:45:32.479 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-09-12T13:45:32.574 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING741934232mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary102e48042f-ba7f-45c0-9287-a6bae29dd722up+replayingreplaying, {"bytes_per_second":3311411.2,"bytes_per_snapshot":66228224.0,"last_snapshot_bytes":66228224,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726148664,"remote_snapshot_timestamp":1726148664,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:45:17image-primary4747f0ef0-de52-4d30-a780-d09d65c61a3bup+replayingreplaying, {"bytes_per_second":3382067.2,"bytes_per_snapshot":67641344.0,"last_snapshot_bytes":67641344,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726148666,"remote_snapshot_timestamp":1726148666,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:45:17image-primary519fef5ab-79b5-4712-8250-75fffa450f0aup+replayingreplaying, {"bytes_per_second":3340083.2,"bytes_per_snapshot":66801664.0,"last_snapshot_bytes":66801664,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726148664,"remote_snapshot_timestamp":1726148664,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:45:17image-primary685abb8e7-1989-4d4a-a874-9bc560cf4e0dup+replayingreplaying, {"bytes_per_second":3262873.6,"bytes_per_snapshot":65257472.0,"last_snapshot_bytes":65257472,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726148666,"remote_snapshot_timestamp":1726148666,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:45:17image-primary76ca46845-7741-492c-99fb-642d3ab53fabup+replayingreplaying, {"bytes_per_second":3228876.8,"bytes_per_snapshot":64577536.0,"last_snapshot_bytes":64577536,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726148667,"remote_snapshot_timestamp":1726148667,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:45:17image-primary811646481-daec-461e-b7dc-ed4d9a9fd543up+replayingreplaying, {"bytes_per_second":3386982.4,"bytes_per_snapshot":67739648.0,"last_snapshot_bytes":67739648,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726148664,"remote_snapshot_timestamp":1726148664,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:45:17image-primary9b98c3fed-4807-46cf-b9ef-4cbefc529d10up+replayingreplaying, {"bytes_per_second":3216179.2,"bytes_per_snapshot":64323584.0,"last_snapshot_bytes":64323584,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726148661,"remote_snapshot_timestamp":1726148661,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:45:17' 2024-09-12T13:45:32.574 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:45:32.575 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary4'\'']/last_update' 2024-09-12T13:45:32.578 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:45:17' 2024-09-12T13:45:32.579 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary4'\'']/state' 2024-09-12T13:45:32.583 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+replaying 2024-09-12T13:45:32.583 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+replaying 2024-09-12T13:45:32.584 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+replaying 2024-09-12T13:45:32.586 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+replaying 2024-09-12T13:45:32.587 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:45:32.587 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:45:32.587 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ remove_image cluster2 mirror image-primary4 2024-09-12T13:45:32.587 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:45:32.587 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:45:32.587 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:45:32.587 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 snap purge mirror/image-primary4 2024-09-12T13:45:32.646 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 rm mirror/image-primary4 2024-09-12T13:45:32.786 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:45:32.787+0000 7f67e4800640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x56055ba3fd40 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-09-12T13:45:32.786 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:45:32.787+0000 7f67eca00640 -1 rbd::mirror::ImageReplayer: 0x5605573cb180 [3/747f0ef0-de52-4d30-a780-d09d65c61a3b] operator(): start failed: (85) Interrupted system call should be restarted 2024-09-12T13:45:34.322 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-09-12T13:45:34.328 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_image_removal cluster1 mirror image-primary4 2024-09-12T13:45:34.328 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:45:34.328 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:45:34.328 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:45:34.328 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:45:34.328 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 ls mirror 2024-09-12T13:45:34.329 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep -wq image-primary4 2024-09-12T13:45:34.378 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:45:34.378 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rm -rf test-primary4 2024-09-12T13:45:34.381 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:45:34.381 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ IMG=image-primary5 2024-09-12T13:45:34.381 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_replaying_status_in_pool_dir cluster1 mirror image-primary5 2024-09-12T13:45:34.381 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:45:34.381 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:45:34.381 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:45:34.381 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' snapshot = journal ']' 2024-09-12T13:45:34.382 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary5 up+replaying 2024-09-12T13:45:34.382 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:45:34.382 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:45:34.382 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:45:34.382 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:45:34.382 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:45:34.382 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:45:34.382 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:45:34.382 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:45:35.383 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary5 up+replaying '' '' 2024-09-12T13:45:35.383 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:45:35.383 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:45:35.383 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:45:35.383 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:45:35.384 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:45:35.384 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:45:35.384 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:45:35.384 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:45:35.384 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary5 2024-09-12T13:45:35.465 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary5: 2024-09-12T13:45:35.465 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 19fef5ab-79b5-4712-8250-75fffa450f0a 2024-09-12T13:45:35.465 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+replaying 2024-09-12T13:45:35.465 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: replaying, {"bytes_per_second":3340083.2,"bytes_per_snapshot":66801664.0,"last_snapshot_bytes":66801664,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726148664,"remote_snapshot_timestamp":1726148664,"replay_state":"idle"} 2024-09-12T13:45:35.466 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:45:35.466 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:45:17' 2024-09-12T13:45:35.466 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+replaying' 2024-09-12T13:45:35.468 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+replaying 2024-09-12T13:45:35.468 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:45:35.471 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: replaying, {"bytes_per_second":3340083.2,"bytes_per_snapshot":66801664.0,"last_snapshot_bytes":66801664,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726148664,"remote_snapshot_timestamp":1726148664,"replay_state":"idle"} 2024-09-12T13:45:35.471 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:45:35.472 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+replaying 2024-09-12T13:45:35.472 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:45:35.474 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+replaying 2024-09-12T13:45:35.474 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:45:35.477 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:45:35.477 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:45:35.478 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:45:35.482 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:45:17' 2024-09-12T13:45:35.482 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary5 up+replaying '2024-09-12 13:45:17' 2024-09-12T13:45:35.482 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:45:35.482 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:45:35.483 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:45:35.483 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:45:35.483 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:45:17' 2024-09-12T13:45:35.483 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:45:35.483 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:45:35.483 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-09-12T13:45:35.579 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING641934232mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary102e48042f-ba7f-45c0-9287-a6bae29dd722up+replayingreplaying, {"bytes_per_second":3311411.2,"bytes_per_snapshot":66228224.0,"last_snapshot_bytes":66228224,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726148664,"remote_snapshot_timestamp":1726148664,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:45:17image-primary519fef5ab-79b5-4712-8250-75fffa450f0aup+replayingreplaying, {"bytes_per_second":3340083.2,"bytes_per_snapshot":66801664.0,"last_snapshot_bytes":66801664,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726148664,"remote_snapshot_timestamp":1726148664,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:45:17image-primary685abb8e7-1989-4d4a-a874-9bc560cf4e0dup+replayingreplaying, {"bytes_per_second":3262873.6,"bytes_per_snapshot":65257472.0,"last_snapshot_bytes":65257472,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726148666,"remote_snapshot_timestamp":1726148666,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:45:17image-primary76ca46845-7741-492c-99fb-642d3ab53fabup+replayingreplaying, {"bytes_per_second":3228876.8,"bytes_per_snapshot":64577536.0,"last_snapshot_bytes":64577536,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726148667,"remote_snapshot_timestamp":1726148667,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:45:17image-primary811646481-daec-461e-b7dc-ed4d9a9fd543up+replayingreplaying, {"bytes_per_second":3386982.4,"bytes_per_snapshot":67739648.0,"last_snapshot_bytes":67739648,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726148664,"remote_snapshot_timestamp":1726148664,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:45:17image-primary9b98c3fed-4807-46cf-b9ef-4cbefc529d10up+replayingreplaying, {"bytes_per_second":3216179.2,"bytes_per_snapshot":64323584.0,"last_snapshot_bytes":64323584,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726148661,"remote_snapshot_timestamp":1726148661,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:45:17' 2024-09-12T13:45:35.580 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:45:35.580 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary5'\'']/last_update' 2024-09-12T13:45:35.584 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:45:17' 2024-09-12T13:45:35.585 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary5'\'']/state' 2024-09-12T13:45:35.588 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+replaying 2024-09-12T13:45:35.589 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+replaying 2024-09-12T13:45:35.590 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+replaying 2024-09-12T13:45:35.591 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+replaying 2024-09-12T13:45:35.592 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:45:35.592 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:45:35.592 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ remove_image cluster2 mirror image-primary5 2024-09-12T13:45:35.592 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:45:35.592 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:45:35.592 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:45:35.592 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 snap purge mirror/image-primary5 2024-09-12T13:45:35.661 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 rm mirror/image-primary5 2024-09-12T13:45:35.809 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:45:35.810+0000 7f67e4800640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x56055bf609c0 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-09-12T13:45:35.809 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:45:35.810+0000 7f67eca00640 -1 rbd::mirror::ImageReplayer: 0x5605573caa00 [3/19fef5ab-79b5-4712-8250-75fffa450f0a] operator(): start failed: (85) Interrupted system call should be restarted 2024-09-12T13:45:37.339 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-09-12T13:45:37.345 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_image_removal cluster1 mirror image-primary5 2024-09-12T13:45:37.346 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:45:37.346 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:45:37.346 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:45:37.346 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:45:37.346 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 ls mirror 2024-09-12T13:45:37.346 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep -wq image-primary5 2024-09-12T13:45:37.395 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:45:37.395 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rm -rf test-primary5 2024-09-12T13:45:37.398 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:45:37.398 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ IMG=image-primary6 2024-09-12T13:45:37.398 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_replaying_status_in_pool_dir cluster1 mirror image-primary6 2024-09-12T13:45:37.398 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:45:37.398 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:45:37.398 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:45:37.399 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' snapshot = journal ']' 2024-09-12T13:45:37.399 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary6 up+replaying 2024-09-12T13:45:37.399 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:45:37.399 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:45:37.399 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:45:37.399 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:45:37.399 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:45:37.399 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:45:37.399 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:45:37.399 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:45:38.400 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary6 up+replaying '' '' 2024-09-12T13:45:38.400 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:45:38.400 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:45:38.401 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:45:38.401 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:45:38.401 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:45:38.401 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:45:38.401 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:45:38.401 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:45:38.401 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary6 2024-09-12T13:45:38.484 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary6: 2024-09-12T13:45:38.484 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 85abb8e7-1989-4d4a-a874-9bc560cf4e0d 2024-09-12T13:45:38.484 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+replaying 2024-09-12T13:45:38.484 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: replaying, {"bytes_per_second":3262873.6,"bytes_per_snapshot":65257472.0,"last_snapshot_bytes":65257472,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726148666,"remote_snapshot_timestamp":1726148666,"replay_state":"idle"} 2024-09-12T13:45:38.484 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:45:38.484 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:45:17' 2024-09-12T13:45:38.484 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+replaying' 2024-09-12T13:45:38.486 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+replaying 2024-09-12T13:45:38.487 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:45:38.489 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: replaying, {"bytes_per_second":3262873.6,"bytes_per_snapshot":65257472.0,"last_snapshot_bytes":65257472,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726148666,"remote_snapshot_timestamp":1726148666,"replay_state":"idle"} 2024-09-12T13:45:38.490 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:45:38.490 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+replaying 2024-09-12T13:45:38.491 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:45:38.493 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+replaying 2024-09-12T13:45:38.493 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:45:38.496 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:45:38.496 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:45:38.497 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:45:38.501 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:45:17' 2024-09-12T13:45:38.501 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary6 up+replaying '2024-09-12 13:45:17' 2024-09-12T13:45:38.501 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:45:38.501 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:45:38.501 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:45:38.501 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:45:38.502 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:45:17' 2024-09-12T13:45:38.502 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:45:38.502 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:45:38.502 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-09-12T13:45:38.592 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING541934232mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary102e48042f-ba7f-45c0-9287-a6bae29dd722up+replayingreplaying, {"bytes_per_second":3311411.2,"bytes_per_snapshot":66228224.0,"last_snapshot_bytes":66228224,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726148664,"remote_snapshot_timestamp":1726148664,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:45:17image-primary685abb8e7-1989-4d4a-a874-9bc560cf4e0dup+replayingreplaying, {"bytes_per_second":3262873.6,"bytes_per_snapshot":65257472.0,"last_snapshot_bytes":65257472,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726148666,"remote_snapshot_timestamp":1726148666,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:45:17image-primary76ca46845-7741-492c-99fb-642d3ab53fabup+replayingreplaying, {"bytes_per_second":3228876.8,"bytes_per_snapshot":64577536.0,"last_snapshot_bytes":64577536,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726148667,"remote_snapshot_timestamp":1726148667,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:45:17image-primary811646481-daec-461e-b7dc-ed4d9a9fd543up+replayingreplaying, {"bytes_per_second":3386982.4,"bytes_per_snapshot":67739648.0,"last_snapshot_bytes":67739648,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726148664,"remote_snapshot_timestamp":1726148664,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:45:17image-primary9b98c3fed-4807-46cf-b9ef-4cbefc529d10up+replayingreplaying, {"bytes_per_second":3216179.2,"bytes_per_snapshot":64323584.0,"last_snapshot_bytes":64323584,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726148661,"remote_snapshot_timestamp":1726148661,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:45:17' 2024-09-12T13:45:38.592 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:45:38.593 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary6'\'']/last_update' 2024-09-12T13:45:38.596 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:45:17' 2024-09-12T13:45:38.597 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary6'\'']/state' 2024-09-12T13:45:38.601 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+replaying 2024-09-12T13:45:38.601 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+replaying 2024-09-12T13:45:38.602 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+replaying 2024-09-12T13:45:38.604 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+replaying 2024-09-12T13:45:38.604 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:45:38.605 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:45:38.605 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ remove_image cluster2 mirror image-primary6 2024-09-12T13:45:38.605 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:45:38.605 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:45:38.605 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:45:38.605 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 snap purge mirror/image-primary6 2024-09-12T13:45:38.665 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 rm mirror/image-primary6 2024-09-12T13:45:38.807 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:45:38.808+0000 7f67e4800640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x56055b972680 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-09-12T13:45:38.807 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:45:38.808+0000 7f67ed400640 -1 rbd::mirror::ImageReplayer: 0x56055c198c80 [3/85abb8e7-1989-4d4a-a874-9bc560cf4e0d] operator(): start failed: (85) Interrupted system call should be restarted 2024-09-12T13:45:40.328 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-09-12T13:45:40.334 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_image_removal cluster1 mirror image-primary6 2024-09-12T13:45:40.334 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:45:40.334 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:45:40.334 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:45:40.334 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:45:40.335 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 ls mirror 2024-09-12T13:45:40.335 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep -wq image-primary6 2024-09-12T13:45:40.391 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:45:40.391 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rm -rf test-primary6 2024-09-12T13:45:40.394 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:45:40.394 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ IMG=image-primary7 2024-09-12T13:45:40.394 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_replaying_status_in_pool_dir cluster1 mirror image-primary7 2024-09-12T13:45:40.394 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:45:40.394 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:45:40.394 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:45:40.394 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' snapshot = journal ']' 2024-09-12T13:45:40.394 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary7 up+replaying 2024-09-12T13:45:40.394 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:45:40.395 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:45:40.395 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:45:40.395 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:45:40.395 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:45:40.395 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:45:40.395 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:45:40.395 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:45:41.396 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary7 up+replaying '' '' 2024-09-12T13:45:41.396 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:45:41.396 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:45:41.396 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:45:41.396 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:45:41.396 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:45:41.396 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:45:41.397 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:45:41.397 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:45:41.397 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary7 2024-09-12T13:45:41.473 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary7: 2024-09-12T13:45:41.474 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 6ca46845-7741-492c-99fb-642d3ab53fab 2024-09-12T13:45:41.474 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+replaying 2024-09-12T13:45:41.474 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: replaying, {"bytes_per_second":3228876.8,"bytes_per_snapshot":64577536.0,"last_snapshot_bytes":64577536,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726148667,"remote_snapshot_timestamp":1726148667,"replay_state":"idle"} 2024-09-12T13:45:41.474 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:45:41.474 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:45:17' 2024-09-12T13:45:41.474 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+replaying' 2024-09-12T13:45:41.476 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+replaying 2024-09-12T13:45:41.476 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:45:41.479 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: replaying, {"bytes_per_second":3228876.8,"bytes_per_snapshot":64577536.0,"last_snapshot_bytes":64577536,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726148667,"remote_snapshot_timestamp":1726148667,"replay_state":"idle"} 2024-09-12T13:45:41.479 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:45:41.480 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+replaying 2024-09-12T13:45:41.480 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:45:41.482 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+replaying 2024-09-12T13:45:41.482 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:45:41.485 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:45:41.485 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:45:41.486 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:45:41.490 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:45:17' 2024-09-12T13:45:41.490 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary7 up+replaying '2024-09-12 13:45:17' 2024-09-12T13:45:41.490 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:45:41.490 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:45:41.491 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:45:41.491 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:45:41.491 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:45:17' 2024-09-12T13:45:41.491 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:45:41.491 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:45:41.491 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-09-12T13:45:41.575 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING441934232mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary102e48042f-ba7f-45c0-9287-a6bae29dd722up+replayingreplaying, {"bytes_per_second":3311411.2,"bytes_per_snapshot":66228224.0,"last_snapshot_bytes":66228224,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726148664,"remote_snapshot_timestamp":1726148664,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:45:17image-primary76ca46845-7741-492c-99fb-642d3ab53fabup+replayingreplaying, {"bytes_per_second":3228876.8,"bytes_per_snapshot":64577536.0,"last_snapshot_bytes":64577536,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726148667,"remote_snapshot_timestamp":1726148667,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:45:17image-primary811646481-daec-461e-b7dc-ed4d9a9fd543up+replayingreplaying, {"bytes_per_second":3386982.4,"bytes_per_snapshot":67739648.0,"last_snapshot_bytes":67739648,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726148664,"remote_snapshot_timestamp":1726148664,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:45:17image-primary9b98c3fed-4807-46cf-b9ef-4cbefc529d10up+replayingreplaying, {"bytes_per_second":3216179.2,"bytes_per_snapshot":64323584.0,"last_snapshot_bytes":64323584,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726148661,"remote_snapshot_timestamp":1726148661,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:45:17' 2024-09-12T13:45:41.575 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:45:41.575 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary7'\'']/last_update' 2024-09-12T13:45:41.581 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:45:17' 2024-09-12T13:45:41.581 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary7'\'']/state' 2024-09-12T13:45:41.583 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+replaying 2024-09-12T13:45:41.584 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+replaying 2024-09-12T13:45:41.584 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+replaying 2024-09-12T13:45:41.587 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+replaying 2024-09-12T13:45:41.588 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:45:41.588 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:45:41.588 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ remove_image cluster2 mirror image-primary7 2024-09-12T13:45:41.588 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:45:41.589 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:45:41.589 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:45:41.589 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 snap purge mirror/image-primary7 2024-09-12T13:45:41.647 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 rm mirror/image-primary7 2024-09-12T13:45:41.782 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:45:41.782+0000 7f67e4800640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x56055b7eeea0 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-09-12T13:45:41.782 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:45:41.782+0000 7f67eca00640 -1 rbd::mirror::ImageReplayer: 0x5605561ac000 [3/6ca46845-7741-492c-99fb-642d3ab53fab] operator(): start failed: (85) Interrupted system call should be restarted 2024-09-12T13:45:43.349 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-09-12T13:45:43.356 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_image_removal cluster1 mirror image-primary7 2024-09-12T13:45:43.356 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:45:43.356 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:45:43.356 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:45:43.356 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:45:43.357 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 ls mirror 2024-09-12T13:45:43.357 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep -wq image-primary7 2024-09-12T13:45:43.407 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:45:43.407 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rm -rf test-primary7 2024-09-12T13:45:43.409 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:45:43.409 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ IMG=image-primary8 2024-09-12T13:45:43.409 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_replaying_status_in_pool_dir cluster1 mirror image-primary8 2024-09-12T13:45:43.410 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:45:43.410 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:45:43.410 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:45:43.410 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' snapshot = journal ']' 2024-09-12T13:45:43.410 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary8 up+replaying 2024-09-12T13:45:43.410 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:45:43.410 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:45:43.410 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:45:43.410 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:45:43.410 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:45:43.410 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:45:43.410 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:45:43.411 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:45:44.412 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary8 up+replaying '' '' 2024-09-12T13:45:44.412 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:45:44.412 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:45:44.412 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:45:44.412 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:45:44.412 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:45:44.412 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:45:44.412 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:45:44.413 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:45:44.413 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary8 2024-09-12T13:45:44.489 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary8: 2024-09-12T13:45:44.489 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 11646481-daec-461e-b7dc-ed4d9a9fd543 2024-09-12T13:45:44.489 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+replaying 2024-09-12T13:45:44.489 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: replaying, {"bytes_per_second":3386982.4,"bytes_per_snapshot":67739648.0,"last_snapshot_bytes":67739648,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726148664,"remote_snapshot_timestamp":1726148664,"replay_state":"idle"} 2024-09-12T13:45:44.489 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:45:44.489 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:45:17' 2024-09-12T13:45:44.490 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+replaying' 2024-09-12T13:45:44.492 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+replaying 2024-09-12T13:45:44.492 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:45:44.495 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: replaying, {"bytes_per_second":3386982.4,"bytes_per_snapshot":67739648.0,"last_snapshot_bytes":67739648,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726148664,"remote_snapshot_timestamp":1726148664,"replay_state":"idle"} 2024-09-12T13:45:44.495 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:45:44.495 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+replaying 2024-09-12T13:45:44.496 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:45:44.498 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+replaying 2024-09-12T13:45:44.498 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:45:44.501 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:45:44.501 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:45:44.502 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:45:44.506 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:45:17' 2024-09-12T13:45:44.506 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary8 up+replaying '2024-09-12 13:45:17' 2024-09-12T13:45:44.506 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:45:44.506 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:45:44.506 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:45:44.506 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:45:44.506 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:45:17' 2024-09-12T13:45:44.506 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:45:44.507 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:45:44.507 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-09-12T13:45:44.591 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING341934232mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary102e48042f-ba7f-45c0-9287-a6bae29dd722up+replayingreplaying, {"bytes_per_second":3311411.2,"bytes_per_snapshot":66228224.0,"last_snapshot_bytes":66228224,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726148664,"remote_snapshot_timestamp":1726148664,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:45:17image-primary811646481-daec-461e-b7dc-ed4d9a9fd543up+replayingreplaying, {"bytes_per_second":3386982.4,"bytes_per_snapshot":67739648.0,"last_snapshot_bytes":67739648,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726148664,"remote_snapshot_timestamp":1726148664,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:45:17image-primary9b98c3fed-4807-46cf-b9ef-4cbefc529d10up+replayingreplaying, {"bytes_per_second":3216179.2,"bytes_per_snapshot":64323584.0,"last_snapshot_bytes":64323584,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726148661,"remote_snapshot_timestamp":1726148661,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:45:17' 2024-09-12T13:45:44.591 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:45:44.591 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary8'\'']/last_update' 2024-09-12T13:45:44.595 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:45:17' 2024-09-12T13:45:44.596 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary8'\'']/state' 2024-09-12T13:45:44.599 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+replaying 2024-09-12T13:45:44.600 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+replaying 2024-09-12T13:45:44.600 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+replaying 2024-09-12T13:45:44.602 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+replaying 2024-09-12T13:45:44.603 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:45:44.603 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:45:44.603 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ remove_image cluster2 mirror image-primary8 2024-09-12T13:45:44.603 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:45:44.603 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:45:44.604 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:45:44.604 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 snap purge mirror/image-primary8 2024-09-12T13:45:44.664 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 rm mirror/image-primary8 2024-09-12T13:45:44.806 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:45:44.806+0000 7f67e5200640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x56055c053a00 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-09-12T13:45:44.806 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:45:44.806+0000 7f67eca00640 -1 rbd::mirror::ImageReplayer: 0x5605561ac280 [3/11646481-daec-461e-b7dc-ed4d9a9fd543] operator(): start failed: (85) Interrupted system call should be restarted 2024-09-12T13:45:46.371 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-09-12T13:45:46.377 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_image_removal cluster1 mirror image-primary8 2024-09-12T13:45:46.377 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:45:46.377 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:45:46.377 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:45:46.377 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:45:46.378 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 ls mirror 2024-09-12T13:45:46.378 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep -wq image-primary8 2024-09-12T13:45:46.428 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:45:46.428 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rm -rf test-primary8 2024-09-12T13:45:46.431 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:45:46.431 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ IMG=image-primary9 2024-09-12T13:45:46.431 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_replaying_status_in_pool_dir cluster1 mirror image-primary9 2024-09-12T13:45:46.431 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:45:46.431 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:45:46.431 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:45:46.431 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' snapshot = journal ']' 2024-09-12T13:45:46.431 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary9 up+replaying 2024-09-12T13:45:46.431 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:45:46.431 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:45:46.432 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:45:46.432 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:45:46.432 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:45:46.432 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:45:46.432 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:45:46.432 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:45:47.433 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary9 up+replaying '' '' 2024-09-12T13:45:47.433 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:45:47.433 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:45:47.433 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:45:47.433 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:45:47.433 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:45:47.433 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:45:47.433 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:45:47.434 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:45:47.434 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary9 2024-09-12T13:45:47.512 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary9: 2024-09-12T13:45:47.512 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: b98c3fed-4807-46cf-b9ef-4cbefc529d10 2024-09-12T13:45:47.512 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+replaying 2024-09-12T13:45:47.512 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: replaying, {"bytes_per_second":3216179.2,"bytes_per_snapshot":64323584.0,"last_snapshot_bytes":64323584,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726148661,"remote_snapshot_timestamp":1726148661,"replay_state":"idle"} 2024-09-12T13:45:47.512 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:45:47.512 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:45:17' 2024-09-12T13:45:47.512 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+replaying' 2024-09-12T13:45:47.514 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+replaying 2024-09-12T13:45:47.515 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:45:47.517 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: replaying, {"bytes_per_second":3216179.2,"bytes_per_snapshot":64323584.0,"last_snapshot_bytes":64323584,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726148661,"remote_snapshot_timestamp":1726148661,"replay_state":"idle"} 2024-09-12T13:45:47.517 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:45:47.518 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+replaying 2024-09-12T13:45:47.518 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:45:47.520 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+replaying 2024-09-12T13:45:47.520 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:45:47.523 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:45:47.523 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:45:47.524 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:45:47.528 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:45:17' 2024-09-12T13:45:47.528 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary9 up+replaying '2024-09-12 13:45:17' 2024-09-12T13:45:47.528 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:45:47.528 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:45:47.528 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:45:47.528 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:45:47.528 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:45:17' 2024-09-12T13:45:47.528 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:45:47.529 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:45:47.529 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-09-12T13:45:47.613 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING241934232mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary102e48042f-ba7f-45c0-9287-a6bae29dd722up+replayingreplaying, {"bytes_per_second":3311411.2,"bytes_per_snapshot":66228224.0,"last_snapshot_bytes":66228224,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726148664,"remote_snapshot_timestamp":1726148664,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:45:17image-primary9b98c3fed-4807-46cf-b9ef-4cbefc529d10up+replayingreplaying, {"bytes_per_second":3216179.2,"bytes_per_snapshot":64323584.0,"last_snapshot_bytes":64323584,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726148661,"remote_snapshot_timestamp":1726148661,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:45:17' 2024-09-12T13:45:47.614 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:45:47.614 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary9'\'']/last_update' 2024-09-12T13:45:47.618 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:45:17' 2024-09-12T13:45:47.619 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary9'\'']/state' 2024-09-12T13:45:47.622 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+replaying 2024-09-12T13:45:47.623 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+replaying 2024-09-12T13:45:47.623 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+replaying 2024-09-12T13:45:47.625 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+replaying 2024-09-12T13:45:47.626 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:45:47.626 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:45:47.626 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ remove_image cluster2 mirror image-primary9 2024-09-12T13:45:47.626 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:45:47.626 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:45:47.626 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:45:47.626 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 snap purge mirror/image-primary9 2024-09-12T13:45:47.687 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 rm mirror/image-primary9 2024-09-12T13:45:47.834 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:45:47.834+0000 7f67e5200640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x56055c0524e0 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-09-12T13:45:47.834 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:45:47.834+0000 7f67ed400640 -1 rbd::mirror::ImageReplayer: 0x5605561ad680 [3/b98c3fed-4807-46cf-b9ef-4cbefc529d10] operator(): start failed: (85) Interrupted system call should be restarted 2024-09-12T13:45:47.899 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:45:47.899+0000 7f67e4800640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x56055b7ee680 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-09-12T13:45:47.899 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:45:47.899+0000 7f67ed400640 -1 rbd::mirror::ImageReplayer: 0x5605561ad680 [3/b98c3fed-4807-46cf-b9ef-4cbefc529d10] operator(): start failed: (85) Interrupted system call should be restarted 2024-09-12T13:45:49.362 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-09-12T13:45:49.368 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_image_removal cluster1 mirror image-primary9 2024-09-12T13:45:49.368 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:45:49.368 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:45:49.368 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:45:49.368 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:45:49.369 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 ls mirror 2024-09-12T13:45:49.369 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep -wq image-primary9 2024-09-12T13:45:49.422 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:45:49.422 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rm -rf test-primary9 2024-09-12T13:45:49.424 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:45:49.424 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ IMG=image-primary10 2024-09-12T13:45:49.424 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_replaying_status_in_pool_dir cluster1 mirror image-primary10 2024-09-12T13:45:49.425 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:45:49.425 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:45:49.425 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:45:49.425 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' snapshot = journal ']' 2024-09-12T13:45:49.425 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary10 up+replaying 2024-09-12T13:45:49.425 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:45:49.425 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:45:49.425 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:45:49.425 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:45:49.425 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:45:49.425 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:45:49.425 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:45:49.425 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:45:50.427 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary10 up+replaying '' '' 2024-09-12T13:45:50.427 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:45:50.427 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:45:50.427 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:45:50.427 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:45:50.427 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:45:50.427 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:45:50.427 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:45:50.428 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:45:50.428 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary10 2024-09-12T13:45:50.501 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary10: 2024-09-12T13:45:50.502 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 2e48042f-ba7f-45c0-9287-a6bae29dd722 2024-09-12T13:45:50.502 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+replaying 2024-09-12T13:45:50.502 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: replaying, {"bytes_per_second":0.0,"bytes_per_snapshot":66228224.0,"last_snapshot_bytes":66228224,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726148664,"remote_snapshot_timestamp":1726148664,"replay_state":"idle"} 2024-09-12T13:45:50.502 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:45:50.502 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:45:47' 2024-09-12T13:45:50.502 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+replaying' 2024-09-12T13:45:50.504 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+replaying 2024-09-12T13:45:50.504 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:45:50.507 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: replaying, {"bytes_per_second":0.0,"bytes_per_snapshot":66228224.0,"last_snapshot_bytes":66228224,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726148664,"remote_snapshot_timestamp":1726148664,"replay_state":"idle"} 2024-09-12T13:45:50.507 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:45:50.507 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+replaying 2024-09-12T13:45:50.508 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:45:50.510 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+replaying 2024-09-12T13:45:50.510 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:45:50.513 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:45:50.513 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:45:50.513 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:45:50.517 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:45:47' 2024-09-12T13:45:50.517 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary10 up+replaying '2024-09-12 13:45:47' 2024-09-12T13:45:50.517 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:45:50.517 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:45:50.517 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:45:50.518 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:45:50.518 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:45:47' 2024-09-12T13:45:50.518 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:45:50.518 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:45:50.518 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-09-12T13:45:50.595 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING141934232mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary102e48042f-ba7f-45c0-9287-a6bae29dd722up+replayingreplaying, {"bytes_per_second":0.0,"bytes_per_snapshot":66228224.0,"last_snapshot_bytes":66228224,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726148664,"remote_snapshot_timestamp":1726148664,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:45:47' 2024-09-12T13:45:50.596 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:45:50.596 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary10'\'']/last_update' 2024-09-12T13:45:50.600 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:45:47' 2024-09-12T13:45:50.600 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary10'\'']/state' 2024-09-12T13:45:50.604 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+replaying 2024-09-12T13:45:50.605 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+replaying 2024-09-12T13:45:50.605 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+replaying 2024-09-12T13:45:50.607 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+replaying 2024-09-12T13:45:50.608 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:45:50.608 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:45:50.608 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ remove_image cluster2 mirror image-primary10 2024-09-12T13:45:50.608 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:45:50.608 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:45:50.608 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:45:50.608 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 snap purge mirror/image-primary10 2024-09-12T13:45:50.673 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 rm mirror/image-primary10 2024-09-12T13:45:50.810 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:45:50.810+0000 7f67e4800640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x560556dfc1a0 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-09-12T13:45:50.810 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:45:50.810+0000 7f67eca00640 -1 rbd::mirror::ImageReplayer: 0x560556cef180 [3/2e48042f-ba7f-45c0-9287-a6bae29dd722] operator(): start failed: (85) Interrupted system call should be restarted 2024-09-12T13:45:52.410 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-09-12T13:45:52.416 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_image_removal cluster1 mirror image-primary10 2024-09-12T13:45:52.416 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:45:52.417 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:45:52.417 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:45:52.417 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:45:52.417 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 ls mirror 2024-09-12T13:45:52.417 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep -wq image-primary10 2024-09-12T13:45:52.466 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:45:52.466 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rm -rf test-primary10 2024-09-12T13:45:52.469 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..10} 2024-09-12T13:45:52.469 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEVS=() 2024-09-12T13:45:52.469 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ SNAP_PIDS=() 2024-09-12T13:45:52.469 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ COMPARE_PIDS=() 2024-09-12T13:45:52.469 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ WORKLOAD_PIDS=() 2024-09-12T13:45:52.469 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ RET=0 2024-09-12T13:45:52.470 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:45:52.470 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ IMG=image-primary1 2024-09-12T13:45:52.470 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ MNTPT=test-primary1 2024-09-12T13:45:52.470 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ create_image_and_enable_mirror cluster1 mirror image-primary1 snapshot 10G 2024-09-12T13:45:52.470 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:45:52.470 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:45:52.470 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:45:52.470 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:45:52.470 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:45:52.470 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:45:52.470 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mode=snapshot 2024-09-12T13:45:52.470 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n snapshot ']' 2024-09-12T13:45:52.470 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:45:52.471 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ create_image cluster1 mirror image-primary1 10G 2024-09-12T13:45:52.471 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:45:52.471 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:45:52.471 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:45:52.472 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:45:52.472 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:45:52.472 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:45:52.472 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local size=128 2024-09-12T13:45:52.472 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n 10G ']' 2024-09-12T13:45:52.472 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ size=10G 2024-09-12T13:45:52.472 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:45:52.472 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 create --size 10G --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image-primary1 2024-09-12T13:45:52.541 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ is_pool_mirror_mode_image mirror 2024-09-12T13:45:52.542 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:45:52.542 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' image = image ']' 2024-09-12T13:45:52.542 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:45:52.542 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ enable_mirror cluster1 mirror image-primary1 snapshot 2024-09-12T13:45:52.542 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:45:52.542 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:45:52.542 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:45:52.542 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mode=snapshot 2024-09-12T13:45:52.542 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 mirror image enable mirror/image-primary1 snapshot 2024-09-12T13:45:53.245 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Mirroring enabled 2024-09-12T13:45:53.260 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 info mirror/image-primary1 2024-09-12T13:45:53.316 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:rbd image 'image-primary1': 2024-09-12T13:45:53.316 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: size 10 GiB in 2560 objects 2024-09-12T13:45:53.316 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: order 22 (4 MiB objects) 2024-09-12T13:45:53.316 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: snapshot_count: 1 2024-09-12T13:45:53.316 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: id: 2e425b60265a 2024-09-12T13:45:53.317 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: block_name_prefix: rbd_data.2e425b60265a 2024-09-12T13:45:53.317 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: format: 2 2024-09-12T13:45:53.317 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: features: layering, exclusive-lock, object-map, fast-diff 2024-09-12T13:45:53.317 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: op_features: 2024-09-12T13:45:53.317 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: flags: 2024-09-12T13:45:53.317 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: create_timestamp: Thu Sep 12 13:45:52 2024 2024-09-12T13:45:53.317 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: access_timestamp: Thu Sep 12 13:45:52 2024 2024-09-12T13:45:53.317 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: modify_timestamp: Thu Sep 12 13:45:52 2024 2024-09-12T13:45:53.317 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring state: enabled 2024-09-12T13:45:53.317 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring mode: snapshot 2024-09-12T13:45:53.317 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring global id: f6ac15bd-c3ff-4c34-86f6-e9bbc701f22f 2024-09-12T13:45:53.317 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring primary: true 2024-09-12T13:45:53.321 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \n\b\d ]] 2024-09-12T13:45:53.321 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \k\r\b\d ]] 2024-09-12T13:45:53.322 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo rbd --cluster cluster1 device map -t krbd mirror/image-primary1 2024-09-12T13:45:53.460 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEV=/dev/rbd0 2024-09-12T13:45:53.460 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEVS+=($DEV) 2024-09-12T13:45:53.460 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo mkfs.ext4 /dev/rbd0 2024-09-12T13:45:53.564 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:mke2fs 1.46.5 (30-Dec-2021) 2024-09-12T13:45:53.576 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Discarding device blocks: 0/2621440 done 2024-09-12T13:45:53.576 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Creating filesystem with 2621440 4k blocks and 655360 inodes 2024-09-12T13:45:53.576 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Filesystem UUID: 3cc76b74-3d2b-4ab4-b531-3e18050511b8 2024-09-12T13:45:53.576 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Superblock backups stored on blocks: 2024-09-12T13:45:53.576 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 2024-09-12T13:45:53.576 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 2024-09-12T13:45:53.576 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Allocating group tables: 0/80 done 2024-09-12T13:45:53.576 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Writing inode tables: 0/80 done 2024-09-12T13:45:53.581 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Creating journal (16384 blocks): done 2024-09-12T13:45:53.645 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Writing superblocks and filesystem accounting information: 0/80 done 2024-09-12T13:45:53.645 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 2024-09-12T13:45:53.645 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mkdir test-primary1 2024-09-12T13:45:53.681 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo mount /dev/rbd0 test-primary1 2024-09-12T13:45:53.737 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ whoami 2024-09-12T13:45:53.750 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo chown ubuntu test-primary1 2024-09-12T13:45:53.781 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ SNAP_PIDS+=($!) 2024-09-12T13:45:53.782 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ take_mirror_snapshots cluster1 mirror image-primary1 2024-09-12T13:45:53.782 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:45:53.782 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:45:53.782 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:45:53.782 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:45:53.782 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:45:53.782 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:45:53.782 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:45:53.782 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:45:53.782 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:45:53.783 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ WORKLOAD_PIDS+=($!) 2024-09-12T13:45:53.783 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:45:53.783 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ IMG=image-primary2 2024-09-12T13:45:53.783 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ MNTPT=test-primary2 2024-09-12T13:45:53.783 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ create_image_and_enable_mirror cluster1 mirror image-primary2 snapshot 10G 2024-09-12T13:45:53.783 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:45:53.783 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:45:53.783 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:45:53.783 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:45:53.783 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:45:53.783 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:45:53.783 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mode=snapshot 2024-09-12T13:45:53.784 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n snapshot ']' 2024-09-12T13:45:53.784 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:45:53.784 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ create_image cluster1 mirror image-primary2 10G 2024-09-12T13:45:53.784 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:45:53.784 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:45:53.784 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:45:53.784 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:45:53.784 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:45:53.784 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:45:53.784 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local size=128 2024-09-12T13:45:53.784 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n 10G ']' 2024-09-12T13:45:53.784 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ size=10G 2024-09-12T13:45:53.784 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:45:53.785 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 create --size 10G --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image-primary2 2024-09-12T13:45:53.785 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ slow_untar_workload test-primary1 2024-09-12T13:45:53.785 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mountpt=test-primary1 2024-09-12T13:45:53.785 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ cp linux-5.4.tar.gz test-primary1 2024-09-12T13:45:53.856 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ is_pool_mirror_mode_image mirror 2024-09-12T13:45:53.856 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:45:53.856 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' image = image ']' 2024-09-12T13:45:53.856 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:45:53.856 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ enable_mirror cluster1 mirror image-primary2 snapshot 2024-09-12T13:45:53.857 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:45:53.857 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:45:53.857 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:45:53.857 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mode=snapshot 2024-09-12T13:45:53.857 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 mirror image enable mirror/image-primary2 snapshot 2024-09-12T13:45:54.071 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local ret=0 2024-09-12T13:45:54.072 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ timeout 5m bash -c 'zcat test-primary1/linux-5.4.tar.gz | pv -L 256K | tar xf - -C test-primary1' 2024-09-12T13:45:54.237 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Mirroring enabled 2024-09-12T13:45:54.243 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1984 2024-09-12T13:45:54.252 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:45:54.252 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 info mirror/image-primary2 2024-09-12T13:45:54.309 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:rbd image 'image-primary2': 2024-09-12T13:45:54.309 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: size 10 GiB in 2560 objects 2024-09-12T13:45:54.309 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: order 22 (4 MiB objects) 2024-09-12T13:45:54.310 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: snapshot_count: 1 2024-09-12T13:45:54.310 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: id: 2e52fafdf04f 2024-09-12T13:45:54.310 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: block_name_prefix: rbd_data.2e52fafdf04f 2024-09-12T13:45:54.310 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: format: 2 2024-09-12T13:45:54.310 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: features: layering, exclusive-lock, object-map, fast-diff 2024-09-12T13:45:54.310 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: op_features: 2024-09-12T13:45:54.310 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: flags: 2024-09-12T13:45:54.310 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: create_timestamp: Thu Sep 12 13:45:53 2024 2024-09-12T13:45:54.310 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: access_timestamp: Thu Sep 12 13:45:53 2024 2024-09-12T13:45:54.310 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: modify_timestamp: Thu Sep 12 13:45:53 2024 2024-09-12T13:45:54.310 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring state: enabled 2024-09-12T13:45:54.310 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring mode: snapshot 2024-09-12T13:45:54.310 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring global id: 8399b5a6-24d7-4c4c-9667-28e949d25fb3 2024-09-12T13:45:54.311 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring primary: true 2024-09-12T13:45:54.314 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \n\b\d ]] 2024-09-12T13:45:54.315 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \k\r\b\d ]] 2024-09-12T13:45:54.315 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo rbd --cluster cluster1 device map -t krbd mirror/image-primary2 2024-09-12T13:45:54.435 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEV=/dev/rbd1 2024-09-12T13:45:54.435 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEVS+=($DEV) 2024-09-12T13:45:54.435 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo mkfs.ext4 /dev/rbd1 2024-09-12T13:45:54.455 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:mke2fs 1.46.5 (30-Dec-2021) 2024-09-12T13:45:54.466 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Discarding device blocks: 0/2621440 done 2024-09-12T13:45:54.466 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Creating filesystem with 2621440 4k blocks and 655360 inodes 2024-09-12T13:45:54.466 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Filesystem UUID: 3357a0b7-2ba8-4d0b-9c3a-4d94b8a7e160 2024-09-12T13:45:54.466 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Superblock backups stored on blocks: 2024-09-12T13:45:54.466 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 2024-09-12T13:45:54.466 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 2024-09-12T13:45:54.466 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Allocating group tables: 0/80 done 2024-09-12T13:45:54.466 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Writing inode tables: 0/80 done 2024-09-12T13:45:54.471 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Creating journal (16384 blocks): done 2024-09-12T13:45:54.529 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Writing superblocks and filesystem accounting information: 0/80 done 2024-09-12T13:45:54.529 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 2024-09-12T13:45:54.532 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mkdir test-primary2 2024-09-12T13:45:54.535 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo mount /dev/rbd1 test-primary2 2024-09-12T13:45:54.592 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ whoami 2024-09-12T13:45:54.593 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo chown ubuntu test-primary2 2024-09-12T13:45:54.618 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ SNAP_PIDS+=($!) 2024-09-12T13:45:54.618 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ take_mirror_snapshots cluster1 mirror image-primary2 2024-09-12T13:45:54.618 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:45:54.618 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:45:54.618 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:45:54.618 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:45:54.618 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:45:54.618 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:45:54.618 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ WORKLOAD_PIDS+=($!) 2024-09-12T13:45:54.618 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:45:54.619 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:45:54.619 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:45:54.619 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:45:54.619 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ IMG=image-primary3 2024-09-12T13:45:54.619 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ MNTPT=test-primary3 2024-09-12T13:45:54.619 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ create_image_and_enable_mirror cluster1 mirror image-primary3 snapshot 10G 2024-09-12T13:45:54.619 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:45:54.619 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:45:54.619 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:45:54.619 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:45:54.619 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:45:54.619 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:45:54.620 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mode=snapshot 2024-09-12T13:45:54.620 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n snapshot ']' 2024-09-12T13:45:54.620 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:45:54.620 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ create_image cluster1 mirror image-primary3 10G 2024-09-12T13:45:54.620 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:45:54.620 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:45:54.620 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:45:54.620 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:45:54.620 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:45:54.620 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:45:54.620 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ slow_untar_workload test-primary2 2024-09-12T13:45:54.620 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local size=128 2024-09-12T13:45:54.620 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n 10G ']' 2024-09-12T13:45:54.621 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ size=10G 2024-09-12T13:45:54.621 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:45:54.621 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 create --size 10G --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image-primary3 2024-09-12T13:45:54.621 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mountpt=test-primary2 2024-09-12T13:45:54.621 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ cp linux-5.4.tar.gz test-primary2 2024-09-12T13:45:54.690 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ is_pool_mirror_mode_image mirror 2024-09-12T13:45:54.690 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:45:54.690 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' image = image ']' 2024-09-12T13:45:54.690 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:45:54.690 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ enable_mirror cluster1 mirror image-primary3 snapshot 2024-09-12T13:45:54.690 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:45:54.690 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:45:54.690 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:45:54.690 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mode=snapshot 2024-09-12T13:45:54.691 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 mirror image enable mirror/image-primary3 snapshot 2024-09-12T13:45:54.905 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local ret=0 2024-09-12T13:45:54.906 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ timeout 5m bash -c 'zcat test-primary2/linux-5.4.tar.gz | pv -L 256K | tar xf - -C test-primary2' 2024-09-12T13:45:55.240 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1986 2024-09-12T13:45:55.244 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Mirroring enabled 2024-09-12T13:45:55.252 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:45:55.259 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 info mirror/image-primary3 2024-09-12T13:45:55.327 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:rbd image 'image-primary3': 2024-09-12T13:45:55.327 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: size 10 GiB in 2560 objects 2024-09-12T13:45:55.327 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: order 22 (4 MiB objects) 2024-09-12T13:45:55.327 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: snapshot_count: 1 2024-09-12T13:45:55.327 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: id: 2e61c7e0a73e 2024-09-12T13:45:55.327 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: block_name_prefix: rbd_data.2e61c7e0a73e 2024-09-12T13:45:55.327 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: format: 2 2024-09-12T13:45:55.327 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: features: layering, exclusive-lock, object-map, fast-diff 2024-09-12T13:45:55.327 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: op_features: 2024-09-12T13:45:55.327 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: flags: 2024-09-12T13:45:55.328 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: create_timestamp: Thu Sep 12 13:45:54 2024 2024-09-12T13:45:55.328 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: access_timestamp: Thu Sep 12 13:45:54 2024 2024-09-12T13:45:55.328 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: modify_timestamp: Thu Sep 12 13:45:54 2024 2024-09-12T13:45:55.328 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring state: enabled 2024-09-12T13:45:55.328 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring mode: snapshot 2024-09-12T13:45:55.328 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring global id: 73afcdb2-236d-496e-9163-3c6ff6e95e23 2024-09-12T13:45:55.328 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring primary: true 2024-09-12T13:45:55.332 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \n\b\d ]] 2024-09-12T13:45:55.332 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \k\r\b\d ]] 2024-09-12T13:45:55.333 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo rbd --cluster cluster1 device map -t krbd mirror/image-primary3 2024-09-12T13:45:55.449 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEV=/dev/rbd2 2024-09-12T13:45:55.449 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEVS+=($DEV) 2024-09-12T13:45:55.449 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo mkfs.ext4 /dev/rbd2 2024-09-12T13:45:55.470 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:mke2fs 1.46.5 (30-Dec-2021) 2024-09-12T13:45:55.485 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Discarding device blocks: 0/2621440 done 2024-09-12T13:45:55.485 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Creating filesystem with 2621440 4k blocks and 655360 inodes 2024-09-12T13:45:55.485 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Filesystem UUID: fbccdcc0-dd6c-4c62-b23f-456df2bfe30f 2024-09-12T13:45:55.485 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Superblock backups stored on blocks: 2024-09-12T13:45:55.485 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 2024-09-12T13:45:55.485 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 2024-09-12T13:45:55.485 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Allocating group tables: 0/80 done 2024-09-12T13:45:55.485 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Writing inode tables: 0/80 done 2024-09-12T13:45:55.488 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Creating journal (16384 blocks): done 2024-09-12T13:45:55.547 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Writing superblocks and filesystem accounting information: 0/80 done 2024-09-12T13:45:55.547 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 2024-09-12T13:45:55.550 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mkdir test-primary3 2024-09-12T13:45:55.552 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo mount /dev/rbd2 test-primary3 2024-09-12T13:45:55.610 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ whoami 2024-09-12T13:45:55.612 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo chown ubuntu test-primary3 2024-09-12T13:45:55.636 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ SNAP_PIDS+=($!) 2024-09-12T13:45:55.636 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ take_mirror_snapshots cluster1 mirror image-primary3 2024-09-12T13:45:55.637 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:45:55.637 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:45:55.637 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:45:55.637 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:45:55.637 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:45:55.637 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:45:55.637 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:45:55.637 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:45:55.637 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:45:55.637 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ WORKLOAD_PIDS+=($!) 2024-09-12T13:45:55.637 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:45:55.637 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ IMG=image-primary4 2024-09-12T13:45:55.637 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ MNTPT=test-primary4 2024-09-12T13:45:55.638 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ create_image_and_enable_mirror cluster1 mirror image-primary4 snapshot 10G 2024-09-12T13:45:55.638 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ slow_untar_workload test-primary3 2024-09-12T13:45:55.638 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:45:55.638 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:45:55.638 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:45:55.638 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:45:55.638 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:45:55.638 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:45:55.638 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mode=snapshot 2024-09-12T13:45:55.638 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n snapshot ']' 2024-09-12T13:45:55.638 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:45:55.638 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ create_image cluster1 mirror image-primary4 10G 2024-09-12T13:45:55.638 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:45:55.639 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:45:55.639 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:45:55.639 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mountpt=test-primary3 2024-09-12T13:45:55.639 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:45:55.639 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:45:55.639 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:45:55.639 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local size=128 2024-09-12T13:45:55.639 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n 10G ']' 2024-09-12T13:45:55.639 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ cp linux-5.4.tar.gz test-primary3 2024-09-12T13:45:55.639 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ size=10G 2024-09-12T13:45:55.639 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:45:55.639 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 create --size 10G --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image-primary4 2024-09-12T13:45:55.707 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ is_pool_mirror_mode_image mirror 2024-09-12T13:45:55.707 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:45:55.707 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' image = image ']' 2024-09-12T13:45:55.707 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:45:55.707 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ enable_mirror cluster1 mirror image-primary4 snapshot 2024-09-12T13:45:55.708 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:45:55.708 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:45:55.708 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:45:55.708 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mode=snapshot 2024-09-12T13:45:55.708 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 mirror image enable mirror/image-primary4 snapshot 2024-09-12T13:45:55.924 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local ret=0 2024-09-12T13:45:55.924 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ timeout 5m bash -c 'zcat test-primary3/linux-5.4.tar.gz | pv -L 256K | tar xf - -C test-primary3' 2024-09-12T13:45:56.253 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Mirroring enabled 2024-09-12T13:45:56.255 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1988 2024-09-12T13:45:56.265 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:45:56.269 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 info mirror/image-primary4 2024-09-12T13:45:56.328 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:rbd image 'image-primary4': 2024-09-12T13:45:56.328 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: size 10 GiB in 2560 objects 2024-09-12T13:45:56.328 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: order 22 (4 MiB objects) 2024-09-12T13:45:56.328 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: snapshot_count: 1 2024-09-12T13:45:56.328 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: id: 2e70e3ee5f33 2024-09-12T13:45:56.328 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: block_name_prefix: rbd_data.2e70e3ee5f33 2024-09-12T13:45:56.328 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: format: 2 2024-09-12T13:45:56.328 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: features: layering, exclusive-lock, object-map, fast-diff 2024-09-12T13:45:56.329 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: op_features: 2024-09-12T13:45:56.329 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: flags: 2024-09-12T13:45:56.329 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: create_timestamp: Thu Sep 12 13:45:55 2024 2024-09-12T13:45:56.329 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: access_timestamp: Thu Sep 12 13:45:55 2024 2024-09-12T13:45:56.329 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: modify_timestamp: Thu Sep 12 13:45:55 2024 2024-09-12T13:45:56.329 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring state: enabled 2024-09-12T13:45:56.329 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring mode: snapshot 2024-09-12T13:45:56.329 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring global id: 6025c1db-ec31-4451-96a8-922baeed4ef6 2024-09-12T13:45:56.329 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring primary: true 2024-09-12T13:45:56.333 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \n\b\d ]] 2024-09-12T13:45:56.333 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \k\r\b\d ]] 2024-09-12T13:45:56.334 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo rbd --cluster cluster1 device map -t krbd mirror/image-primary4 2024-09-12T13:45:56.454 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEV=/dev/rbd3 2024-09-12T13:45:56.455 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEVS+=($DEV) 2024-09-12T13:45:56.455 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo mkfs.ext4 /dev/rbd3 2024-09-12T13:45:56.475 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:mke2fs 1.46.5 (30-Dec-2021) 2024-09-12T13:45:56.487 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Discarding device blocks: 0/2621440 done 2024-09-12T13:45:56.487 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Creating filesystem with 2621440 4k blocks and 655360 inodes 2024-09-12T13:45:56.487 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Filesystem UUID: 77fb7365-6966-44b2-9b67-dd3bc1ce96d4 2024-09-12T13:45:56.488 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Superblock backups stored on blocks: 2024-09-12T13:45:56.488 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 2024-09-12T13:45:56.488 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 2024-09-12T13:45:56.488 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Allocating group tables: 0/80 done 2024-09-12T13:45:56.488 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Writing inode tables: 0/80 done 2024-09-12T13:45:56.492 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Creating journal (16384 blocks): done 2024-09-12T13:45:56.550 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Writing superblocks and filesystem accounting information: 0/80 done 2024-09-12T13:45:56.550 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 2024-09-12T13:45:56.552 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mkdir test-primary4 2024-09-12T13:45:56.555 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo mount /dev/rbd3 test-primary4 2024-09-12T13:45:56.614 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ whoami 2024-09-12T13:45:56.615 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo chown ubuntu test-primary4 2024-09-12T13:45:56.641 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ SNAP_PIDS+=($!) 2024-09-12T13:45:56.642 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ take_mirror_snapshots cluster1 mirror image-primary4 2024-09-12T13:45:56.642 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:45:56.642 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:45:56.642 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:45:56.642 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:45:56.642 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:45:56.642 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:45:56.642 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:45:56.642 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:45:56.642 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ WORKLOAD_PIDS+=($!) 2024-09-12T13:45:56.642 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:45:56.642 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:45:56.642 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ IMG=image-primary5 2024-09-12T13:45:56.643 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ MNTPT=test-primary5 2024-09-12T13:45:56.643 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ slow_untar_workload test-primary4 2024-09-12T13:45:56.643 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ create_image_and_enable_mirror cluster1 mirror image-primary5 snapshot 10G 2024-09-12T13:45:56.643 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:45:56.643 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:45:56.643 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:45:56.643 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:45:56.643 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:45:56.643 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:45:56.643 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mode=snapshot 2024-09-12T13:45:56.643 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n snapshot ']' 2024-09-12T13:45:56.643 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:45:56.643 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mountpt=test-primary4 2024-09-12T13:45:56.644 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ create_image cluster1 mirror image-primary5 10G 2024-09-12T13:45:56.644 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:45:56.644 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ cp linux-5.4.tar.gz test-primary4 2024-09-12T13:45:56.644 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:45:56.644 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:45:56.644 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:45:56.644 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:45:56.644 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:45:56.644 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local size=128 2024-09-12T13:45:56.644 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n 10G ']' 2024-09-12T13:45:56.644 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ size=10G 2024-09-12T13:45:56.644 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:45:56.644 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 create --size 10G --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image-primary5 2024-09-12T13:45:56.713 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ is_pool_mirror_mode_image mirror 2024-09-12T13:45:56.713 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:45:56.713 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' image = image ']' 2024-09-12T13:45:56.713 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:45:56.713 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ enable_mirror cluster1 mirror image-primary5 snapshot 2024-09-12T13:45:56.714 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:45:56.714 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:45:56.714 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:45:56.714 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mode=snapshot 2024-09-12T13:45:56.714 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 mirror image enable mirror/image-primary5 snapshot 2024-09-12T13:45:56.931 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local ret=0 2024-09-12T13:45:56.932 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ timeout 5m bash -c 'zcat test-primary4/linux-5.4.tar.gz | pv -L 256K | tar xf - -C test-primary4' 2024-09-12T13:45:57.254 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:45:57.255 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:45:57.255 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:45:57.255 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:45:57.255 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:45:57.255 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:45:57.277 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Mirroring enabled 2024-09-12T13:45:57.290 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1990 2024-09-12T13:45:57.318 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:45:57.318 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 info mirror/image-primary5 2024-09-12T13:45:57.403 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:rbd image 'image-primary5': 2024-09-12T13:45:57.403 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: size 10 GiB in 2560 objects 2024-09-12T13:45:57.403 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: order 22 (4 MiB objects) 2024-09-12T13:45:57.403 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: snapshot_count: 1 2024-09-12T13:45:57.403 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: id: 2e7f8be5aed1 2024-09-12T13:45:57.403 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: block_name_prefix: rbd_data.2e7f8be5aed1 2024-09-12T13:45:57.403 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: format: 2 2024-09-12T13:45:57.403 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: features: layering, exclusive-lock, object-map, fast-diff 2024-09-12T13:45:57.403 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: op_features: 2024-09-12T13:45:57.404 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: flags: 2024-09-12T13:45:57.404 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: create_timestamp: Thu Sep 12 13:45:56 2024 2024-09-12T13:45:57.404 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: access_timestamp: Thu Sep 12 13:45:56 2024 2024-09-12T13:45:57.404 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: modify_timestamp: Thu Sep 12 13:45:56 2024 2024-09-12T13:45:57.404 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring state: enabled 2024-09-12T13:45:57.404 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring mode: snapshot 2024-09-12T13:45:57.404 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring global id: 6de1d859-a420-4abe-953c-031047e292ec 2024-09-12T13:45:57.404 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring primary: true 2024-09-12T13:45:57.408 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \n\b\d ]] 2024-09-12T13:45:57.408 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \k\r\b\d ]] 2024-09-12T13:45:57.409 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo rbd --cluster cluster1 device map -t krbd mirror/image-primary5 2024-09-12T13:45:57.564 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEV=/dev/rbd4 2024-09-12T13:45:57.564 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEVS+=($DEV) 2024-09-12T13:45:57.564 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo mkfs.ext4 /dev/rbd4 2024-09-12T13:45:57.584 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:mke2fs 1.46.5 (30-Dec-2021) 2024-09-12T13:45:57.599 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Discarding device blocks: 0/2621440 done 2024-09-12T13:45:57.599 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Creating filesystem with 2621440 4k blocks and 655360 inodes 2024-09-12T13:45:57.600 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Filesystem UUID: 865eeb80-8744-4d40-a1a2-db2d8cd59214 2024-09-12T13:45:57.600 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Superblock backups stored on blocks: 2024-09-12T13:45:57.600 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 2024-09-12T13:45:57.600 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 2024-09-12T13:45:57.600 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Allocating group tables: 0/80 done 2024-09-12T13:45:57.600 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Writing inode tables: 0/80 done 2024-09-12T13:45:57.605 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Creating journal (16384 blocks): done 2024-09-12T13:45:57.710 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Writing superblocks and filesystem accounting information: 0/80 done 2024-09-12T13:45:57.710 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 2024-09-12T13:45:57.713 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mkdir test-primary5 2024-09-12T13:45:57.715 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo mount /dev/rbd4 test-primary5 2024-09-12T13:45:57.803 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ whoami 2024-09-12T13:45:57.805 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo chown ubuntu test-primary5 2024-09-12T13:45:57.829 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ SNAP_PIDS+=($!) 2024-09-12T13:45:57.829 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ take_mirror_snapshots cluster1 mirror image-primary5 2024-09-12T13:45:57.830 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:45:57.830 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:45:57.830 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:45:57.830 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ WORKLOAD_PIDS+=($!) 2024-09-12T13:45:57.830 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:45:57.830 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:45:57.830 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:45:57.830 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ IMG=image-primary6 2024-09-12T13:45:57.830 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ MNTPT=test-primary6 2024-09-12T13:45:57.830 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:45:57.830 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ create_image_and_enable_mirror cluster1 mirror image-primary6 snapshot 10G 2024-09-12T13:45:57.831 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:45:57.831 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:45:57.831 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ slow_untar_workload test-primary5 2024-09-12T13:45:57.831 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:45:57.831 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:45:57.831 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:45:57.831 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:45:57.831 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:45:57.831 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:45:57.831 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:45:57.831 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mode=snapshot 2024-09-12T13:45:57.831 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n snapshot ']' 2024-09-12T13:45:57.832 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:45:57.832 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ create_image cluster1 mirror image-primary6 10G 2024-09-12T13:45:57.832 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:45:57.832 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:45:57.832 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:45:57.832 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:45:57.832 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:45:57.832 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:45:57.832 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local size=128 2024-09-12T13:45:57.832 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n 10G ']' 2024-09-12T13:45:57.833 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mountpt=test-primary5 2024-09-12T13:45:57.833 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ size=10G 2024-09-12T13:45:57.833 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:45:57.833 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 create --size 10G --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image-primary6 2024-09-12T13:45:57.833 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ cp linux-5.4.tar.gz test-primary5 2024-09-12T13:45:57.934 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ is_pool_mirror_mode_image mirror 2024-09-12T13:45:57.934 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:45:57.934 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' image = image ']' 2024-09-12T13:45:57.934 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:45:57.935 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ enable_mirror cluster1 mirror image-primary6 snapshot 2024-09-12T13:45:57.935 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:45:57.935 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:45:57.935 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:45:57.935 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mode=snapshot 2024-09-12T13:45:57.935 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 mirror image enable mirror/image-primary6 snapshot 2024-09-12T13:45:58.134 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local ret=0 2024-09-12T13:45:58.134 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ timeout 5m bash -c 'zcat test-primary5/linux-5.4.tar.gz | pv -L 256K | tar xf - -C test-primary5' 2024-09-12T13:45:58.256 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:45:58.256 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:45:58.257 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:45:58.257 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:45:58.257 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:45:58.257 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:45:58.307 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1992 2024-09-12T13:45:58.313 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Mirroring enabled 2024-09-12T13:45:58.317 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1993 2024-09-12T13:45:58.342 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:45:58.351 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:45:58.360 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 info mirror/image-primary6 2024-09-12T13:45:58.457 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:rbd image 'image-primary6': 2024-09-12T13:45:58.457 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: size 10 GiB in 2560 objects 2024-09-12T13:45:58.457 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: order 22 (4 MiB objects) 2024-09-12T13:45:58.458 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: snapshot_count: 1 2024-09-12T13:45:58.458 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: id: 2e91a2fca030 2024-09-12T13:45:58.458 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: block_name_prefix: rbd_data.2e91a2fca030 2024-09-12T13:45:58.458 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: format: 2 2024-09-12T13:45:58.458 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: features: layering, exclusive-lock, object-map, fast-diff 2024-09-12T13:45:58.458 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: op_features: 2024-09-12T13:45:58.458 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: flags: 2024-09-12T13:45:58.458 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: create_timestamp: Thu Sep 12 13:45:57 2024 2024-09-12T13:45:58.458 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: access_timestamp: Thu Sep 12 13:45:57 2024 2024-09-12T13:45:58.458 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: modify_timestamp: Thu Sep 12 13:45:57 2024 2024-09-12T13:45:58.458 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring state: enabled 2024-09-12T13:45:58.458 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring mode: snapshot 2024-09-12T13:45:58.458 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring global id: 88271fa2-1e10-4868-befd-15807c269565 2024-09-12T13:45:58.459 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring primary: true 2024-09-12T13:45:58.462 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \n\b\d ]] 2024-09-12T13:45:58.463 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \k\r\b\d ]] 2024-09-12T13:45:58.463 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo rbd --cluster cluster1 device map -t krbd mirror/image-primary6 2024-09-12T13:45:58.634 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEV=/dev/rbd5 2024-09-12T13:45:58.634 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEVS+=($DEV) 2024-09-12T13:45:58.634 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo mkfs.ext4 /dev/rbd5 2024-09-12T13:45:58.654 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:mke2fs 1.46.5 (30-Dec-2021) 2024-09-12T13:45:58.667 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Discarding device blocks: 0/2621440 done 2024-09-12T13:45:58.667 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Creating filesystem with 2621440 4k blocks and 655360 inodes 2024-09-12T13:45:58.668 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Filesystem UUID: 9cecab6d-186c-4d46-b469-ba281134d423 2024-09-12T13:45:58.668 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Superblock backups stored on blocks: 2024-09-12T13:45:58.668 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 2024-09-12T13:45:58.668 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 2024-09-12T13:45:58.668 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Allocating group tables: 0/80 done 2024-09-12T13:45:58.668 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Writing inode tables: 0/80 done 2024-09-12T13:45:58.673 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Creating journal (16384 blocks): done 2024-09-12T13:45:58.784 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Writing superblocks and filesystem accounting information: 0/80 done 2024-09-12T13:45:58.784 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 2024-09-12T13:45:58.787 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mkdir test-primary6 2024-09-12T13:45:58.789 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo mount /dev/rbd5 test-primary6 2024-09-12T13:45:58.965 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ whoami 2024-09-12T13:45:58.967 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo chown ubuntu test-primary6 2024-09-12T13:45:58.992 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ SNAP_PIDS+=($!) 2024-09-12T13:45:58.993 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ take_mirror_snapshots cluster1 mirror image-primary6 2024-09-12T13:45:58.993 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:45:58.993 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:45:58.993 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:45:58.993 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:45:58.993 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:45:58.993 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:45:58.993 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:45:58.993 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:45:58.993 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:45:58.993 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ WORKLOAD_PIDS+=($!) 2024-09-12T13:45:58.993 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:45:58.993 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ IMG=image-primary7 2024-09-12T13:45:58.994 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ slow_untar_workload test-primary6 2024-09-12T13:45:58.994 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ MNTPT=test-primary7 2024-09-12T13:45:58.994 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ create_image_and_enable_mirror cluster1 mirror image-primary7 snapshot 10G 2024-09-12T13:45:58.994 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:45:58.994 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:45:58.994 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:45:58.994 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:45:58.994 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:45:58.994 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:45:58.994 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mode=snapshot 2024-09-12T13:45:58.994 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mountpt=test-primary6 2024-09-12T13:45:58.994 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n snapshot ']' 2024-09-12T13:45:58.994 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:45:58.994 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ create_image cluster1 mirror image-primary7 10G 2024-09-12T13:45:58.995 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ cp linux-5.4.tar.gz test-primary6 2024-09-12T13:45:58.995 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:45:58.995 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:45:58.995 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:45:58.995 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:45:58.995 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:45:58.995 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:45:58.995 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local size=128 2024-09-12T13:45:58.995 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n 10G ']' 2024-09-12T13:45:58.995 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ size=10G 2024-09-12T13:45:58.995 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:45:58.995 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 create --size 10G --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image-primary7 2024-09-12T13:45:59.087 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ is_pool_mirror_mode_image mirror 2024-09-12T13:45:59.088 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:45:59.088 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' image = image ']' 2024-09-12T13:45:59.088 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:45:59.088 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ enable_mirror cluster1 mirror image-primary7 snapshot 2024-09-12T13:45:59.088 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:45:59.088 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:45:59.088 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:45:59.088 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mode=snapshot 2024-09-12T13:45:59.088 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 mirror image enable mirror/image-primary7 snapshot 2024-09-12T13:45:59.271 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:45:59.271 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:45:59.271 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:45:59.272 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:45:59.272 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:45:59.272 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:45:59.295 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1996 2024-09-12T13:45:59.310 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Mirroring enabled 2024-09-12T13:45:59.325 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1995 2024-09-12T13:45:59.346 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 info mirror/image-primary7 2024-09-12T13:45:59.361 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:45:59.361 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:45:59.387 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local ret=0 2024-09-12T13:45:59.387 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ timeout 5m bash -c 'zcat test-primary6/linux-5.4.tar.gz | pv -L 256K | tar xf - -C test-primary6' 2024-09-12T13:45:59.426 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:rbd image 'image-primary7': 2024-09-12T13:45:59.427 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: size 10 GiB in 2560 objects 2024-09-12T13:45:59.427 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: order 22 (4 MiB objects) 2024-09-12T13:45:59.427 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: snapshot_count: 1 2024-09-12T13:45:59.427 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: id: 2ea15ea2332e 2024-09-12T13:45:59.427 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: block_name_prefix: rbd_data.2ea15ea2332e 2024-09-12T13:45:59.427 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: format: 2 2024-09-12T13:45:59.427 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: features: layering, exclusive-lock, object-map, fast-diff 2024-09-12T13:45:59.427 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: op_features: 2024-09-12T13:45:59.427 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: flags: 2024-09-12T13:45:59.427 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: create_timestamp: Thu Sep 12 13:45:59 2024 2024-09-12T13:45:59.427 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: access_timestamp: Thu Sep 12 13:45:59 2024 2024-09-12T13:45:59.427 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: modify_timestamp: Thu Sep 12 13:45:59 2024 2024-09-12T13:45:59.428 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring state: enabled 2024-09-12T13:45:59.428 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring mode: snapshot 2024-09-12T13:45:59.428 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring global id: ecec213c-dc11-4547-88c6-ded8df679ff6 2024-09-12T13:45:59.428 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring primary: true 2024-09-12T13:45:59.432 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \n\b\d ]] 2024-09-12T13:45:59.432 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \k\r\b\d ]] 2024-09-12T13:45:59.433 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo rbd --cluster cluster1 device map -t krbd mirror/image-primary7 2024-09-12T13:45:59.598 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEV=/dev/rbd6 2024-09-12T13:45:59.598 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEVS+=($DEV) 2024-09-12T13:45:59.598 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo mkfs.ext4 /dev/rbd6 2024-09-12T13:45:59.619 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:mke2fs 1.46.5 (30-Dec-2021) 2024-09-12T13:45:59.632 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Discarding device blocks: 0/2621440 done 2024-09-12T13:45:59.632 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Creating filesystem with 2621440 4k blocks and 655360 inodes 2024-09-12T13:45:59.632 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Filesystem UUID: ae70519c-beb8-400e-b2fa-4d691bcd3392 2024-09-12T13:45:59.632 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Superblock backups stored on blocks: 2024-09-12T13:45:59.632 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 2024-09-12T13:45:59.632 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 2024-09-12T13:45:59.633 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Allocating group tables: 0/80 done 2024-09-12T13:45:59.633 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Writing inode tables: 0/80 done 2024-09-12T13:45:59.637 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Creating journal (16384 blocks): done 2024-09-12T13:45:59.739 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Writing superblocks and filesystem accounting information: 0/80 done 2024-09-12T13:45:59.740 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 2024-09-12T13:45:59.742 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mkdir test-primary7 2024-09-12T13:45:59.745 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo mount /dev/rbd6 test-primary7 2024-09-12T13:45:59.804 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ whoami 2024-09-12T13:45:59.807 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo chown ubuntu test-primary7 2024-09-12T13:45:59.831 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ SNAP_PIDS+=($!) 2024-09-12T13:45:59.832 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ take_mirror_snapshots cluster1 mirror image-primary7 2024-09-12T13:45:59.833 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ WORKLOAD_PIDS+=($!) 2024-09-12T13:45:59.833 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:45:59.833 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:45:59.833 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ IMG=image-primary8 2024-09-12T13:45:59.833 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ MNTPT=test-primary8 2024-09-12T13:45:59.833 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:45:59.833 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ create_image_and_enable_mirror cluster1 mirror image-primary8 snapshot 10G 2024-09-12T13:45:59.833 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ slow_untar_workload test-primary7 2024-09-12T13:45:59.833 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:45:59.834 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:45:59.835 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:45:59.835 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:45:59.835 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:45:59.835 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:45:59.835 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:45:59.835 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:45:59.835 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:45:59.835 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:45:59.835 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:45:59.835 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:45:59.835 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mode=snapshot 2024-09-12T13:45:59.835 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n snapshot ']' 2024-09-12T13:45:59.836 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:45:59.836 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:45:59.836 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ create_image cluster1 mirror image-primary8 10G 2024-09-12T13:45:59.836 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mountpt=test-primary7 2024-09-12T13:45:59.965 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ cp linux-5.4.tar.gz test-primary7 2024-09-12T13:45:59.965 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:45:59.965 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:45:59.966 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:45:59.966 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:45:59.966 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:45:59.966 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:45:59.966 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local size=128 2024-09-12T13:45:59.966 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n 10G ']' 2024-09-12T13:45:59.967 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ size=10G 2024-09-12T13:45:59.967 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:45:59.967 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 create --size 10G --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image-primary8 2024-09-12T13:45:59.967 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ is_pool_mirror_mode_image mirror 2024-09-12T13:45:59.967 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:45:59.967 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' image = image ']' 2024-09-12T13:45:59.968 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:45:59.968 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ enable_mirror cluster1 mirror image-primary8 snapshot 2024-09-12T13:45:59.968 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:45:59.968 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:45:59.968 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:45:59.968 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mode=snapshot 2024-09-12T13:45:59.969 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 mirror image enable mirror/image-primary8 snapshot 2024-09-12T13:46:00.253 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Mirroring enabled 2024-09-12T13:46:00.260 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 1998 2024-09-12T13:46:00.263 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2000 2024-09-12T13:46:00.284 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 info mirror/image-primary8 2024-09-12T13:46:00.285 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:46:00.289 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:46:00.322 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:46:00.323 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:46:00.323 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:46:00.323 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:46:00.323 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:46:00.323 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:46:00.396 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:rbd image 'image-primary8': 2024-09-12T13:46:00.396 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: size 10 GiB in 2560 objects 2024-09-12T13:46:00.396 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: order 22 (4 MiB objects) 2024-09-12T13:46:00.396 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: snapshot_count: 1 2024-09-12T13:46:00.396 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: id: 2eb0dbce1dc5 2024-09-12T13:46:00.397 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: block_name_prefix: rbd_data.2eb0dbce1dc5 2024-09-12T13:46:00.397 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: format: 2 2024-09-12T13:46:00.397 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: features: layering, exclusive-lock, object-map, fast-diff 2024-09-12T13:46:00.397 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: op_features: 2024-09-12T13:46:00.397 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: flags: 2024-09-12T13:46:00.397 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: create_timestamp: Thu Sep 12 13:45:59 2024 2024-09-12T13:46:00.397 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: access_timestamp: Thu Sep 12 13:45:59 2024 2024-09-12T13:46:00.397 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: modify_timestamp: Thu Sep 12 13:45:59 2024 2024-09-12T13:46:00.398 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring state: enabled 2024-09-12T13:46:00.398 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring mode: snapshot 2024-09-12T13:46:00.398 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring global id: 901b32e6-a0c7-4be3-ada0-5f0083a2314c 2024-09-12T13:46:00.398 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring primary: true 2024-09-12T13:46:00.406 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \n\b\d ]] 2024-09-12T13:46:00.406 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \k\r\b\d ]] 2024-09-12T13:46:00.408 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo rbd --cluster cluster1 device map -t krbd mirror/image-primary8 2024-09-12T13:46:00.538 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEV=/dev/rbd7 2024-09-12T13:46:00.538 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEVS+=($DEV) 2024-09-12T13:46:00.538 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo mkfs.ext4 /dev/rbd7 2024-09-12T13:46:00.559 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:mke2fs 1.46.5 (30-Dec-2021) 2024-09-12T13:46:00.571 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Discarding device blocks: 0/2621440 done 2024-09-12T13:46:00.571 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Creating filesystem with 2621440 4k blocks and 655360 inodes 2024-09-12T13:46:00.572 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Filesystem UUID: a92bbcb0-9fa1-4ce3-890a-526ccd94c968 2024-09-12T13:46:00.572 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Superblock backups stored on blocks: 2024-09-12T13:46:00.572 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 2024-09-12T13:46:00.572 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 2024-09-12T13:46:00.572 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Allocating group tables: 0/80 done 2024-09-12T13:46:00.572 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Writing inode tables: 0/80 done 2024-09-12T13:46:00.577 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Creating journal (16384 blocks): done 2024-09-12T13:46:00.628 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local ret=0 2024-09-12T13:46:00.628 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ timeout 5m bash -c 'zcat test-primary7/linux-5.4.tar.gz | pv -L 256K | tar xf - -C test-primary7' 2024-09-12T13:46:00.632 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Writing superblocks and filesystem accounting information: 0/80 done 2024-09-12T13:46:00.633 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 2024-09-12T13:46:00.635 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mkdir test-primary8 2024-09-12T13:46:00.638 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo mount /dev/rbd7 test-primary8 2024-09-12T13:46:00.693 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ whoami 2024-09-12T13:46:00.695 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo chown ubuntu test-primary8 2024-09-12T13:46:00.720 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ SNAP_PIDS+=($!) 2024-09-12T13:46:00.720 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ take_mirror_snapshots cluster1 mirror image-primary8 2024-09-12T13:46:00.720 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:46:00.720 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:46:00.720 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:46:00.720 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:46:00.720 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:46:00.720 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:46:00.720 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:46:00.720 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ WORKLOAD_PIDS+=($!) 2024-09-12T13:46:00.721 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:46:00.721 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:46:00.721 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:46:00.721 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ IMG=image-primary9 2024-09-12T13:46:00.721 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ MNTPT=test-primary9 2024-09-12T13:46:00.721 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ create_image_and_enable_mirror cluster1 mirror image-primary9 snapshot 10G 2024-09-12T13:46:00.721 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ slow_untar_workload test-primary8 2024-09-12T13:46:00.722 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:46:00.722 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:46:00.722 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:46:00.722 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:46:00.723 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:46:00.723 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:46:00.723 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mode=snapshot 2024-09-12T13:46:00.723 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n snapshot ']' 2024-09-12T13:46:00.723 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:46:00.723 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ create_image cluster1 mirror image-primary9 10G 2024-09-12T13:46:00.723 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:46:00.723 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:46:00.723 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:46:00.723 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:46:00.723 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:46:00.724 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:46:00.724 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mountpt=test-primary8 2024-09-12T13:46:00.724 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local size=128 2024-09-12T13:46:00.724 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n 10G ']' 2024-09-12T13:46:00.724 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ size=10G 2024-09-12T13:46:00.724 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:46:00.724 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ cp linux-5.4.tar.gz test-primary8 2024-09-12T13:46:00.725 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 create --size 10G --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image-primary9 2024-09-12T13:46:00.792 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ is_pool_mirror_mode_image mirror 2024-09-12T13:46:00.792 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:46:00.792 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' image = image ']' 2024-09-12T13:46:00.792 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:46:00.792 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ enable_mirror cluster1 mirror image-primary9 snapshot 2024-09-12T13:46:00.793 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:46:00.793 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:46:00.793 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:46:00.793 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mode=snapshot 2024-09-12T13:46:00.793 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 mirror image enable mirror/image-primary9 snapshot 2024-09-12T13:46:01.018 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local ret=0 2024-09-12T13:46:01.018 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ timeout 5m bash -c 'zcat test-primary8/linux-5.4.tar.gz | pv -L 256K | tar xf - -C test-primary8' 2024-09-12T13:46:01.264 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2001 2024-09-12T13:46:01.266 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Mirroring enabled 2024-09-12T13:46:01.282 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2002 2024-09-12T13:46:01.290 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:46:01.296 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 info mirror/image-primary9 2024-09-12T13:46:01.304 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:46:01.352 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:46:01.352 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:46:01.352 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:46:01.352 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:46:01.353 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:46:01.353 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:46:01.356 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:46:01.356 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:46:01.356 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:46:01.356 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:46:01.356 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:46:01.356 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:46:01.386 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:rbd image 'image-primary9': 2024-09-12T13:46:01.386 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: size 10 GiB in 2560 objects 2024-09-12T13:46:01.386 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: order 22 (4 MiB objects) 2024-09-12T13:46:01.386 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: snapshot_count: 1 2024-09-12T13:46:01.386 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: id: 2ec3d566ecee 2024-09-12T13:46:01.386 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: block_name_prefix: rbd_data.2ec3d566ecee 2024-09-12T13:46:01.386 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: format: 2 2024-09-12T13:46:01.386 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: features: layering, exclusive-lock, object-map, fast-diff 2024-09-12T13:46:01.387 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: op_features: 2024-09-12T13:46:01.387 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: flags: 2024-09-12T13:46:01.387 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: create_timestamp: Thu Sep 12 13:46:00 2024 2024-09-12T13:46:01.387 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: access_timestamp: Thu Sep 12 13:46:00 2024 2024-09-12T13:46:01.387 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: modify_timestamp: Thu Sep 12 13:46:00 2024 2024-09-12T13:46:01.387 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring state: enabled 2024-09-12T13:46:01.387 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring mode: snapshot 2024-09-12T13:46:01.387 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring global id: 43c84ab2-a21e-441e-82d9-32f1b27926a0 2024-09-12T13:46:01.387 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring primary: true 2024-09-12T13:46:01.397 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \n\b\d ]] 2024-09-12T13:46:01.397 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \k\r\b\d ]] 2024-09-12T13:46:01.398 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo rbd --cluster cluster1 device map -t krbd mirror/image-primary9 2024-09-12T13:46:01.522 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEV=/dev/rbd8 2024-09-12T13:46:01.522 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEVS+=($DEV) 2024-09-12T13:46:01.522 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo mkfs.ext4 /dev/rbd8 2024-09-12T13:46:01.543 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:mke2fs 1.46.5 (30-Dec-2021) 2024-09-12T13:46:01.556 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Discarding device blocks: 0/2621440 done 2024-09-12T13:46:01.556 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Creating filesystem with 2621440 4k blocks and 655360 inodes 2024-09-12T13:46:01.556 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Filesystem UUID: c4ccbfdf-6af1-4d36-899a-a386e3149f28 2024-09-12T13:46:01.556 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Superblock backups stored on blocks: 2024-09-12T13:46:01.556 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 2024-09-12T13:46:01.556 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 2024-09-12T13:46:01.556 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Allocating group tables: 0/80 done 2024-09-12T13:46:01.556 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Writing inode tables: 0/80 done 2024-09-12T13:46:01.561 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Creating journal (16384 blocks): done 2024-09-12T13:46:01.618 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Writing superblocks and filesystem accounting information: 0/80 done 2024-09-12T13:46:01.618 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 2024-09-12T13:46:01.621 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mkdir test-primary9 2024-09-12T13:46:01.623 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo mount /dev/rbd8 test-primary9 2024-09-12T13:46:01.678 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ whoami 2024-09-12T13:46:01.680 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo chown ubuntu test-primary9 2024-09-12T13:46:01.705 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ SNAP_PIDS+=($!) 2024-09-12T13:46:01.705 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ take_mirror_snapshots cluster1 mirror image-primary9 2024-09-12T13:46:01.705 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:46:01.705 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:46:01.705 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:46:01.705 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:46:01.705 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:46:01.705 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:46:01.705 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:46:01.705 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:46:01.705 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:46:01.705 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ WORKLOAD_PIDS+=($!) 2024-09-12T13:46:01.706 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:46:01.706 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ IMG=image-primary10 2024-09-12T13:46:01.707 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ MNTPT=test-primary10 2024-09-12T13:46:01.707 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ create_image_and_enable_mirror cluster1 mirror image-primary10 snapshot 10G 2024-09-12T13:46:01.707 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ slow_untar_workload test-primary9 2024-09-12T13:46:01.707 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:46:01.707 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:46:01.707 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:46:01.708 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:46:01.708 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:46:01.708 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:46:01.708 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mode=snapshot 2024-09-12T13:46:01.708 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n snapshot ']' 2024-09-12T13:46:01.708 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:46:01.708 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ create_image cluster1 mirror image-primary10 10G 2024-09-12T13:46:01.708 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:46:01.708 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:46:01.708 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:46:01.708 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:46:01.708 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:46:01.708 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:46:01.709 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mountpt=test-primary9 2024-09-12T13:46:01.709 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local size=128 2024-09-12T13:46:01.709 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n 10G ']' 2024-09-12T13:46:01.709 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ size=10G 2024-09-12T13:46:01.709 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:46:01.709 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ cp linux-5.4.tar.gz test-primary9 2024-09-12T13:46:01.709 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 create --size 10G --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image-primary10 2024-09-12T13:46:01.777 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ is_pool_mirror_mode_image mirror 2024-09-12T13:46:01.777 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:46:01.777 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' image = image ']' 2024-09-12T13:46:01.777 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:46:01.777 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ enable_mirror cluster1 mirror image-primary10 snapshot 2024-09-12T13:46:01.778 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:46:01.778 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:46:01.778 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:46:01.778 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mode=snapshot 2024-09-12T13:46:01.778 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 mirror image enable mirror/image-primary10 snapshot 2024-09-12T13:46:02.001 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local ret=0 2024-09-12T13:46:02.002 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ timeout 5m bash -c 'zcat test-primary9/linux-5.4.tar.gz | pv -L 256K | tar xf - -C test-primary9' 2024-09-12T13:46:02.274 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2004 2024-09-12T13:46:02.282 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Mirroring enabled 2024-09-12T13:46:02.288 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2006 2024-09-12T13:46:02.289 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2005 2024-09-12T13:46:02.302 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:46:02.308 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:46:02.316 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:46:02.321 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 info mirror/image-primary10 2024-09-12T13:46:02.365 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:46:02.366 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:46:02.366 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:46:02.366 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:46:02.366 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:46:02.366 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:46:02.369 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:46:02.370 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:46:02.370 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:46:02.370 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:46:02.370 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:46:02.370 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:46:02.627 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:rbd image 'image-primary10': 2024-09-12T13:46:02.627 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: size 10 GiB in 2560 objects 2024-09-12T13:46:02.628 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: order 22 (4 MiB objects) 2024-09-12T13:46:02.628 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: snapshot_count: 1 2024-09-12T13:46:02.628 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: id: 2ed8dff63328 2024-09-12T13:46:02.628 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: block_name_prefix: rbd_data.2ed8dff63328 2024-09-12T13:46:02.628 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: format: 2 2024-09-12T13:46:02.628 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: features: layering, exclusive-lock, object-map, fast-diff 2024-09-12T13:46:02.628 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: op_features: 2024-09-12T13:46:02.628 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: flags: 2024-09-12T13:46:02.628 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: create_timestamp: Thu Sep 12 13:46:01 2024 2024-09-12T13:46:02.628 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: access_timestamp: Thu Sep 12 13:46:01 2024 2024-09-12T13:46:02.628 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: modify_timestamp: Thu Sep 12 13:46:01 2024 2024-09-12T13:46:02.628 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring state: enabled 2024-09-12T13:46:02.628 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring mode: snapshot 2024-09-12T13:46:02.628 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring global id: 02633bcb-38b0-4c95-9ca5-a988ab7c0dd0 2024-09-12T13:46:02.629 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring primary: true 2024-09-12T13:46:02.633 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \n\b\d ]] 2024-09-12T13:46:02.633 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \k\r\b\d ]] 2024-09-12T13:46:02.634 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo rbd --cluster cluster1 device map -t krbd mirror/image-primary10 2024-09-12T13:46:02.761 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEV=/dev/rbd9 2024-09-12T13:46:02.762 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEVS+=($DEV) 2024-09-12T13:46:02.762 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo mkfs.ext4 /dev/rbd9 2024-09-12T13:46:02.782 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:mke2fs 1.46.5 (30-Dec-2021) 2024-09-12T13:46:02.795 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Discarding device blocks: 0/2621440 done 2024-09-12T13:46:02.795 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Creating filesystem with 2621440 4k blocks and 655360 inodes 2024-09-12T13:46:02.795 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Filesystem UUID: bad45549-66f5-4e00-a55d-681b9b634fc8 2024-09-12T13:46:02.795 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Superblock backups stored on blocks: 2024-09-12T13:46:02.796 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 2024-09-12T13:46:02.796 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 2024-09-12T13:46:02.796 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Allocating group tables: 0/80 done 2024-09-12T13:46:02.796 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Writing inode tables: 0/80 done 2024-09-12T13:46:02.800 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Creating journal (16384 blocks): done 2024-09-12T13:46:02.862 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Writing superblocks and filesystem accounting information: 0/80 done 2024-09-12T13:46:02.863 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 2024-09-12T13:46:02.865 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mkdir test-primary10 2024-09-12T13:46:02.867 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo mount /dev/rbd9 test-primary10 2024-09-12T13:46:03.014 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ whoami 2024-09-12T13:46:03.016 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo chown ubuntu test-primary10 2024-09-12T13:46:03.040 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ SNAP_PIDS+=($!) 2024-09-12T13:46:03.040 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ take_mirror_snapshots cluster1 mirror image-primary10 2024-09-12T13:46:03.041 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:46:03.041 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:46:03.041 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:46:03.041 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:46:03.041 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:46:03.041 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:46:03.041 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:46:03.041 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:46:03.041 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:46:03.041 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ WORKLOAD_PIDS+=($!) 2024-09-12T13:46:03.041 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${SNAP_PIDS[@]} 2024-09-12T13:46:03.041 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ slow_untar_workload test-primary10 2024-09-12T13:46:03.041 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 106402 2024-09-12T13:46:03.042 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mountpt=test-primary10 2024-09-12T13:46:03.042 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ cp linux-5.4.tar.gz test-primary10 2024-09-12T13:46:03.249 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2010 2024-09-12T13:46:03.257 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2008 2024-09-12T13:46:03.263 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2009 2024-09-12T13:46:03.291 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:46:03.292 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:46:03.292 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:46:03.292 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:46:03.292 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:46:03.292 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:46:03.292 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:46:03.297 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:46:03.297 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:46:03.298 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:46:03.298 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:46:03.298 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:46:03.298 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:46:03.298 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:46:03.382 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:46:03.541 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local ret=0 2024-09-12T13:46:03.542 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ timeout 5m bash -c 'zcat test-primary10/linux-5.4.tar.gz | pv -L 256K | tar xf - -C test-primary10' 2024-09-12T13:46:04.282 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2011 2024-09-12T13:46:04.283 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2012 2024-09-12T13:46:04.294 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:46:04.294 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:46:04.294 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:46:04.294 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:46:04.295 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:46:04.295 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:46:04.309 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:46:04.309 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:46:04.309 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:46:04.309 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:46:04.309 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:46:04.309 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:46:04.336 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:46:04.343 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:46:05.309 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:46:05.309 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:46:05.309 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:46:05.309 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:46:05.309 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:46:05.309 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:46:05.314 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:46:05.315 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:46:05.315 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:46:05.315 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:46:05.316 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:46:05.316 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:46:05.318 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2014 2024-09-12T13:46:05.324 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:46:05.324 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:46:05.324 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:46:05.324 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:46:05.324 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:46:05.324 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:46:05.325 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2013 2024-09-12T13:46:05.361 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:46:05.366 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:46:06.300 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:46:06.301 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:46:06.301 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:46:06.301 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:46:06.301 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:46:06.301 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:46:06.302 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:46:06.302 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:46:06.302 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:46:06.302 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:46:06.302 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:46:06.302 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:46:06.344 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2017 2024-09-12T13:46:06.345 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2015 2024-09-12T13:46:06.357 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2016 2024-09-12T13:46:06.387 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:46:06.394 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:46:06.394 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:46:06.394 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:46:06.394 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:46:06.394 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:46:06.394 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:46:06.401 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:46:06.412 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:46:07.341 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:46:07.341 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:46:07.341 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:46:07.341 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:46:07.342 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:46:07.342 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:46:07.348 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:46:07.348 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:46:07.348 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:46:07.348 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:46:07.348 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:46:07.348 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:46:07.395 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2019 2024-09-12T13:46:07.399 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2020 2024-09-12T13:46:07.406 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2018 2024-09-12T13:46:07.455 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:46:07.461 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:46:07.492 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:46:08.371 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:46:08.372 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:46:08.372 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:46:08.372 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:46:08.372 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:46:08.372 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:46:08.372 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:46:08.372 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:46:08.372 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:46:08.372 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:46:08.372 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:46:08.373 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:46:08.467 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2021 2024-09-12T13:46:08.472 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2022 2024-09-12T13:46:08.498 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:46:08.508 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:46:09.335 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2023 2024-09-12T13:46:09.336 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2024 2024-09-12T13:46:09.358 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:46:09.392 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:46:09.395 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:46:09.395 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:46:09.395 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:46:09.395 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:46:09.395 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:46:09.395 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:46:09.411 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:46:09.411 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:46:09.411 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:46:09.411 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:46:09.411 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:46:09.411 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:46:09.419 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:46:09.419 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:46:09.419 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:46:09.419 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:46:09.419 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:46:09.419 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:46:10.376 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2025 2024-09-12T13:46:10.381 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2026 2024-09-12T13:46:10.414 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2027 2024-09-12T13:46:10.418 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:46:10.421 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:46:10.444 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:46:10.463 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:46:10.463 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:46:10.463 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:46:10.463 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:46:10.463 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:46:10.463 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:46:10.474 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:46:10.474 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:46:10.474 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:46:10.474 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:46:10.474 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:46:10.474 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:46:10.496 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:46:10.496 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:46:10.496 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:46:10.497 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:46:10.497 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:46:10.497 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:46:11.371 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2030 2024-09-12T13:46:11.383 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2028 2024-09-12T13:46:11.398 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2029 2024-09-12T13:46:11.425 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:46:11.427 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:46:11.454 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:46:11.501 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:46:11.501 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:46:11.501 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:46:11.502 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:46:11.502 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:46:11.502 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:46:11.511 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:46:11.511 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:46:11.511 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:46:11.512 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:46:11.512 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:46:11.512 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:46:12.366 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:46:12.366 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:46:12.367 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:46:12.367 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:46:12.367 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:46:12.367 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:46:12.401 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:46:12.401 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:46:12.401 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:46:12.401 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:46:12.401 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:46:12.402 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:46:12.421 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2032 2024-09-12T13:46:12.433 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2031 2024-09-12T13:46:12.476 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:46:12.480 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:46:13.257 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2034 2024-09-12T13:46:13.269 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2033 2024-09-12T13:46:13.275 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:46:13.314 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:46:13.423 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:46:13.423 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:46:13.423 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:46:13.423 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:46:13.423 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:46:13.423 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:46:13.426 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:46:13.426 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:46:13.426 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:46:13.426 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:46:13.426 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:46:13.426 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:46:13.447 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:46:13.448 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:46:13.448 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:46:13.448 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:46:13.448 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:46:13.448 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:46:14.273 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2035 2024-09-12T13:46:14.281 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2036 2024-09-12T13:46:14.287 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:46:14.361 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:46:14.429 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:46:14.429 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:46:14.429 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:46:14.429 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:46:14.429 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:46:14.429 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:46:14.431 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:46:14.432 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:46:14.432 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:46:14.432 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:46:14.432 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:46:14.432 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:46:14.460 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:46:14.460 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:46:14.460 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:46:14.461 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:46:14.461 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:46:14.461 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:46:15.307 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2037 2024-09-12T13:46:15.325 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:46:15.441 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2039 2024-09-12T13:46:15.447 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2038 2024-09-12T13:46:15.464 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:46:15.477 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:46:15.482 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:46:15.482 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:46:15.482 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:46:15.482 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:46:15.482 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:46:15.482 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:46:15.488 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:46:15.488 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:46:15.488 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:46:15.488 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:46:15.488 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:46:15.488 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:46:16.280 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:46:16.280 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:46:16.280 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:46:16.280 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:46:16.280 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:46:16.280 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:46:16.319 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:46:16.320 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:46:16.320 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:46:16.320 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:46:16.320 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:46:16.320 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:46:16.390 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2040 2024-09-12T13:46:16.454 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:46:16.479 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2041 2024-09-12T13:46:16.497 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:46:17.292 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:46:17.292 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:46:17.292 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:46:17.292 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:46:17.292 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:46:17.292 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:46:17.368 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:46:17.368 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:46:17.368 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:46:17.368 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:46:17.369 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:46:17.369 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:46:17.396 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:46:17.396+0000 7f67dd000640 -1 rbd::mirror::image_replayer::snapshot::Replayer: 0x5605568d0800 handle_unlink_peer: failed to unlink local peer from remote image: (22) Invalid argument 2024-09-12T13:46:17.467 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2043 2024-09-12T13:46:17.480 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:46:18.219 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2042 2024-09-12T13:46:18.263 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2044 2024-09-12T13:46:18.270 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:46:18.301 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:46:18.312 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2045 2024-09-12T13:46:18.328 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:46:18.329 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:46:18.329 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:46:18.329 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:46:18.330 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:46:18.330 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:46:18.330 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:46:18.467 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:46:18.467 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:46:18.467 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:46:18.467 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:46:18.468 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:46:18.468 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:46:18.480 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:46:18.480 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:46:18.480 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:46:18.480 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:46:18.480 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:46:18.480 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:46:19.275 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2046 2024-09-12T13:46:19.295 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:46:19.303 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2048 2024-09-12T13:46:19.322 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:46:19.457 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:46:19.457 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:46:19.457 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:46:19.457 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:46:19.458 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:46:19.458 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:46:19.500 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:46:19.501 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:46:19.501 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:46:19.501 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:46:19.501 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:46:19.501 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:46:20.322 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2049 2024-09-12T13:46:20.330 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2047 2024-09-12T13:46:20.346 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:46:20.362 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:46:20.483 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:46:20.483 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:46:20.483 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:46:20.483 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:46:20.483 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:46:20.483 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:46:21.263 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2050 2024-09-12T13:46:21.273 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:46:21.274 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:46:21.275 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:46:21.275 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:46:21.275 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:46:21.275 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:46:21.295 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2051 2024-09-12T13:46:21.309 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:46:21.311 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:46:21.311 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:46:21.311 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:46:21.311 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:46:21.311 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:46:21.311 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:46:21.327 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:46:21.332 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:46:21.332 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:46:21.332 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:46:21.332 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:46:21.332 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:46:21.332 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:46:22.298 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:46:22.299 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:46:22.299 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:46:22.299 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:46:22.299 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:46:22.299 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:46:22.302 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2052 2024-09-12T13:46:22.314 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2054 2024-09-12T13:46:22.316 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:46:22.327 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:46:22.327 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:46:22.327 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:46:22.327 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:46:22.327 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:46:22.327 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:46:22.339 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:46:23.323 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2055 2024-09-12T13:46:23.334 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2053 2024-09-12T13:46:23.350 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:46:23.350 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:46:23.350 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:46:23.350 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:46:23.351 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:46:23.351 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:46:23.365 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:46:23.365 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:46:23.366 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:46:23.366 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:46:23.366 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:46:23.366 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:46:23.366 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:46:23.373 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:46:24.314 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:46:24.315 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:46:24.315 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:46:24.315 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:46:24.315 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:46:24.315 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:46:24.327 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2056 2024-09-12T13:46:24.331 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:46:24.331 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:46:24.331 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:46:24.331 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:46:24.331 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:46:24.331 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:46:24.346 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:46:25.257 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2057 2024-09-12T13:46:25.317 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:46:25.320 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:46:25.320 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:46:25.320 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:46:25.320 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:46:25.320 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:46:25.320 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:46:25.341 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:46:25.342 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:46:25.342 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:46:25.342 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:46:25.342 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:46:25.342 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:46:26.263 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2059 2024-09-12T13:46:26.289 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2058 2024-09-12T13:46:26.334 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2060 2024-09-12T13:46:26.336 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:46:26.373 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:46:26.373 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:46:26.374 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:46:26.374 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:46:26.374 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:46:26.374 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:46:26.378 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:46:26.379 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:46:26.379 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:46:26.380 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:46:26.380 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:46:26.382 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:46:26.392 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:46:26.448 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:46:26.581 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2061 2024-09-12T13:46:26.605 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:46:27.312 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2062 2024-09-12T13:46:27.327 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:46:27.351 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:46:27.351 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:46:27.351 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:46:27.351 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:46:27.351 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:46:27.351 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:46:27.390 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2063 2024-09-12T13:46:27.441 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:46:28.267 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2064 2024-09-12T13:46:28.294 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2065 2024-09-12T13:46:28.313 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:46:28.320 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:46:28.321 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:46:28.321 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:46:28.321 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:46:28.321 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:46:28.321 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:46:28.324 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:46:29.311 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2066 2024-09-12T13:46:29.322 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2067 2024-09-12T13:46:29.329 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:46:29.340 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:46:29.340 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:46:29.340 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:46:29.340 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:46:29.341 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:46:29.341 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:46:29.346 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:46:29.404 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:46:29.404 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:46:29.404 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:46:29.405 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:46:29.405 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:46:29.405 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:46:29.456 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:46:29.457 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:46:29.461 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:46:29.461 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:46:29.462 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:46:29.462 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:46:29.608 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:46:29.609 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:46:29.609 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:46:29.609 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:46:29.609 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:46:29.609 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:46:30.317 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2069 2024-09-12T13:46:30.327 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2068 2024-09-12T13:46:30.331 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:46:30.331 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:46:30.331 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:46:30.331 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:46:30.331 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:46:30.331 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:46:30.335 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:46:30.337 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:46:30.443 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:46:30.443 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:46:30.443 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:46:30.444 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:46:30.444 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:46:30.444 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:46:31.318 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:46:31.318 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:46:31.318 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:46:31.318 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:46:31.318 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:46:31.318 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:46:31.330 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:46:31.330 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:46:31.330 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:46:31.330 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:46:31.330 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:46:31.330 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:46:31.401 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2071 2024-09-12T13:46:31.437 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:46:32.284 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2070 2024-09-12T13:46:32.328 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2073 2024-09-12T13:46:32.330 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:46:32.333 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:46:32.333 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:46:32.334 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:46:32.334 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:46:32.334 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:46:32.334 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:46:32.342 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2072 2024-09-12T13:46:32.350 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:46:32.350 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:46:32.350 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:46:32.350 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:46:32.351 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:46:32.351 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:46:32.357 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:46:32.366 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:46:33.313 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2074 2024-09-12T13:46:33.323 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2075 2024-09-12T13:46:33.338 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:46:33.339 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:46:33.339 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:46:33.339 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:46:33.339 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:46:33.339 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:46:33.340 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:46:33.340 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:46:33.340 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:46:33.340 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:46:33.340 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:46:33.340 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:46:33.342 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:46:33.355 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:46:34.397 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2076 2024-09-12T13:46:34.401 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2077 2024-09-12T13:46:34.417 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:46:34.436 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:46:34.441 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:46:34.441 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:46:34.441 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:46:34.441 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:46:34.441 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:46:34.441 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:46:35.334 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:46:35.334 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:46:35.334 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:46:35.334 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:46:35.334 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:46:35.334 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:46:35.360 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:46:35.360 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:46:35.360 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:46:35.360 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:46:35.360 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:46:35.360 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:46:35.372 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:46:35.373 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:46:35.373 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:46:35.373 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:46:35.373 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:46:35.373 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:46:35.373 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:46:35.370+0000 7f67dc600640 -1 rbd::mirror::image_replayer::snapshot::Replayer: 0x5605568d1400 handle_unlink_peer: failed to unlink local peer from remote image: (22) Invalid argument 2024-09-12T13:46:36.293 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2079 2024-09-12T13:46:36.309 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2078 2024-09-12T13:46:36.347 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:46:36.347 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:46:36.347 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:46:36.347 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:46:36.347 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:46:36.347 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:46:36.348 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:46:36.349 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:46:36.360 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:46:36.360 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:46:36.360 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:46:36.360 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:46:36.360 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:46:36.360 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:46:36.361 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2080 2024-09-12T13:46:36.387 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:46:36.403 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2081 2024-09-12T13:46:36.418 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:46:37.348 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2083 2024-09-12T13:46:37.410 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2082 2024-09-12T13:46:37.421 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:46:37.421 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:46:37.422 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:46:37.422 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:46:37.422 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:46:37.422 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:46:37.422 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:46:37.430 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:46:37.439 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:46:37.439 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:46:37.439 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:46:37.440 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:46:37.440 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:46:37.440 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:46:38.298 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2085 2024-09-12T13:46:38.319 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:46:38.326 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2084 2024-09-12T13:46:38.345 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:46:39.352 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:46:39.352 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:46:39.352 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:46:39.352 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:46:39.353 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:46:39.353 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:46:39.353 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:46:39.353 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:46:39.353 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:46:39.353 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:46:39.353 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:46:39.353 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:46:39.368 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2087 2024-09-12T13:46:39.373 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2086 2024-09-12T13:46:39.393 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:46:39.393 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:46:39.393 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:46:39.393 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:46:39.393 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:46:39.393 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:46:39.403 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:46:39.407 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:46:39.421 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:46:39.421 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:46:39.422 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:46:39.422 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:46:39.422 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:46:39.422 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:46:40.379 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2089 2024-09-12T13:46:40.390 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2088 2024-09-12T13:46:40.397 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:46:40.399 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:46:40.427 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:46:40.428 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:46:40.428 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:46:40.428 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:46:40.428 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:46:40.428 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:46:40.433 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:46:40.433 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:46:40.433 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:46:40.433 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:46:40.434 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:46:40.434 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:46:41.324 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:46:41.324 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:46:41.324 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:46:41.325 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:46:41.325 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:46:41.325 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:46:41.349 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:46:41.349 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:46:41.349 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:46:41.350 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:46:41.350 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:46:41.350 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:46:42.320 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2091 2024-09-12T13:46:42.329 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2090 2024-09-12T13:46:42.377 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:46:42.394 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:46:42.409 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:46:42.410 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:46:42.410 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:46:42.410 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:46:42.410 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:46:42.410 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:46:42.412 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:46:42.413 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:46:42.413 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:46:42.413 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:46:42.413 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:46:42.413 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:46:42.419 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2093 2024-09-12T13:46:42.491 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:46:43.250 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2092 2024-09-12T13:46:43.304 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2094 2024-09-12T13:46:43.319 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2095 2024-09-12T13:46:43.319 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:46:43.350 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:46:43.353 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:46:43.400 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:46:43.400 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:46:43.400 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:46:43.400 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:46:43.401 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:46:43.401 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:46:43.402 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:46:43.402 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:46:43.402 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:46:43.402 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:46:43.402 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:46:43.402 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:46:44.417 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2096 2024-09-12T13:46:44.467 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:46:44.490 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2099 2024-09-12T13:46:44.525 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:46:45.343 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2097 2024-09-12T13:46:45.375 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:46:45.381 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:46:45.382 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:46:45.382 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:46:45.382 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:46:45.382 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:46:45.382 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:46:45.398 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:46:45.399 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:46:45.399 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:46:45.399 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:46:45.399 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:46:45.399 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:46:45.494 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:46:45.494 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:46:45.494 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:46:45.494 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:46:45.494 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:46:45.495 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:46:46.322 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:46:46.322 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:46:46.322 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:46:46.322 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:46:46.322 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:46:46.322 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:46:46.353 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2098 2024-09-12T13:46:46.354 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:46:46.354 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:46:46.354 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:46:46.354 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:46:46.354 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:46:46.354 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:46:46.357 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:46:46.358 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:46:46.358 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:46:46.358 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:46:46.358 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:46:46.358 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:46:46.396 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:46:46.432 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2101 2024-09-12T13:46:46.442 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2100 2024-09-12T13:46:46.455 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:46:46.457 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:46:47.427 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2103 2024-09-12T13:46:47.444 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:46:47.470 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:46:47.470 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:46:47.470 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:46:47.470 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:46:47.470 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:46:47.470 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:46:47.528 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:46:47.528 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:46:47.528 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:46:47.528 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:46:47.528 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:46:47.528 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:46:48.247 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2102 2024-09-12T13:46:48.285 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2105 2024-09-12T13:46:48.298 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:46:48.307 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2104 2024-09-12T13:46:48.322 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:46:48.324 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:46:48.348 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2107 2024-09-12T13:46:48.363 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:46:48.378 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:46:48.379 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:46:48.379 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:46:48.379 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:46:48.379 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:46:48.379 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:46:49.288 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2106 2024-09-12T13:46:49.298 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2108 2024-09-12T13:46:49.302 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:46:49.319 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:46:49.399 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:46:49.399 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:46:49.399 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:46:49.400 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:46:49.400 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:46:49.400 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:46:49.457 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:46:49.457 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:46:49.457 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:46:49.458 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:46:49.458 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:46:49.458 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:46:49.460 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:46:49.460 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:46:49.460 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:46:49.460 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:46:49.461 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:46:49.461 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:46:50.312 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2109 2024-09-12T13:46:50.324 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:46:50.447 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:46:50.447 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:46:50.447 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:46:50.447 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:46:50.447 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:46:50.448 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:46:51.299 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2110 2024-09-12T13:46:51.303 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:46:51.303 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:46:51.303 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:46:51.303 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:46:51.303 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:46:51.303 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:46:51.326 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:46:51.326 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:46:51.326 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:46:51.326 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:46:51.327 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:46:51.327 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:46:51.327 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:46:51.327 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:46:51.327 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:46:51.327 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:46:51.327 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:46:51.327 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:46:51.333 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2111 2024-09-12T13:46:51.341 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:46:51.360 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:46:51.368 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:46:51.368 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:46:51.369 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:46:51.369 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:46:51.369 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:46:51.369 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:46:52.305 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:46:52.305 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:46:52.306 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:46:52.306 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:46:52.306 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:46:52.306 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:46:52.322 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:46:52.322 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:46:52.322 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:46:52.322 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:46:52.322 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:46:52.322 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:46:52.391 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2113 2024-09-12T13:46:52.427 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:46:53.251 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2112 2024-09-12T13:46:53.327 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:46:53.327 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:46:53.327 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:46:53.327 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:46:53.327 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:46:53.327 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:46:53.340 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:46:53.429 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:46:53.429+0000 7f67dd000640 -1 rbd::mirror::image_replayer::snapshot::Replayer: 0x56055b9f2000 handle_unlink_peer: failed to unlink local peer from remote image: (22) Invalid argument 2024-09-12T13:46:54.253 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2114 2024-09-12T13:46:54.270 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2115 2024-09-12T13:46:54.301 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:46:54.317 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2118 2024-09-12T13:46:54.329 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:46:54.344 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:46:54.344 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:46:54.344 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:46:54.344 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:46:54.344 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:46:54.344 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:46:54.360 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:46:54.363 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:46:54.363 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:46:54.363 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:46:54.363 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:46:54.364 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:46:54.364 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:46:55.265 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2117 2024-09-12T13:46:55.314 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:46:55.323 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:46:55.323+0000 7f67dd000640 -1 rbd::mirror::image_replayer::snapshot::Replayer: 0x56055b9f3800 handle_unlink_peer: failed to unlink local peer from remote image: (22) Invalid argument 2024-09-12T13:46:55.430 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:46:55.431 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:46:55.431 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:46:55.431 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:46:55.431 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:46:55.432 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:46:56.250 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2116 2024-09-12T13:46:56.295 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2119 2024-09-12T13:46:56.309 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:46:56.318 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2121 2024-09-12T13:46:56.337 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:46:56.343 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:46:56.343 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:46:56.343 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:46:56.343 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:46:56.343 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:46:56.343 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:46:56.356 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:46:57.277 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2120 2024-09-12T13:46:57.304 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:46:57.304 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:46:57.304 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:46:57.304 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:46:57.304 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:46:57.305 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:46:57.307 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2122 2024-09-12T13:46:57.319 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:46:57.331 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:46:57.332 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:46:57.332 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:46:57.333 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:46:57.333 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:46:57.333 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:46:57.333 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:46:57.339 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2123 2024-09-12T13:46:57.349 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:46:57.364 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:46:57.364 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:46:57.364 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:46:57.364 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:46:57.364 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:46:57.365 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:46:58.258 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2124 2024-09-12T13:46:58.267 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2125 2024-09-12T13:46:58.280 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:46:58.280 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:46:58.317 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:46:58.317 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:46:58.317 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:46:58.317 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:46:58.317 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:46:58.317 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:46:59.294 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2126 2024-09-12T13:46:59.313 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:46:59.313 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:46:59.313 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:46:59.313 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:46:59.313 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:46:59.313 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:46:59.334 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2127 2024-09-12T13:46:59.336 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:46:59.342 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:46:59.342 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:46:59.342 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:46:59.342 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:46:59.342 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:46:59.342 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:46:59.356 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:46:59.359 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:46:59.359 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:46:59.359 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:46:59.359 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:46:59.360 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:46:59.360 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:47:00.311 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2129 2024-09-12T13:47:00.312 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2128 2024-09-12T13:47:00.322 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:47:00.322 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:47:00.322 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:47:00.322 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:47:00.322 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:47:00.322 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:47:00.323 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:47:00.330 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:47:00.334 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:47:00.335 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:47:00.335 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:47:00.335 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:47:00.335 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:47:00.335 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:47:00.352 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:47:00.352 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:47:00.352 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:47:00.352 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:47:00.353 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:47:00.353 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:47:01.284 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:47:01.284 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:47:01.285 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:47:01.285 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:47:01.285 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:47:01.285 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:47:01.285 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:47:01.285 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:47:01.285 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:47:01.285 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:47:01.285 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:47:01.285 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:47:01.300 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2130 2024-09-12T13:47:01.334 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2131 2024-09-12T13:47:01.339 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:47:01.360 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:47:02.313 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2132 2024-09-12T13:47:02.314 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2133 2024-09-12T13:47:02.343 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:47:02.343 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:47:02.343 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:47:02.343 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:47:02.343 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:47:02.343 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:47:02.343 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:47:02.350 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:47:02.357 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2134 2024-09-12T13:47:02.359 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:47:02.359 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:47:02.359 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:47:02.359 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:47:02.359 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:47:02.359 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:47:02.373 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:47:03.258 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2135 2024-09-12T13:47:03.288 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:47:03.327 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:47:03.327 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:47:03.327 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:47:03.327 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:47:03.327 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:47:03.327 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:47:03.333 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:47:03.333 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:47:03.333 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:47:03.333 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:47:03.333 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:47:03.333 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:47:04.339 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2136 2024-09-12T13:47:04.343 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:47:04.343 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:47:04.343 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:47:04.343 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:47:04.343 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:47:04.343 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:47:04.357 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2137 2024-09-12T13:47:04.359 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:47:04.362 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:47:04.362 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:47:04.362 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:47:04.363 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:47:04.363 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:47:04.363 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:47:04.369 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:47:04.371 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2139 2024-09-12T13:47:04.389 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:47:05.350 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:47:05.350 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:47:05.350 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:47:05.350 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:47:05.350 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:47:05.350 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:47:05.355 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:47:05.355 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:47:05.355 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:47:05.355 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:47:05.355 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:47:05.355 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:47:05.376 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:47:05.376 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:47:05.377 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:47:05.377 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:47:05.377 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:47:05.377 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:47:06.293 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:47:06.294 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:47:06.294 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:47:06.294 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:47:06.294 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:47:06.294 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:47:06.310 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2138 2024-09-12T13:47:06.357 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:47:06.375 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:47:06.375+0000 7f67dc600640 -1 rbd::mirror::image_replayer::snapshot::Replayer: 0x56055c2e1c00 handle_unlink_peer: failed to unlink local peer from remote image: (22) Invalid argument 2024-09-12T13:47:06.380 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:47:06.379+0000 7f67dc600640 -1 rbd::mirror::image_replayer::snapshot::Replayer: 0x560556cc3c00 handle_unlink_peer: failed to unlink local peer from remote image: (22) Invalid argument 2024-09-12T13:47:06.387 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2144 2024-09-12T13:47:06.407 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:47:07.314 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2140 2024-09-12T13:47:07.323 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2141 2024-09-12T13:47:07.347 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2142 2024-09-12T13:47:07.355 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:47:07.363 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:47:07.363 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:47:07.363 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:47:07.363 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:47:07.363 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:47:07.363 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:47:07.366 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:47:07.372 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:47:07.372 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:47:07.372 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:47:07.372 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:47:07.372 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:47:07.372 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:47:07.379 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:47:07.392 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:47:07.393 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:47:07.393 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:47:07.393 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:47:07.393 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:47:07.393 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:47:08.557 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2143 2024-09-12T13:47:08.611 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:47:08.624 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:47:08.624+0000 7f67dc600640 -1 rbd::mirror::image_replayer::snapshot::Replayer: 0x56055c300c00 handle_unlink_peer: failed to unlink local peer from remote image: (22) Invalid argument 2024-09-12T13:47:08.635 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2148 2024-09-12T13:47:08.653 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:47:09.303 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2145 2024-09-12T13:47:09.346 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:47:09.359 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:47:09.359 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:47:09.360 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:47:09.360 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:47:09.360 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:47:09.360 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:47:09.410 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:47:09.410 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:47:09.410 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:47:09.411 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:47:09.411 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:47:09.411 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:47:10.315 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2146 2024-09-12T13:47:10.322 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2147 2024-09-12T13:47:10.348 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:47:10.351 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:47:10.358 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:47:10.358 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:47:10.358 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:47:10.358 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:47:10.358 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:47:10.358 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:47:10.369 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:47:10.369 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:47:10.369 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:47:10.369 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:47:10.369 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:47:10.369 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:47:10.371 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2150 2024-09-12T13:47:10.375 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2149 2024-09-12T13:47:10.383 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:47:10.383 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:47:10.383 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:47:10.383 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:47:10.384 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:47:10.384 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:47:10.387 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:47:10.394 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:47:11.342 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2152 2024-09-12T13:47:11.346 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2151 2024-09-12T13:47:11.360 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:47:11.362 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:47:11.614 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:47:11.614 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:47:11.614 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:47:11.614 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:47:11.614 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:47:11.614 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:47:11.656 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:47:11.656 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:47:11.657 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:47:11.657 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:47:11.657 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:47:11.657 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:47:12.334 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2153 2024-09-12T13:47:12.349 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:47:12.349 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:47:12.349 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:47:12.349 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:47:12.349 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:47:12.349 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:47:12.355 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:47:12.357 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2154 2024-09-12T13:47:12.370 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2155 2024-09-12T13:47:12.377 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:47:12.380 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:47:13.254 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2156 2024-09-12T13:47:13.268 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:47:13.351 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:47:13.351 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:47:13.351 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:47:13.351 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:47:13.352 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:47:13.352 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:47:13.353 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:47:13.354 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:47:13.354 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:47:13.354 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:47:13.354 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:47:13.354 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:47:13.390 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:47:13.390 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:47:13.390 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:47:13.390 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:47:13.390 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:47:13.390 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:47:13.397 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:47:13.397 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:47:13.397 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:47:13.397 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:47:13.397 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:47:13.397 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:47:14.361 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2157 2024-09-12T13:47:14.363 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:47:14.363 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:47:14.363 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:47:14.363 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:47:14.363 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:47:14.363 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:47:14.365 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:47:14.365 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:47:14.365 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:47:14.365 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:47:14.365 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:47:14.365 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:47:14.373 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2158 2024-09-12T13:47:14.385 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:47:14.390 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:47:15.332 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2159 2024-09-12T13:47:15.358 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:47:15.358 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:47:15.358 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:47:15.358 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:47:15.358 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:47:15.358 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:47:15.370 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:47:15.380 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:47:15.380 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:47:15.380 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:47:15.380 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:47:15.380 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:47:15.380 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:47:15.382 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:47:15.383 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:47:15.383 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:47:15.383 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:47:15.383 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:47:15.383 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:47:15.383 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2162 2024-09-12T13:47:15.387 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2161 2024-09-12T13:47:15.402 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:47:15.408 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:47:16.271 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:47:16.272 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:47:16.272 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:47:16.272 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:47:16.272 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:47:16.272 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:47:16.333 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2160 2024-09-12T13:47:16.358 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:47:17.338 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2165 2024-09-12T13:47:17.377 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:47:17.387 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:47:17.388 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:47:17.388 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:47:17.388 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:47:17.388 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:47:17.388 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:47:17.392 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:47:17.392 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:47:17.393 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:47:17.393 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:47:17.393 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:47:17.393 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:47:17.393 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2166 2024-09-12T13:47:17.410 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:47:18.256 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2164 2024-09-12T13:47:18.264 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2163 2024-09-12T13:47:18.311 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:47:18.313 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:47:18.374 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:47:18.374 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:47:18.374 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:47:18.375 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:47:18.375 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:47:18.375 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:47:18.405 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:47:18.405 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:47:18.405 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:47:18.406 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:47:18.406 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:47:18.406 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:47:18.411 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:47:18.411 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:47:18.412 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:47:18.412 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:47:18.412 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:47:18.412 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:47:19.361 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:47:19.361 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:47:19.361 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:47:19.361 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:47:19.362 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:47:19.362 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:47:19.386 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2167 2024-09-12T13:47:19.410 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:47:20.343 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2168 2024-09-12T13:47:20.351 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2169 2024-09-12T13:47:20.379 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:47:20.380 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:47:20.380 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:47:20.380 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:47:20.380 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:47:20.380 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:47:20.392 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:47:20.392 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2171 2024-09-12T13:47:20.402 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:47:20.403 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2170 2024-09-12T13:47:20.412 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:47:20.412 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:47:20.412 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:47:20.412 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:47:20.412 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:47:20.412 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:47:20.434 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:47:20.465 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:47:20.476 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2172 2024-09-12T13:47:20.486 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:47:21.314 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:47:21.314 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:47:21.314 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:47:21.314 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:47:21.315 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:47:21.315 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:47:21.318 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:47:21.318 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:47:21.318 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:47:21.319 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:47:21.319 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:47:21.319 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:47:22.414 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:47:22.414 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:47:22.414 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:47:22.414 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:47:22.415 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:47:22.415 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:47:22.418 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2175 2024-09-12T13:47:22.418 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2176 2024-09-12T13:47:22.435 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:47:22.435 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:47:23.292 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2173 2024-09-12T13:47:23.292 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2174 2024-09-12T13:47:23.312 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:47:23.318 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:47:23.395 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:47:23.395 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:47:23.396 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:47:23.396 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:47:23.396 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:47:23.396 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:47:23.405 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:47:23.405 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:47:23.406 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:47:23.406 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:47:23.406 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:47:23.406 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:47:23.436 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:47:23.436 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:47:23.437 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:47:23.437 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:47:23.437 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:47:23.437 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:47:23.468 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:47:23.468 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:47:23.469 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:47:23.469 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:47:23.469 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:47:23.469 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:47:23.489 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:47:23.489 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:47:23.489 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:47:23.489 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:47:23.490 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:47:23.490 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:47:24.266 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2177 2024-09-12T13:47:24.299 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:47:24.319 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2178 2024-09-12T13:47:24.364 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:47:25.325 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2182 2024-09-12T13:47:25.347 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:47:25.438 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:47:25.438 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:47:25.438 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:47:25.438 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:47:25.438 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:47:25.438 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:47:25.438 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:47:25.438 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:47:25.438 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:47:25.438 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:47:25.438 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:47:25.439 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:47:26.316 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:47:26.316 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:47:26.316 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:47:26.317 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:47:26.317 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:47:26.317 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:47:26.320 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:47:26.322 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:47:26.322 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:47:26.322 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:47:26.322 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:47:26.322 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:47:27.240 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2180 2024-09-12T13:47:27.244 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2181 2024-09-12T13:47:27.303 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:47:27.303 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:47:27.303 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:47:27.304 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:47:27.304 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:47:27.305 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:47:27.313 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2184 2024-09-12T13:47:27.328 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:47:27.336 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:47:27.359 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2179 2024-09-12T13:47:27.367 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:47:27.367 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:47:27.367 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:47:27.367 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:47:27.367 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:47:27.367 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:47:27.381 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:47:27.383 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:47:27.400 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:47:27.401+0000 7f67dd000640 -1 rbd::mirror::image_replayer::snapshot::Replayer: 0x56055ba78800 handle_unlink_peer: failed to unlink local peer from remote image: (22) Invalid argument 2024-09-12T13:47:28.248 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2183 2024-09-12T13:47:28.283 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2185 2024-09-12T13:47:28.310 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:47:28.331 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:47:28.351 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:47:28.351 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:47:28.351 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:47:28.351 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:47:28.351 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:47:28.352 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:47:29.287 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2188 2024-09-12T13:47:29.304 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2187 2024-09-12T13:47:29.323 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:47:29.336 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:47:30.303 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2189 2024-09-12T13:47:30.316 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:47:30.334 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:47:30.334 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:47:30.334 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:47:30.334 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:47:30.334 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:47:30.334 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:47:30.340 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:47:30.340 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:47:30.340 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:47:30.340 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:47:30.340 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:47:30.340 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:47:30.384 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:47:30.384 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:47:30.384 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:47:30.384 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:47:30.384 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:47:30.384 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:47:30.386 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:47:30.386 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:47:30.386 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:47:30.386 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:47:30.386 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:47:30.386 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:47:31.260 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2186 2024-09-12T13:47:31.289 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:47:31.310 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2191 2024-09-12T13:47:31.312 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2190 2024-09-12T13:47:31.314 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:47:31.314 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:47:31.314 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:47:31.314 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:47:31.314 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:47:31.314 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:47:31.328 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:47:31.329 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:47:31.334 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:47:31.334 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:47:31.334 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:47:31.334 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:47:31.335 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:47:31.335 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:47:32.305 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2192 2024-09-12T13:47:32.326 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:47:32.326 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:47:32.326 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:47:32.326 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:47:32.326 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:47:32.327 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:47:32.340 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:47:32.340 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:47:32.340 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:47:32.340 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:47:32.340 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:47:32.340 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:47:32.345 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:47:32.346 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2194 2024-09-12T13:47:32.373 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:47:33.279 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2195 2024-09-12T13:47:33.304 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:47:33.320 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:47:33.320 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:47:33.320 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:47:33.320 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:47:33.320 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:47:33.321 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:47:34.275 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2193 2024-09-12T13:47:34.294 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:47:34.294 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:47:34.294 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:47:34.294 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:47:34.294 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:47:34.294 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:47:34.299 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2196 2024-09-12T13:47:34.316 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2197 2024-09-12T13:47:34.327 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:47:34.330 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:47:34.331 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:47:34.331 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:47:34.331 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:47:34.331 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:47:34.331 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:47:34.332 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:47:34.332 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:47:34.332 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:47:34.332 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:47:34.332 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:47:34.332 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:47:34.333 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:47:34.347 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:47:35.332 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2199 2024-09-12T13:47:35.348 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:47:35.349 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:47:35.349 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:47:35.349 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:47:35.349 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:47:35.349 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:47:35.349 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:47:35.377 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:47:35.377 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:47:35.377 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:47:35.377 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:47:35.377 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:47:35.377 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:47:36.291 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2198 2024-09-12T13:47:36.308 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:47:36.309 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:47:36.309 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:47:36.309 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:47:36.309 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:47:36.309 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:47:36.312 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2200 2024-09-12T13:47:36.314 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2201 2024-09-12T13:47:36.339 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:47:36.363 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:47:36.363 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:47:36.372 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2203 2024-09-12T13:47:36.390 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:47:37.300 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2202 2024-09-12T13:47:37.323 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:47:37.331 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:47:37.331 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:47:37.331 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:47:37.331 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:47:37.331 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:47:37.331 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:47:37.335 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:47:37.336 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:47:37.336 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:47:37.336 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:47:37.336 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:47:37.336 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:47:37.350 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:47:37.351 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:47:37.351 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:47:37.351 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:47:37.351 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:47:37.351 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:47:38.263 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2204 2024-09-12T13:47:38.290 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2205 2024-09-12T13:47:38.296 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:47:38.311 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:47:38.351 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:47:38.351 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:47:38.351 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:47:38.351 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:47:38.351 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:47:38.351 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:47:39.316 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2206 2024-09-12T13:47:39.317 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2207 2024-09-12T13:47:39.342 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:47:39.344 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:47:39.344 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:47:39.344 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:47:39.345 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:47:39.345 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:47:39.345 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:47:39.354 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:47:39.367 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:47:39.367 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:47:39.368 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:47:39.368 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:47:39.368 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:47:39.368 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:47:39.368 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:47:39.368 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:47:39.368 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:47:39.368 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:47:39.368 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:47:39.368 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:47:39.393 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:47:39.393 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:47:39.393 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:47:39.393 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:47:39.393 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:47:39.393 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:47:40.327 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:47:40.327 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:47:40.327 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:47:40.327 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:47:40.328 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:47:40.328 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:47:40.341 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2209 2024-09-12T13:47:40.368 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2212 2024-09-12T13:47:40.380 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:47:40.388 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:47:41.295 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2208 2024-09-12T13:47:41.302 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:47:41.302 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:47:41.302 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:47:41.302 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:47:41.302 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:47:41.302 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:47:41.316 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:47:41.316 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:47:41.316 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:47:41.316 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:47:41.316 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:47:41.316 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:47:41.338 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2211 2024-09-12T13:47:41.341 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:47:41.349 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2210 2024-09-12T13:47:41.363 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:47:41.369 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:47:42.316 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2213 2024-09-12T13:47:42.344 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:47:42.347 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:47:42.347 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:47:42.347 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:47:42.347 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:47:42.348 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:47:42.348 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:47:42.358 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:47:42.358 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:47:42.358 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:47:42.358 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:47:42.358 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:47:42.359 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:47:43.284 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2215 2024-09-12T13:47:43.291 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2214 2024-09-12T13:47:43.313 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:47:43.314 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:47:43.384 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:47:43.385 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:47:43.385 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:47:43.385 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:47:43.385 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:47:43.385 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:47:43.391 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:47:43.391 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:47:43.391 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:47:43.391 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:47:43.391 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:47:43.391 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:47:44.341 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2217 2024-09-12T13:47:44.342 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2216 2024-09-12T13:47:44.349 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:47:44.349 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:47:44.349 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:47:44.350 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:47:44.350 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:47:44.350 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:47:44.368 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:47:44.368 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:47:44.368 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:47:44.368 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:47:44.368 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:47:44.368 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:47:44.371 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:47:44.372 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:47:44.373 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:47:44.373 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:47:44.373 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:47:44.373 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:47:44.373 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:47:44.373 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:47:44.386 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2219 2024-09-12T13:47:44.400 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:47:45.349 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:47:45.349 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:47:45.349 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:47:45.349 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:47:45.349 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:47:45.349 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:47:45.362 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2220 2024-09-12T13:47:45.375 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:47:46.318 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:47:46.318 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:47:46.318 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:47:46.318 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:47:46.318 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:47:46.318 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:47:46.319 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:47:46.319 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:47:46.319 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:47:46.319 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:47:46.319 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:47:46.319 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:47:46.334 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2218 2024-09-12T13:47:46.360 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2221 2024-09-12T13:47:46.382 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:47:46.386 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2222 2024-09-12T13:47:46.404 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:47:46.420 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:47:47.348 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2223 2024-09-12T13:47:47.375 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:47:47.375 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:47:47.376 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:47:47.376 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:47:47.376 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:47:47.376 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:47:47.376 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:47:47.376 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:47:47.377 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:47:47.377 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:47:47.377 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:47:47.377 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:47:47.391 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:47:47.409 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:47:47.410 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:47:47.410 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:47:47.410 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:47:47.410 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:47:47.410 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:47:48.296 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2225 2024-09-12T13:47:48.304 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2224 2024-09-12T13:47:48.333 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:47:48.352 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:47:48.378 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:47:48.378 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:47:48.378 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:47:48.378 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:47:48.378 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:47:48.378 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:47:49.325 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2228 2024-09-12T13:47:49.370 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2226 2024-09-12T13:47:49.383 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:47:49.386 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:47:49.387 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:47:49.387 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:47:49.387 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:47:49.387 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:47:49.387 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:47:49.413 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:47:49.413 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:47:49.413 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:47:49.413 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:47:49.413 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:47:49.413 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:47:49.425 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:47:49.427 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:47:49.427 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:47:49.427 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:47:49.427 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:47:49.427 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:47:49.432 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:47:50.313 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2227 2024-09-12T13:47:50.354 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:47:50.390 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2230 2024-09-12T13:47:50.396 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:47:50.396 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:47:50.396 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:47:50.396 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:47:50.396 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:47:50.396 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:47:50.410 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:47:51.337 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:47:51.338 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:47:51.338 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:47:51.338 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:47:51.338 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:47:51.338 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:47:51.341 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2229 2024-09-12T13:47:51.356 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:47:51.356 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:47:51.356 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:47:51.356 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:47:51.357 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:47:51.357 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:47:51.375 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:47:51.375+0000 7f67dc600640 -1 rbd::mirror::image_replayer::snapshot::Replayer: 0x560556cd3400 handle_unlink_peer: failed to unlink local peer from remote image: (22) Invalid argument 2024-09-12T13:47:51.386 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:47:52.334 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2231 2024-09-12T13:47:52.339 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2232 2024-09-12T13:47:52.384 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:47:52.387 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:47:52.387 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:47:52.388 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:47:52.388 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:47:52.388 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:47:52.388 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:47:52.393 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2233 2024-09-12T13:47:52.403 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:47:52.423 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:47:52.435 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:47:52.435 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:47:52.435 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:47:52.435 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:47:52.435 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:47:52.436 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:47:53.279 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2235 2024-09-12T13:47:53.280 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2234 2024-09-12T13:47:53.300 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:47:53.300 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:47:53.357 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:47:53.357 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:47:53.357 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:47:53.357 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:47:53.357 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:47:53.357 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:47:53.413 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:47:53.413 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:47:53.413 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:47:53.414 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:47:53.414 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:47:53.414 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:47:54.391 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:47:54.391 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:47:54.391 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:47:54.391 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:47:54.391 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:47:54.391 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:47:54.412 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2238 2024-09-12T13:47:54.441 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:47:55.351 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2237 2024-09-12T13:47:55.387 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:47:55.388 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:47:55.388 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:47:55.388 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:47:55.388 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:47:55.388 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:47:55.388 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:47:55.407 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:47:55.407 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:47:55.407 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:47:55.407 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:47:55.407 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:47:55.407 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:47:55.420 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2240 2024-09-12T13:47:55.426 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:47:55.426 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:47:55.426 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:47:55.426 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:47:55.426 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:47:55.426 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:47:55.437 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:47:56.303 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:47:56.303 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:47:56.303 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:47:56.303 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:47:56.303 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:47:56.304 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:47:56.304 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:47:56.304 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:47:56.304 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:47:56.304 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:47:56.304 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:47:56.304 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:47:56.355 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2239 2024-09-12T13:47:56.404 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:47:56.433 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2241 2024-09-12T13:47:56.438 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2242 2024-09-12T13:47:56.452 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:47:56.457 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:47:57.443 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:47:57.443 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:47:57.443 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:47:57.444 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:47:57.444 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:47:57.444 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:47:58.251 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2236 2024-09-12T13:47:58.270 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2243 2024-09-12T13:47:58.304 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2245 2024-09-12T13:47:58.309 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:47:58.316 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2244 2024-09-12T13:47:58.319 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:47:58.339 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:47:58.347 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:47:58.390 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:47:58.390 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:47:58.390 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:47:58.390 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:47:58.390 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:47:58.390 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:47:58.441 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:47:58.441 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:47:58.441 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:47:58.441 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:47:58.442 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:47:58.442 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:47:59.392 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2247 2024-09-12T13:47:59.407 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:47:59.407 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:47:59.407 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:47:59.407 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:47:59.407 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:47:59.407 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:47:59.409 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:47:59.455 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:47:59.455 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:47:59.455 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:47:59.455 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:47:59.455 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:47:59.456 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:47:59.461 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:47:59.461 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:47:59.461 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:47:59.461 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:47:59.462 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:47:59.462 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:48:00.363 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2246 2024-09-12T13:48:00.394 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:48:00.408 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2250 2024-09-12T13:48:00.413 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2249 2024-09-12T13:48:00.425 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:48:00.429 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:48:01.313 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:48:01.313 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:48:01.313 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:48:01.313 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:48:01.313 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:48:01.313 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:48:01.322 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:48:01.322 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:48:01.322 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:48:01.322 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:48:01.322 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:48:01.322 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:48:01.342 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:48:01.342 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:48:01.342 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:48:01.342 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:48:01.342 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:48:01.342 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:48:01.350 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:48:01.350 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:48:01.350 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:48:01.350 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:48:01.350 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:48:01.351 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:48:01.365 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2248 2024-09-12T13:48:01.394 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:48:02.372 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2251 2024-09-12T13:48:02.402 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:48:02.411 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:48:02.411 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:48:02.411 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:48:02.411 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:48:02.411 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:48:02.411 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:48:02.428 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2253 2024-09-12T13:48:02.443 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2252 2024-09-12T13:48:02.450 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:48:02.461 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:48:03.295 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2254 2024-09-12T13:48:03.295 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2255 2024-09-12T13:48:03.316 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:48:03.317 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:48:03.397 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:48:03.397 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:48:03.397 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:48:03.397 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:48:03.397 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:48:03.398 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:48:03.428 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:48:03.428 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:48:03.428 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:48:03.428 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:48:03.429 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:48:03.429 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:48:03.431 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:48:03.431 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:48:03.431 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:48:03.431 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:48:03.431 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:48:03.431 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:48:04.267 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2256 2024-09-12T13:48:04.292 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2257 2024-09-12T13:48:04.299 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:48:04.305 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2258 2024-09-12T13:48:04.320 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:48:04.333 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:48:04.397 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:48:04.397 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:48:04.397 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:48:04.398 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:48:04.398 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:48:04.398 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:48:05.270 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2259 2024-09-12T13:48:05.290 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2260 2024-09-12T13:48:05.292 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:48:05.306 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:48:05.405 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:48:05.405 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:48:05.405 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:48:05.405 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:48:05.405 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:48:05.405 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:48:05.453 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:48:05.454 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:48:05.454 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:48:05.454 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:48:05.454 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:48:05.454 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:48:05.464 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:48:05.464 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:48:05.464 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:48:05.464 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:48:05.464 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:48:05.464 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:48:06.319 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:48:06.319 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:48:06.319 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:48:06.319 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:48:06.319 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:48:06.319 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:48:06.320 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:48:06.320 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:48:06.320 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:48:06.320 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:48:06.320 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:48:06.320 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:48:06.403 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2261 2024-09-12T13:48:06.419 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:48:07.303 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:48:07.303 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:48:07.303 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:48:07.303 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:48:07.303 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:48:07.303 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:48:07.323 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:48:07.323 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:48:07.323 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:48:07.323 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:48:07.323 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:48:07.323 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:48:07.336 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:48:07.336 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:48:07.336 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:48:07.336 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:48:07.336 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:48:07.336 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:48:08.244 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2262 2024-09-12T13:48:08.275 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2263 2024-09-12T13:48:08.296 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:48:08.296 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:48:08.296 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:48:08.296 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:48:08.296 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:48:08.296 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:48:08.301 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:48:08.306 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2264 2024-09-12T13:48:08.309 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:48:08.309 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:48:08.309 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:48:08.309 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:48:08.309 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:48:08.309 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:48:08.330 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:48:08.331 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2265 2024-09-12T13:48:08.352 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:48:08.394 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2268 2024-09-12T13:48:08.402 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:48:08.412 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:48:09.422 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:48:09.422 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:48:09.422 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:48:09.422 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:48:09.422 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:48:09.423 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:48:10.385 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2266 2024-09-12T13:48:10.392 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2267 2024-09-12T13:48:10.429 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:48:10.449 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:48:11.305 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:48:11.305 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:48:11.305 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:48:11.305 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:48:11.305 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:48:11.305 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:48:11.335 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:48:11.335 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:48:11.335 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:48:11.335 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:48:11.335 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:48:11.336 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:48:11.356 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:48:11.356 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:48:11.356 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:48:11.356 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:48:11.356 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:48:11.356 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:48:11.373 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2270 2024-09-12T13:48:11.397 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2269 2024-09-12T13:48:11.406 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:48:11.406 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:48:11.406 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:48:11.407 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:48:11.407 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:48:11.407 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:48:11.415 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:48:11.415 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:48:11.415 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:48:11.415 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:48:11.416 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:48:11.416 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:48:11.423 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:48:11.436 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:48:11.436+0000 7f67dd000640 -1 rbd::mirror::image_replayer::snapshot::Replayer: 0x56055c2b2000 handle_unlink_peer: failed to unlink local peer from remote image: (22) Invalid argument 2024-09-12T13:48:11.451 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:48:12.391 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2271 2024-09-12T13:48:12.418 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:48:12.449 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2272 2024-09-12T13:48:12.451 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2273 2024-09-12T13:48:12.467 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:48:12.470 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:48:13.254 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2276 2024-09-12T13:48:13.287 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2275 2024-09-12T13:48:13.290 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2274 2024-09-12T13:48:13.308 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:48:13.336 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:48:13.340 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:48:13.435 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:48:13.435 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:48:13.436 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:48:13.436 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:48:13.436 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:48:13.436 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:48:13.453 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:48:13.453 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:48:13.453 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:48:13.453 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:48:13.453 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:48:13.453 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:48:14.412 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2277 2024-09-12T13:48:14.414 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2278 2024-09-12T13:48:14.425 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:48:14.426 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:48:14.426 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:48:14.426 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:48:14.426 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:48:14.426 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:48:14.429 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:48:14.429 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:48:14.455 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:48:14.455 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:48:14.455 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:48:14.455 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:48:14.455 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:48:14.455 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:48:15.421 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:48:15.421 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:48:15.422 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:48:15.422 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:48:15.422 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:48:15.422 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:48:15.422 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2280 2024-09-12T13:48:15.424 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2279 2024-09-12T13:48:15.438 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:48:15.440 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:48:15.470 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:48:15.470 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:48:15.470 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:48:15.470 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:48:15.470 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:48:15.470 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:48:15.473 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:48:15.473 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:48:15.473 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:48:15.473 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:48:15.473 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:48:15.473 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:48:16.336 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:48:16.337 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:48:16.337 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:48:16.337 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:48:16.337 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:48:16.337 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:48:16.343 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:48:16.344 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:48:16.344 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:48:16.344 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:48:16.344 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:48:16.344 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:48:16.434 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2281 2024-09-12T13:48:16.452 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:48:17.431 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:48:17.432 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:48:17.432 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:48:17.432 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:48:17.432 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:48:17.432 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:48:17.432 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:48:17.432 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:48:17.432 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:48:17.432 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:48:17.432 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:48:17.432 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:48:18.255 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2282 2024-09-12T13:48:18.258 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2283 2024-09-12T13:48:18.312 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:48:18.314 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:48:18.317 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2284 2024-09-12T13:48:18.322 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2285 2024-09-12T13:48:18.343 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:48:18.350 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:48:18.442 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${SNAP_PIDS[@]} 2024-09-12T13:48:18.442 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 106534 2024-09-12T13:48:18.442 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${SNAP_PIDS[@]} 2024-09-12T13:48:18.442 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 106666 2024-09-12T13:48:18.443 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:48:18.443 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:48:18.443 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:48:18.443 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:48:18.443 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:48:18.443 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:48:19.455 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:48:19.455 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:48:19.455 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:48:19.455 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:48:19.455 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:48:19.455 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:48:20.408 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2286 2024-09-12T13:48:20.421 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2287 2024-09-12T13:48:20.450 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:48:20.464 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:48:21.316 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:48:21.316 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:48:21.316 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:48:21.316 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:48:21.316 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:48:21.316 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:48:21.317 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${SNAP_PIDS[@]} 2024-09-12T13:48:21.317 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 106798 2024-09-12T13:48:21.346 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${SNAP_PIDS[@]} 2024-09-12T13:48:21.346 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 106950 2024-09-12T13:48:21.353 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:48:21.353 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:48:21.353 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:48:21.353 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:48:21.354 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:48:21.354 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:48:21.419 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2288 2024-09-12T13:48:21.450 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:48:22.417 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2289 2024-09-12T13:48:22.437 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:48:22.458 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2290 2024-09-12T13:48:22.473 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:48:23.453 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:48:23.453 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:48:23.453 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:48:23.453 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:48:23.453 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:48:23.453 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:48:23.467 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${SNAP_PIDS[@]} 2024-09-12T13:48:23.467 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 107103 2024-09-12T13:48:24.254 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2291 2024-09-12T13:48:24.278 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:48:24.305 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2292 2024-09-12T13:48:24.315 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:48:24.453 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${SNAP_PIDS[@]} 2024-09-12T13:48:24.453 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 107256 2024-09-12T13:48:25.440 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${SNAP_PIDS[@]} 2024-09-12T13:48:25.441 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 107409 2024-09-12T13:48:25.476 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${SNAP_PIDS[@]} 2024-09-12T13:48:25.476 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 107582 2024-09-12T13:48:27.282 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${SNAP_PIDS[@]} 2024-09-12T13:48:27.283 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 107756 2024-09-12T13:48:27.318 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ (( RET != 0 )) 2024-09-12T13:48:27.318 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${WORKLOAD_PIDS[@]} 2024-09-12T13:48:27.318 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 106403 2024-09-12T13:50:54.076 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ ret=124 2024-09-12T13:50:54.076 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ (( ret != 124 )) 2024-09-12T13:50:54.077 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${WORKLOAD_PIDS[@]} 2024-09-12T13:50:54.077 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 106535 2024-09-12T13:50:54.909 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ ret=124 2024-09-12T13:50:54.909 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ (( ret != 124 )) 2024-09-12T13:50:54.910 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${WORKLOAD_PIDS[@]} 2024-09-12T13:50:54.910 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 106667 2024-09-12T13:50:55.927 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ ret=124 2024-09-12T13:50:55.927 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ (( ret != 124 )) 2024-09-12T13:50:55.928 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${WORKLOAD_PIDS[@]} 2024-09-12T13:50:55.928 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 106799 2024-09-12T13:50:56.936 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ ret=124 2024-09-12T13:50:56.936 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ (( ret != 124 )) 2024-09-12T13:50:56.936 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${WORKLOAD_PIDS[@]} 2024-09-12T13:50:56.936 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 106951 2024-09-12T13:50:58.137 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ ret=124 2024-09-12T13:50:58.138 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ (( ret != 124 )) 2024-09-12T13:50:58.138 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${WORKLOAD_PIDS[@]} 2024-09-12T13:50:58.138 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 107104 2024-09-12T13:50:59.390 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ ret=124 2024-09-12T13:50:59.391 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ (( ret != 124 )) 2024-09-12T13:50:59.391 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${WORKLOAD_PIDS[@]} 2024-09-12T13:50:59.391 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 107257 2024-09-12T13:51:00.631 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ ret=124 2024-09-12T13:51:00.632 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ (( ret != 124 )) 2024-09-12T13:51:00.632 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${WORKLOAD_PIDS[@]} 2024-09-12T13:51:00.632 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 107410 2024-09-12T13:51:01.021 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ ret=124 2024-09-12T13:51:01.021 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ (( ret != 124 )) 2024-09-12T13:51:01.021 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${WORKLOAD_PIDS[@]} 2024-09-12T13:51:01.021 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 107583 2024-09-12T13:51:02.005 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ ret=124 2024-09-12T13:51:02.005 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ (( ret != 124 )) 2024-09-12T13:51:02.005 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${WORKLOAD_PIDS[@]} 2024-09-12T13:51:02.005 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 107757 2024-09-12T13:51:03.549 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ ret=124 2024-09-12T13:51:03.549 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ (( ret != 124 )) 2024-09-12T13:51:03.549 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ (( RET != 0 )) 2024-09-12T13:51:03.550 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:51:03.550 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ COMPARE_PIDS+=($!) 2024-09-12T13:51:03.550 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:51:03.550 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ compare_demoted_promoted_image 1 2024-09-12T13:51:03.552 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local dev=/dev/rbd0 2024-09-12T13:51:03.552 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ COMPARE_PIDS+=($!) 2024-09-12T13:51:03.552 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local img=image-primary1 2024-09-12T13:51:03.553 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mntpt=test-primary1 2024-09-12T13:51:03.553 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local demote_md5 promote_md5 2024-09-12T13:51:03.553 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo umount test-primary1 2024-09-12T13:51:03.553 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:51:03.553 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ compare_demoted_promoted_image 2 2024-09-12T13:51:03.553 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local dev=/dev/rbd1 2024-09-12T13:51:03.553 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local img=image-primary2 2024-09-12T13:51:03.553 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mntpt=test-primary2 2024-09-12T13:51:03.553 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local demote_md5 promote_md5 2024-09-12T13:51:03.553 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo umount test-primary2 2024-09-12T13:51:03.553 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ COMPARE_PIDS+=($!) 2024-09-12T13:51:03.553 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:51:03.553 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ compare_demoted_promoted_image 3 2024-09-12T13:51:03.553 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local dev=/dev/rbd2 2024-09-12T13:51:03.554 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local img=image-primary3 2024-09-12T13:51:03.554 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mntpt=test-primary3 2024-09-12T13:51:03.554 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local demote_md5 promote_md5 2024-09-12T13:51:03.554 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo umount test-primary3 2024-09-12T13:51:03.554 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ COMPARE_PIDS+=($!) 2024-09-12T13:51:03.554 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:51:03.554 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ compare_demoted_promoted_image 4 2024-09-12T13:51:03.554 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local dev=/dev/rbd3 2024-09-12T13:51:03.554 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local img=image-primary4 2024-09-12T13:51:03.554 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mntpt=test-primary4 2024-09-12T13:51:03.554 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local demote_md5 promote_md5 2024-09-12T13:51:03.554 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo umount test-primary4 2024-09-12T13:51:03.554 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ COMPARE_PIDS+=($!) 2024-09-12T13:51:03.555 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:51:03.555 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ compare_demoted_promoted_image 5 2024-09-12T13:51:03.556 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local dev=/dev/rbd4 2024-09-12T13:51:03.556 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local img=image-primary5 2024-09-12T13:51:03.556 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mntpt=test-primary5 2024-09-12T13:51:03.556 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local demote_md5 promote_md5 2024-09-12T13:51:03.557 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo umount test-primary5 2024-09-12T13:51:03.557 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ COMPARE_PIDS+=($!) 2024-09-12T13:51:03.557 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:51:03.557 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ compare_demoted_promoted_image 6 2024-09-12T13:51:03.557 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local dev=/dev/rbd5 2024-09-12T13:51:03.557 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local img=image-primary6 2024-09-12T13:51:03.557 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mntpt=test-primary6 2024-09-12T13:51:03.557 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local demote_md5 promote_md5 2024-09-12T13:51:03.557 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo umount test-primary6 2024-09-12T13:51:03.557 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ COMPARE_PIDS+=($!) 2024-09-12T13:51:03.557 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:51:03.557 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ compare_demoted_promoted_image 7 2024-09-12T13:51:03.557 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local dev=/dev/rbd6 2024-09-12T13:51:03.557 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local img=image-primary7 2024-09-12T13:51:03.558 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mntpt=test-primary7 2024-09-12T13:51:03.558 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local demote_md5 promote_md5 2024-09-12T13:51:03.558 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo umount test-primary7 2024-09-12T13:51:03.558 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ COMPARE_PIDS+=($!) 2024-09-12T13:51:03.558 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ compare_demoted_promoted_image 8 2024-09-12T13:51:03.558 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local dev=/dev/rbd7 2024-09-12T13:51:03.558 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local img=image-primary8 2024-09-12T13:51:03.559 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mntpt=test-primary8 2024-09-12T13:51:03.559 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local demote_md5 promote_md5 2024-09-12T13:51:03.559 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo umount test-primary8 2024-09-12T13:51:03.559 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:51:03.559 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ COMPARE_PIDS+=($!) 2024-09-12T13:51:03.559 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:51:03.559 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ COMPARE_PIDS+=($!) 2024-09-12T13:51:03.559 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${COMPARE_PIDS[@]} 2024-09-12T13:51:03.559 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 113711 2024-09-12T13:51:03.560 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ compare_demoted_promoted_image 9 2024-09-12T13:51:03.560 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local dev=/dev/rbd8 2024-09-12T13:51:03.560 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local img=image-primary9 2024-09-12T13:51:03.560 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mntpt=test-primary9 2024-09-12T13:51:03.560 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local demote_md5 promote_md5 2024-09-12T13:51:03.560 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ compare_demoted_promoted_image 10 2024-09-12T13:51:03.560 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo umount test-primary9 2024-09-12T13:51:03.561 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local dev=/dev/rbd9 2024-09-12T13:51:03.561 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local img=image-primary10 2024-09-12T13:51:03.561 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mntpt=test-primary10 2024-09-12T13:51:03.561 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local demote_md5 promote_md5 2024-09-12T13:51:03.561 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo umount test-primary10 2024-09-12T13:51:04.231 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ awk '{print $1}' 2024-09-12T13:51:04.231 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo md5sum /dev/rbd1 2024-09-12T13:51:04.381 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ awk '{print $1}' 2024-09-12T13:51:04.382 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo md5sum /dev/rbd5 2024-09-12T13:51:04.414 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ awk '{print $1}' 2024-09-12T13:51:04.414 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo md5sum /dev/rbd7 2024-09-12T13:51:04.445 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ awk '{print $1}' 2024-09-12T13:51:04.461 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo md5sum /dev/rbd4 2024-09-12T13:51:04.484 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo md5sum /dev/rbd8 2024-09-12T13:51:04.487 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ awk '{print $1}' 2024-09-12T13:51:04.497 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo md5sum /dev/rbd6 2024-09-12T13:51:04.497 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ awk '{print $1}' 2024-09-12T13:51:04.528 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo md5sum /dev/rbd2 2024-09-12T13:51:04.530 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo md5sum /dev/rbd3 2024-09-12T13:51:04.532 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ awk '{print $1}' 2024-09-12T13:51:04.532 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ awk '{print $1}' 2024-09-12T13:51:04.547 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo md5sum /dev/rbd0 2024-09-12T13:51:04.547 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ awk '{print $1}' 2024-09-12T13:51:04.570 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ awk '{print $1}' 2024-09-12T13:51:04.571 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo md5sum /dev/rbd9 2024-09-12T13:51:57.773 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ demote_md5=901ce58fda4db04b03205a885e099dc3 2024-09-12T13:51:57.774 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo rbd --cluster cluster1 device unmap -t krbd mirror/image-primary6 2024-09-12T13:51:57.817 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ demote_md5=1c914606982714846b5a2e92428803c2 2024-09-12T13:51:57.817 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo rbd --cluster cluster1 device unmap -t krbd mirror/image-primary7 2024-09-12T13:51:57.989 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ demote_md5=25356f5bc6839635dcb6e534f32366c6 2024-09-12T13:51:57.989 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo rbd --cluster cluster1 device unmap -t krbd mirror/image-primary1 2024-09-12T13:51:58.130 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ demote_md5=2c7fd5759f9be57aa5c65afcbc50f7a7 2024-09-12T13:51:58.130 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo rbd --cluster cluster1 device unmap -t krbd mirror/image-primary10 2024-09-12T13:51:58.394 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ demote_image cluster1 mirror image-primary1 2024-09-12T13:51:58.394 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:51:58.394 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:51:58.394 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:51:58.394 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 mirror image demote mirror/image-primary1 2024-09-12T13:51:58.396 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ demote_image cluster1 mirror image-primary6 2024-09-12T13:51:58.396 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:51:58.397 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:51:58.397 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:51:58.397 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 mirror image demote mirror/image-primary6 2024-09-12T13:51:58.408 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ demote_image cluster1 mirror image-primary10 2024-09-12T13:51:58.408 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:51:58.408 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:51:58.408 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:51:58.408 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 mirror image demote mirror/image-primary10 2024-09-12T13:51:58.419 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ demote_image cluster1 mirror image-primary7 2024-09-12T13:51:58.419 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:51:58.419 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:51:58.419 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:51:58.419 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 mirror image demote mirror/image-primary7 2024-09-12T13:51:59.139 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ demote_md5=ba8af2f22e67cc1ddd71ae354b21df71 2024-09-12T13:51:59.139 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo rbd --cluster cluster1 device unmap -t krbd mirror/image-primary9 2024-09-12T13:51:59.168 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ demote_md5=dd6e340287a75aa57b1278a1d1583ce1 2024-09-12T13:51:59.168 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo rbd --cluster cluster1 device unmap -t krbd mirror/image-primary5 2024-09-12T13:51:59.388 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Image demoted to non-primary 2024-09-12T13:51:59.389 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Image demoted to non-primary 2024-09-12T13:51:59.404 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary7 up+unknown 2024-09-12T13:51:59.404 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:51:59.404 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:51:59.404 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:51:59.404 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:51:59.404 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:51:59.404 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:51:59.405 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:51:59.405 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:51:59.406 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary6 up+unknown 2024-09-12T13:51:59.406 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:51:59.406 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:51:59.406 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:51:59.406 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:51:59.406 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:51:59.406 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:51:59.406 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:51:59.406 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:51:59.436 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ demote_image cluster1 mirror image-primary9 2024-09-12T13:51:59.437 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:51:59.437 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:51:59.437 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:51:59.437 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 mirror image demote mirror/image-primary9 2024-09-12T13:51:59.521 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ demote_image cluster1 mirror image-primary5 2024-09-12T13:51:59.521 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:51:59.521 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:51:59.521 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:51:59.521 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 mirror image demote mirror/image-primary5 2024-09-12T13:52:00.391 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ demote_md5=7c7d2fca48d3311a4a8196f927c7b885 2024-09-12T13:52:00.391 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo rbd --cluster cluster1 device unmap -t krbd mirror/image-primary3 2024-09-12T13:52:00.419 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary6 up+unknown '' '' 2024-09-12T13:52:00.419 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:52:00.419 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:52:00.419 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:52:00.419 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:52:00.419 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:52:00.419 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:52:00.419 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:52:00.421 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary7 up+unknown '' '' 2024-09-12T13:52:00.421 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:52:00.421 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:52:00.422 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:52:00.422 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:52:00.422 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:52:00.423 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:52:00.423 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:52:00.425 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:52:00.425 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary7 2024-09-12T13:52:00.427 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:52:00.428 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary6 2024-09-12T13:52:00.444 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Image demoted to non-primary 2024-09-12T13:52:00.445 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Image demoted to non-primary 2024-09-12T13:52:00.457 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Image demoted to non-primary 2024-09-12T13:52:00.462 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Image demoted to non-primary 2024-09-12T13:52:00.532 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary10 up+unknown 2024-09-12T13:52:00.532 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:52:00.532 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:52:00.532 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:52:00.532 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:52:00.533 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:52:00.533 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:52:00.533 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:52:00.533 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:52:00.535 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary1 up+unknown 2024-09-12T13:52:00.537 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:52:00.537 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:52:00.537 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:52:00.537 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:52:00.538 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:52:00.538 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:52:00.538 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:52:00.538 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:52:00.546 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary9 up+unknown 2024-09-12T13:52:00.546 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:52:00.546 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:52:00.546 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:52:00.546 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:52:00.546 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:52:00.547 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:52:00.547 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:52:00.547 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:52:00.563 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary5 up+unknown 2024-09-12T13:52:00.563 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:52:00.565 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:52:00.565 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:52:00.565 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:52:00.565 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:52:00.565 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:52:00.565 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:52:00.565 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:52:00.749 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ demote_image cluster1 mirror image-primary3 2024-09-12T13:52:00.749 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:52:00.749 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:52:00.749 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:52:00.750 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 mirror image demote mirror/image-primary3 2024-09-12T13:52:00.750 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary7: 2024-09-12T13:52:00.750 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: ecec213c-dc11-4547-88c6-ded8df679ff6 2024-09-12T13:52:00.750 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:52:00.750 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:52:00.750 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:52:00.751 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:51:47' 2024-09-12T13:52:00.751 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:52:00.769 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:52:00.772 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:52:00.772 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 2 2024-09-12T13:52:00.799 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary6: 2024-09-12T13:52:00.799 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 88271fa2-1e10-4868-befd-15807c269565 2024-09-12T13:52:00.799 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:52:00.799 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:52:00.800 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:52:00.800 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:51:47' 2024-09-12T13:52:00.800 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:52:00.809 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:52:00.810 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:52:00.810 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 2 2024-09-12T13:52:01.219 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ demote_md5=92b616a2adf309704fb48b12236529cf 2024-09-12T13:52:01.220 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo rbd --cluster cluster1 device unmap -t krbd mirror/image-primary4 2024-09-12T13:52:01.329 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ demote_md5=3e247a1608ce7b437a1d237414b07013 2024-09-12T13:52:01.329 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo rbd --cluster cluster1 device unmap -t krbd mirror/image-primary8 2024-09-12T13:52:01.333 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ demote_md5=04830618d390ed33d3109d44602a6361 2024-09-12T13:52:01.333 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo rbd --cluster cluster1 device unmap -t krbd mirror/image-primary2 2024-09-12T13:52:01.366 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ demote_image cluster1 mirror image-primary4 2024-09-12T13:52:01.366 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:52:01.366 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:52:01.366 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:52:01.366 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 mirror image demote mirror/image-primary4 2024-09-12T13:52:01.381 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Image demoted to non-primary 2024-09-12T13:52:01.391 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary3 up+unknown 2024-09-12T13:52:01.391 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:52:01.391 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:52:01.391 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:52:01.392 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:52:01.392 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:52:01.392 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:52:01.392 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:52:01.392 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:52:01.431 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ demote_image cluster1 mirror image-primary2 2024-09-12T13:52:01.431 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:52:01.431 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:52:01.431 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:52:01.431 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 mirror image demote mirror/image-primary2 2024-09-12T13:52:01.431 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ demote_image cluster1 mirror image-primary8 2024-09-12T13:52:01.431 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:52:01.431 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:52:01.432 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:52:01.432 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 mirror image demote mirror/image-primary8 2024-09-12T13:52:01.551 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary10 up+unknown '' '' 2024-09-12T13:52:01.552 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:52:01.552 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:52:01.552 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:52:01.552 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:52:01.552 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:52:01.552 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:52:01.552 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:52:01.552 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary1 up+unknown '' '' 2024-09-12T13:52:01.552 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:52:01.552 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:52:01.552 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:52:01.552 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:52:01.552 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:52:01.553 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:52:01.553 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:52:01.553 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:52:01.553 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary10 2024-09-12T13:52:01.553 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:52:01.553 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary1 2024-09-12T13:52:01.565 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary9 up+unknown '' '' 2024-09-12T13:52:01.565 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:52:01.565 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:52:01.565 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:52:01.566 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:52:01.566 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:52:01.566 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:52:01.566 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:52:01.566 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:52:01.566 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary9 2024-09-12T13:52:01.569 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary5 up+unknown '' '' 2024-09-12T13:52:01.569 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:52:01.569 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:52:01.569 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:52:01.570 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:52:01.570 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:52:01.570 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:52:01.570 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:52:01.570 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:52:01.570 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary5 2024-09-12T13:52:01.643 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary1: 2024-09-12T13:52:01.644 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: f6ac15bd-c3ff-4c34-86f6-e9bbc701f22f 2024-09-12T13:52:01.644 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:52:01.644 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:52:01.644 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:52:01.644 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:51:47' 2024-09-12T13:52:01.644 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:52:01.666 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:52:01.666 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:52:01.666 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 2 2024-09-12T13:52:01.681 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary10: 2024-09-12T13:52:01.681 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 02633bcb-38b0-4c95-9ca5-a988ab7c0dd0 2024-09-12T13:52:01.681 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:52:01.681 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:52:01.681 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:52:01.681 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:51:47' 2024-09-12T13:52:01.681 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:52:01.692 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary5: 2024-09-12T13:52:01.692 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 6de1d859-a420-4abe-953c-031047e292ec 2024-09-12T13:52:01.692 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:52:01.693 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:52:01.693 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:52:01.693 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:51:47' 2024-09-12T13:52:01.693 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:52:01.693 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:52:01.693 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:52:01.693 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 2 2024-09-12T13:52:01.698 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:52:01.698 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:52:01.699 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 2 2024-09-12T13:52:01.723 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary9: 2024-09-12T13:52:01.723 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 43c84ab2-a21e-441e-82d9-32f1b27926a0 2024-09-12T13:52:01.723 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:52:01.723 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:52:01.724 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:52:01.724 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:51:47' 2024-09-12T13:52:01.724 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:52:01.732 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:52:01.732 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:52:01.732 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 2 2024-09-12T13:52:02.394 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary3 up+unknown '' '' 2024-09-12T13:52:02.394 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:52:02.394 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:52:02.395 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:52:02.395 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:52:02.395 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:52:02.395 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:52:02.395 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:52:02.395 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:52:02.395 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary3 2024-09-12T13:52:02.467 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary3: 2024-09-12T13:52:02.467 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 73afcdb2-236d-496e-9163-3c6ff6e95e23 2024-09-12T13:52:02.468 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:52:02.468 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:52:02.468 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:52:02.468 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:51:47' 2024-09-12T13:52:02.468 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:52:02.470 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:52:02.470 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:52:02.470 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 2 2024-09-12T13:52:02.785 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary7 up+unknown '' '' 2024-09-12T13:52:02.785 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:52:02.785 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:52:02.785 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:52:02.785 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:52:02.785 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:52:02.785 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:52:02.785 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:52:02.786 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:52:02.786 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary7 2024-09-12T13:52:02.828 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary6 up+unknown '' '' 2024-09-12T13:52:02.828 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:52:02.828 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:52:02.828 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:52:02.828 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:52:02.829 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:52:02.829 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:52:02.829 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:52:02.829 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:52:02.829 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary6 2024-09-12T13:52:02.988 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary7: 2024-09-12T13:52:02.988 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: ecec213c-dc11-4547-88c6-ded8df679ff6 2024-09-12T13:52:02.988 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:52:02.988 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:52:02.988 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:52:02.988 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:51:47' 2024-09-12T13:52:02.989 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:52:02.993 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:52:02.993 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:52:02.993 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 4 2024-09-12T13:52:03.007 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary6: 2024-09-12T13:52:03.007 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 88271fa2-1e10-4868-befd-15807c269565 2024-09-12T13:52:03.008 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:52:03.008 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:52:03.008 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:52:03.008 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:51:47' 2024-09-12T13:52:03.008 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:52:03.015 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:52:03.016 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:52:03.016 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 4 2024-09-12T13:52:03.390 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Image demoted to non-primary 2024-09-12T13:52:03.411 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary4 up+unknown 2024-09-12T13:52:03.411 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:52:03.411 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:52:03.411 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:52:03.412 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:52:03.412 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:52:03.412 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:52:03.412 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:52:03.412 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:52:03.412 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Image demoted to non-primary 2024-09-12T13:52:03.427 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary8 up+unknown 2024-09-12T13:52:03.427 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:52:03.427 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:52:03.428 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:52:03.428 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:52:03.428 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:52:03.428 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:52:03.428 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:52:03.428 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:52:03.672 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary1 up+unknown '' '' 2024-09-12T13:52:03.672 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:52:03.672 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:52:03.673 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:52:03.673 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:52:03.673 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:52:03.673 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:52:03.673 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:52:03.673 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:52:03.673 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary1 2024-09-12T13:52:03.698 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary10 up+unknown '' '' 2024-09-12T13:52:03.698 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:52:03.698 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:52:03.698 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:52:03.698 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:52:03.698 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:52:03.698 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:52:03.698 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:52:03.700 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:52:03.700 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary10 2024-09-12T13:52:03.704 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary5 up+unknown '' '' 2024-09-12T13:52:03.705 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:52:03.705 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:52:03.705 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:52:03.705 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:52:03.705 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:52:03.705 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:52:03.705 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:52:03.706 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:52:03.706 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary5 2024-09-12T13:52:03.735 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary9 up+unknown '' '' 2024-09-12T13:52:03.736 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:52:03.736 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:52:03.736 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:52:03.736 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:52:03.736 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:52:03.736 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:52:03.736 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:52:03.737 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:52:03.737 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary9 2024-09-12T13:52:03.762 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary1: 2024-09-12T13:52:03.762 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: f6ac15bd-c3ff-4c34-86f6-e9bbc701f22f 2024-09-12T13:52:03.762 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:52:03.762 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:52:03.762 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:52:03.763 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:51:47' 2024-09-12T13:52:03.763 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:52:03.765 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:52:03.765 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:52:03.765 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 4 2024-09-12T13:52:03.781 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary10: 2024-09-12T13:52:03.781 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 02633bcb-38b0-4c95-9ca5-a988ab7c0dd0 2024-09-12T13:52:03.781 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:52:03.781 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:52:03.781 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:52:03.781 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:51:47' 2024-09-12T13:52:03.781 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:52:03.783 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:52:03.784 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:52:03.784 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 4 2024-09-12T13:52:03.786 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary5: 2024-09-12T13:52:03.786 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 6de1d859-a420-4abe-953c-031047e292ec 2024-09-12T13:52:03.787 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:52:03.787 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:52:03.787 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:52:03.787 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:51:47' 2024-09-12T13:52:03.787 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:52:03.789 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:52:03.789 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:52:03.789 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 4 2024-09-12T13:52:03.810 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary9: 2024-09-12T13:52:03.810 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 43c84ab2-a21e-441e-82d9-32f1b27926a0 2024-09-12T13:52:03.810 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:52:03.810 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:52:03.810 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:52:03.810 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:51:47' 2024-09-12T13:52:03.810 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:52:03.813 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:52:03.813 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:52:03.813 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 4 2024-09-12T13:52:04.413 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary4 up+unknown '' '' 2024-09-12T13:52:04.414 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:52:04.414 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:52:04.414 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:52:04.414 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:52:04.414 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:52:04.414 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:52:04.414 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:52:04.415 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:52:04.415 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary4 2024-09-12T13:52:04.429 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary8 up+unknown '' '' 2024-09-12T13:52:04.430 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:52:04.430 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:52:04.430 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:52:04.430 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:52:04.430 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:52:04.430 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:52:04.430 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:52:04.431 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:52:04.431 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary8 2024-09-12T13:52:04.472 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary3 up+unknown '' '' 2024-09-12T13:52:04.472 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:52:04.473 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:52:04.473 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:52:04.473 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:52:04.473 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:52:04.473 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:52:04.473 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:52:04.473 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:52:04.473 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary3 2024-09-12T13:52:04.486 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary4: 2024-09-12T13:52:04.486 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 6025c1db-ec31-4451-96a8-922baeed4ef6 2024-09-12T13:52:04.486 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:52:04.486 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:52:04.486 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:52:04.487 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:51:47' 2024-09-12T13:52:04.488 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:52:04.489 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:52:04.489 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:52:04.490 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 2 2024-09-12T13:52:04.502 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary8: 2024-09-12T13:52:04.502 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 901b32e6-a0c7-4be3-ada0-5f0083a2314c 2024-09-12T13:52:04.502 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:52:04.503 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:52:04.503 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:52:04.503 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:51:47' 2024-09-12T13:52:04.503 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:52:04.505 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:52:04.505 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:52:04.505 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 2 2024-09-12T13:52:04.549 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary3: 2024-09-12T13:52:04.549 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 73afcdb2-236d-496e-9163-3c6ff6e95e23 2024-09-12T13:52:04.549 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:52:04.549 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:52:04.549 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:52:04.549 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:51:47' 2024-09-12T13:52:04.549 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:52:04.552 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:52:04.552 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:52:04.552 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 4 2024-09-12T13:52:06.408 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Image demoted to non-primary 2024-09-12T13:52:06.421 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary2 up+unknown 2024-09-12T13:52:06.421 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:52:06.421 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:52:06.421 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:52:06.421 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:52:06.421 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:52:06.421 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:52:06.421 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:52:06.421 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:52:06.491 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary4 up+unknown '' '' 2024-09-12T13:52:06.491 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:52:06.492 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:52:06.492 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:52:06.492 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:52:06.492 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:52:06.492 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:52:06.492 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:52:06.492 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:52:06.492 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary4 2024-09-12T13:52:06.508 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary8 up+unknown '' '' 2024-09-12T13:52:06.508 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:52:06.508 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:52:06.508 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:52:06.508 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:52:06.508 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:52:06.508 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:52:06.508 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:52:06.509 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:52:06.509 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary8 2024-09-12T13:52:06.571 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary4: 2024-09-12T13:52:06.571 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 6025c1db-ec31-4451-96a8-922baeed4ef6 2024-09-12T13:52:06.571 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:52:06.571 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:52:06.571 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:52:06.572 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:51:47' 2024-09-12T13:52:06.572 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:52:06.574 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:52:06.574 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:52:06.574 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 4 2024-09-12T13:52:06.588 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary8: 2024-09-12T13:52:06.589 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 901b32e6-a0c7-4be3-ada0-5f0083a2314c 2024-09-12T13:52:06.589 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:52:06.589 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:52:06.589 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:52:06.589 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:51:47' 2024-09-12T13:52:06.589 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:52:06.591 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:52:06.591 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:52:06.591 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 4 2024-09-12T13:52:07.000 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary7 up+unknown '' '' 2024-09-12T13:52:07.000 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:52:07.000 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:52:07.000 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:52:07.000 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:52:07.000 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:52:07.000 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:52:07.001 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:52:07.001 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:52:07.001 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary7 2024-09-12T13:52:07.023 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary6 up+unknown '' '' 2024-09-12T13:52:07.023 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:52:07.023 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:52:07.023 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:52:07.023 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:52:07.023 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:52:07.023 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:52:07.023 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:52:07.024 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:52:07.024 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary6 2024-09-12T13:52:07.079 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary7: 2024-09-12T13:52:07.080 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: ecec213c-dc11-4547-88c6-ded8df679ff6 2024-09-12T13:52:07.080 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:52:07.080 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:52:07.080 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:52:07.080 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:51:47' 2024-09-12T13:52:07.080 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:52:07.083 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:52:07.083 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:52:07.083 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 8 2024-09-12T13:52:07.100 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary6: 2024-09-12T13:52:07.101 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 88271fa2-1e10-4868-befd-15807c269565 2024-09-12T13:52:07.101 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:52:07.101 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:52:07.101 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:52:07.101 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:51:47' 2024-09-12T13:52:07.101 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:52:07.103 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:52:07.103 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:52:07.103 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 8 2024-09-12T13:52:07.424 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary2 up+unknown '' '' 2024-09-12T13:52:07.424 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:52:07.424 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:52:07.424 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:52:07.424 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:52:07.424 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:52:07.424 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:52:07.424 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:52:07.425 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:52:07.425 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary2 2024-09-12T13:52:07.505 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary2: 2024-09-12T13:52:07.505 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 8399b5a6-24d7-4c4c-9667-28e949d25fb3 2024-09-12T13:52:07.505 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:52:07.505 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:52:07.505 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:52:07.506 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:51:47' 2024-09-12T13:52:07.506 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:52:07.508 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:52:07.508 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:52:07.508 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 2 2024-09-12T13:52:07.767 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary1 up+unknown '' '' 2024-09-12T13:52:07.768 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:52:07.768 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:52:07.768 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:52:07.768 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:52:07.768 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:52:07.768 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:52:07.768 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:52:07.768 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:52:07.769 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary1 2024-09-12T13:52:07.786 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary10 up+unknown '' '' 2024-09-12T13:52:07.786 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:52:07.786 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:52:07.786 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:52:07.787 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:52:07.787 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:52:07.787 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:52:07.787 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:52:07.787 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:52:07.787 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary10 2024-09-12T13:52:07.792 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary5 up+unknown '' '' 2024-09-12T13:52:07.792 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:52:07.792 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:52:07.792 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:52:07.792 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:52:07.792 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:52:07.792 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:52:07.792 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:52:07.793 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:52:07.793 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary5 2024-09-12T13:52:07.816 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary9 up+unknown '' '' 2024-09-12T13:52:07.817 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:52:07.817 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:52:07.817 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:52:07.817 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:52:07.817 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:52:07.817 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:52:07.817 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:52:07.817 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:52:07.817 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary9 2024-09-12T13:52:07.846 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary1: 2024-09-12T13:52:07.846 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: f6ac15bd-c3ff-4c34-86f6-e9bbc701f22f 2024-09-12T13:52:07.846 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:52:07.846 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:52:07.846 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:52:07.846 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:51:47' 2024-09-12T13:52:07.846 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:52:07.848 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:52:07.849 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:52:07.849 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 8 2024-09-12T13:52:07.864 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary10: 2024-09-12T13:52:07.864 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 02633bcb-38b0-4c95-9ca5-a988ab7c0dd0 2024-09-12T13:52:07.865 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:52:07.865 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:52:07.865 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:52:07.865 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:51:47' 2024-09-12T13:52:07.865 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:52:07.867 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:52:07.867 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:52:07.867 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 8 2024-09-12T13:52:07.871 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary5: 2024-09-12T13:52:07.871 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 6de1d859-a420-4abe-953c-031047e292ec 2024-09-12T13:52:07.871 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:52:07.871 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:52:07.871 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:52:07.871 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:51:47' 2024-09-12T13:52:07.871 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:52:07.874 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:52:07.874 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:52:07.874 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 8 2024-09-12T13:52:07.894 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary9: 2024-09-12T13:52:07.894 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 43c84ab2-a21e-441e-82d9-32f1b27926a0 2024-09-12T13:52:07.894 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:52:07.895 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:52:07.895 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:52:07.895 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:51:47' 2024-09-12T13:52:07.895 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:52:07.897 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:52:07.897 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:52:07.897 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 8 2024-09-12T13:52:08.554 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary3 up+unknown '' '' 2024-09-12T13:52:08.555 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:52:08.555 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:52:08.555 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:52:08.555 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:52:08.555 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:52:08.555 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:52:08.555 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:52:08.555 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:52:08.555 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary3 2024-09-12T13:52:08.634 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary3: 2024-09-12T13:52:08.634 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 73afcdb2-236d-496e-9163-3c6ff6e95e23 2024-09-12T13:52:08.634 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:52:08.634 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:52:08.634 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:52:08.634 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:51:47' 2024-09-12T13:52:08.634 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:52:08.636 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:52:08.636 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:52:08.636 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 8 2024-09-12T13:52:09.511 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary2 up+unknown '' '' 2024-09-12T13:52:09.511 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:52:09.511 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:52:09.511 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:52:09.511 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:52:09.511 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:52:09.511 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:52:09.511 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:52:09.512 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:52:09.512 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary2 2024-09-12T13:52:09.588 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary2: 2024-09-12T13:52:09.588 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 8399b5a6-24d7-4c4c-9667-28e949d25fb3 2024-09-12T13:52:09.588 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:52:09.588 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:52:09.588 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:52:09.589 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:51:47' 2024-09-12T13:52:09.589 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:52:09.591 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:52:09.592 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:52:09.592 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 4 2024-09-12T13:52:10.577 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary4 up+unknown '' '' 2024-09-12T13:52:10.577 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:52:10.577 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:52:10.577 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:52:10.577 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:52:10.577 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:52:10.577 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:52:10.577 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:52:10.578 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:52:10.578 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary4 2024-09-12T13:52:10.594 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary8 up+unknown '' '' 2024-09-12T13:52:10.594 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:52:10.594 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:52:10.594 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:52:10.594 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:52:10.595 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:52:10.595 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:52:10.595 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:52:10.595 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:52:10.595 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary8 2024-09-12T13:52:10.662 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary4: 2024-09-12T13:52:10.662 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 6025c1db-ec31-4451-96a8-922baeed4ef6 2024-09-12T13:52:10.662 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:52:10.662 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:52:10.663 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:52:10.663 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:51:47' 2024-09-12T13:52:10.663 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:52:10.665 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:52:10.665 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:52:10.665 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 8 2024-09-12T13:52:10.675 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary8: 2024-09-12T13:52:10.675 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 901b32e6-a0c7-4be3-ada0-5f0083a2314c 2024-09-12T13:52:10.675 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:52:10.676 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:52:10.676 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:52:10.676 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:51:47' 2024-09-12T13:52:10.676 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:52:10.678 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:52:10.678 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:52:10.678 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 8 2024-09-12T13:52:13.594 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary2 up+unknown '' '' 2024-09-12T13:52:13.594 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:52:13.594 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:52:13.594 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:52:13.594 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:52:13.594 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:52:13.594 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:52:13.594 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:52:13.595 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:52:13.595 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary2 2024-09-12T13:52:13.692 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary2: 2024-09-12T13:52:13.692 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 8399b5a6-24d7-4c4c-9667-28e949d25fb3 2024-09-12T13:52:13.692 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:52:13.692 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:52:13.692 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:52:13.692 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:51:47' 2024-09-12T13:52:13.692 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:52:13.695 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:52:13.695 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:52:13.695 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 8 2024-09-12T13:52:15.085 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary7 up+unknown '' '' 2024-09-12T13:52:15.086 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:52:15.086 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:52:15.086 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:52:15.086 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:52:15.086 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:52:15.086 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:52:15.086 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:52:15.086 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:52:15.086 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary7 2024-09-12T13:52:15.106 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary6 up+unknown '' '' 2024-09-12T13:52:15.106 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:52:15.106 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:52:15.106 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:52:15.106 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:52:15.106 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:52:15.106 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:52:15.106 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:52:15.107 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:52:15.107 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary6 2024-09-12T13:52:15.169 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary7: 2024-09-12T13:52:15.169 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: ecec213c-dc11-4547-88c6-ded8df679ff6 2024-09-12T13:52:15.169 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:52:15.169 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:52:15.170 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:52:15.170 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:51:47' 2024-09-12T13:52:15.170 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:52:15.171 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:52:15.171 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:52:15.172 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 8 2024-09-12T13:52:15.187 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary6: 2024-09-12T13:52:15.187 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 88271fa2-1e10-4868-befd-15807c269565 2024-09-12T13:52:15.187 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:52:15.188 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:52:15.188 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:52:15.188 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:51:47' 2024-09-12T13:52:15.188 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:52:15.189 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:52:15.190 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:52:15.190 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 8 2024-09-12T13:52:15.852 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary1 up+unknown '' '' 2024-09-12T13:52:15.852 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:52:15.852 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:52:15.852 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:52:15.852 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:52:15.852 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:52:15.852 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:52:15.852 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:52:15.853 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:52:15.853 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary1 2024-09-12T13:52:15.870 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary10 up+unknown '' '' 2024-09-12T13:52:15.870 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:52:15.870 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:52:15.870 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:52:15.870 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:52:15.870 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:52:15.870 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:52:15.870 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:52:15.871 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:52:15.871 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary10 2024-09-12T13:52:15.877 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary5 up+unknown '' '' 2024-09-12T13:52:15.877 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:52:15.877 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:52:15.877 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:52:15.877 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:52:15.877 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:52:15.877 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:52:15.878 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:52:15.878 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:52:15.878 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary5 2024-09-12T13:52:15.900 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary9 up+unknown '' '' 2024-09-12T13:52:15.900 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:52:15.900 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:52:15.900 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:52:15.900 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:52:15.901 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:52:15.901 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:52:15.901 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:52:15.901 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:52:15.901 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary9 2024-09-12T13:52:15.939 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary1: 2024-09-12T13:52:15.939 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: f6ac15bd-c3ff-4c34-86f6-e9bbc701f22f 2024-09-12T13:52:15.939 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:52:15.939 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:52:15.939 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:52:15.940 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:51:47' 2024-09-12T13:52:15.940 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:52:15.941 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:52:15.942 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:52:15.942 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 8 2024-09-12T13:52:15.952 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary10: 2024-09-12T13:52:15.952 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 02633bcb-38b0-4c95-9ca5-a988ab7c0dd0 2024-09-12T13:52:15.952 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:52:15.953 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:52:15.953 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:52:15.953 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:51:47' 2024-09-12T13:52:15.953 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:52:15.955 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:52:15.955 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:52:15.955 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 8 2024-09-12T13:52:15.956 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary5: 2024-09-12T13:52:15.956 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 6de1d859-a420-4abe-953c-031047e292ec 2024-09-12T13:52:15.956 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:52:15.956 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:52:15.956 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:52:15.956 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:51:47' 2024-09-12T13:52:15.956 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:52:15.959 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:52:15.959 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:52:15.959 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 8 2024-09-12T13:52:15.982 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary9: 2024-09-12T13:52:15.983 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 43c84ab2-a21e-441e-82d9-32f1b27926a0 2024-09-12T13:52:15.983 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:52:15.983 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:52:15.983 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:52:15.983 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:51:47' 2024-09-12T13:52:15.983 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:52:15.985 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:52:15.985 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:52:15.985 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 8 2024-09-12T13:52:16.640 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary3 up+unknown '' '' 2024-09-12T13:52:16.640 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:52:16.640 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:52:16.640 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:52:16.640 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:52:16.641 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:52:16.641 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:52:16.641 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:52:16.641 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:52:16.641 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary3 2024-09-12T13:52:16.721 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary3: 2024-09-12T13:52:16.721 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 73afcdb2-236d-496e-9163-3c6ff6e95e23 2024-09-12T13:52:16.721 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:52:16.721 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:52:16.721 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:52:16.721 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:51:47' 2024-09-12T13:52:16.721 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:52:16.724 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:52:16.724 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:52:16.724 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 8 2024-09-12T13:52:18.667 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary4 up+unknown '' '' 2024-09-12T13:52:18.667 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:52:18.668 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:52:18.668 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:52:18.668 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:52:18.668 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:52:18.668 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:52:18.668 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:52:18.668 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:52:18.668 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary4 2024-09-12T13:52:18.681 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary8 up+unknown '' '' 2024-09-12T13:52:18.681 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:52:18.681 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:52:18.681 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:52:18.681 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:52:18.681 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:52:18.681 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:52:18.682 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:52:18.682 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:52:18.682 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary8 2024-09-12T13:52:18.749 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary4: 2024-09-12T13:52:18.750 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 6025c1db-ec31-4451-96a8-922baeed4ef6 2024-09-12T13:52:18.750 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+unknown 2024-09-12T13:52:18.750 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: remote image is not primary 2024-09-12T13:52:18.750 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:52:18.750 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:52:17' 2024-09-12T13:52:18.751 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:52:18.752 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+unknown 2024-09-12T13:52:18.753 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:52:18.755 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: remote image is not primary 2024-09-12T13:52:18.755 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:52:18.756 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:52:18.756 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:52:18.758 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:52:18.759 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:52:18.761 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary8: 2024-09-12T13:52:18.761 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 901b32e6-a0c7-4be3-ada0-5f0083a2314c 2024-09-12T13:52:18.761 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+unknown 2024-09-12T13:52:18.761 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: remote image is not primary 2024-09-12T13:52:18.762 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:52:18.762 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:52:17' 2024-09-12T13:52:18.762 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:52:18.762 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:52:18.762 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:52:18.762 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:52:18.764 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+unknown 2024-09-12T13:52:18.764 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:52:18.766 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:52:17' 2024-09-12T13:52:18.766 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary4 up+unknown '2024-09-12 13:52:17' 2024-09-12T13:52:18.766 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:52:18.766 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:52:18.766 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:52:18.767 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:52:18.767 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:52:17' 2024-09-12T13:52:18.767 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:52:18.767 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:52:18.767 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-09-12T13:52:18.767 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: remote image is not primary 2024-09-12T13:52:18.768 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:52:18.768 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:52:18.768 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:52:18.770 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:52:18.770 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:52:18.773 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:52:18.773 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:52:18.775 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:52:18.778 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:52:17' 2024-09-12T13:52:18.778 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary8 up+unknown '2024-09-12 13:52:17' 2024-09-12T13:52:18.778 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:52:18.778 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:52:18.778 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:52:18.778 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:52:18.778 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:52:17' 2024-09-12T13:52:18.779 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:52:18.779 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:52:18.779 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-09-12T13:52:18.900 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041934232mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary1f6ac15bd-c3ff-4c34-86f6-e9bbc701f22fup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:52:17image-primary1002633bcb-38b0-4c95-9ca5-a988ab7c0dd0up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:52:17image-primary28399b5a6-24d7-4c4c-9667-28e949d25fb3up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:52:17image-primary373afcdb2-236d-496e-9163-3c6ff6e95e23up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:52:17image-primary46025c1db-ec31-4451-96a8-922baeed4ef6up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:52:17image-primary56de1d859-a420-4abe-953c-031047e292ecup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:52:17image-primary688271fa2-1e10-4868-befd-15807c269565up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:52:17image-primary7ecec213c-dc11-4547-88c6-ded8df679ff6up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:52:17image-primary8901b32e6-a0c7-4be3-ada0-5f0083a2314cup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:52:17image-primary943c84ab2-a21e-441e-82d9-32f1b27926a0up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:52:17' 2024-09-12T13:52:18.900 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:52:18.900 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary4'\'']/last_update' 2024-09-12T13:52:18.907 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041934232mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary1f6ac15bd-c3ff-4c34-86f6-e9bbc701f22fup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:52:17image-primary1002633bcb-38b0-4c95-9ca5-a988ab7c0dd0up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:52:17image-primary28399b5a6-24d7-4c4c-9667-28e949d25fb3up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:52:17image-primary373afcdb2-236d-496e-9163-3c6ff6e95e23up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:52:17image-primary46025c1db-ec31-4451-96a8-922baeed4ef6up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:52:17image-primary56de1d859-a420-4abe-953c-031047e292ecup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:52:17image-primary688271fa2-1e10-4868-befd-15807c269565up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:52:17image-primary7ecec213c-dc11-4547-88c6-ded8df679ff6up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:52:17image-primary8901b32e6-a0c7-4be3-ada0-5f0083a2314cup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:52:17image-primary943c84ab2-a21e-441e-82d9-32f1b27926a0up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:52:17' 2024-09-12T13:52:18.907 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:52:18.908 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary8'\'']/last_update' 2024-09-12T13:52:18.988 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:52:17' 2024-09-12T13:52:18.989 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:52:17' 2024-09-12T13:52:18.989 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary8'\'']/state' 2024-09-12T13:52:18.989 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary4'\'']/state' 2024-09-12T13:52:18.994 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+unknown 2024-09-12T13:52:18.994 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+unknown 2024-09-12T13:52:18.995 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:52:18.995 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:52:18.995 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+unknown 2024-09-12T13:52:18.995 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+unknown 2024-09-12T13:52:18.997 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:52:18.997 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:52:18.998 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:52:18.999 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:52:18.999 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:52:18.999 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:52:18.999 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster2 mirror image-primary4 up+unknown 2024-09-12T13:52:18.999 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster2 mirror image-primary8 up+unknown 2024-09-12T13:52:18.999 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:52:18.999 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:52:19.000 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:52:19.000 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:52:19.000 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:52:19.000 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:52:19.000 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:52:19.000 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:52:19.000 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:52:19.000 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:52:19.001 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:52:19.001 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:52:19.001 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:52:19.001 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:52:19.001 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:52:19.002 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:52:19.999 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster2 mirror image-primary8 up+unknown '' '' 2024-09-12T13:52:19.999 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster2 mirror image-primary4 up+unknown '' '' 2024-09-12T13:52:19.999 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:52:19.999 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:52:19.999 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:52:20.000 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:52:20.000 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:52:20.000 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:52:20.000 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:52:20.000 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:52:20.000 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:52:20.000 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:52:20.000 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:52:20.000 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:52:20.000 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:52:20.000 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:52:20.001 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:52:20.001 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster2 mirror image status mirror/image-primary8 2024-09-12T13:52:20.001 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:52:20.001 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster2 mirror image status mirror/image-primary4 2024-09-12T13:52:20.075 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary8: 2024-09-12T13:52:20.076 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 901b32e6-a0c7-4be3-ada0-5f0083a2314c 2024-09-12T13:52:20.076 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+unknown 2024-09-12T13:52:20.076 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: remote image is not primary 2024-09-12T13:52:20.076 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:52:20.076 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:52:17' 2024-09-12T13:52:20.076 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:52:20.076 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary4: 2024-09-12T13:52:20.076 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 6025c1db-ec31-4451-96a8-922baeed4ef6 2024-09-12T13:52:20.076 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+unknown 2024-09-12T13:52:20.076 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: remote image is not primary 2024-09-12T13:52:20.076 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:52:20.076 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:52:17' 2024-09-12T13:52:20.076 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:52:20.078 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+unknown 2024-09-12T13:52:20.078 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+unknown 2024-09-12T13:52:20.078 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:52:20.078 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:52:20.081 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: remote image is not primary 2024-09-12T13:52:20.081 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: remote image is not primary 2024-09-12T13:52:20.081 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:52:20.081 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:52:20.082 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:52:20.082 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:52:20.082 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:52:20.082 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:52:20.084 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:52:20.084 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:52:20.085 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:52:20.085 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:52:20.087 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:52:20.087 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:52:20.087 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:52:20.088 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:52:20.088 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:52:20.088 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:52:20.092 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:52:17' 2024-09-12T13:52:20.092 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster2 mirror image-primary4 up+unknown '2024-09-12 13:52:17' 2024-09-12T13:52:20.092 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:52:20.093 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:52:20.093 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:52:20.093 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:52:20.093 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:52:17' 2024-09-12T13:52:20.093 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:52:20.093 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:52:17' 2024-09-12T13:52:20.093 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster2 mirror image-primary8 up+unknown '2024-09-12 13:52:17' 2024-09-12T13:52:20.093 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:52:20.093 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:52:20.093 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:52:20.093 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:52:20.093 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:52:17' 2024-09-12T13:52:20.093 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:52:20.093 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:52:20.094 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2024-09-12T13:52:20.094 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:52:20.094 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2024-09-12T13:52:20.223 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041924231mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary1f6ac15bd-c3ff-4c34-86f6-e9bbc701f22fup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:52:17image-primary1002633bcb-38b0-4c95-9ca5-a988ab7c0dd0up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:52:17image-primary28399b5a6-24d7-4c4c-9667-28e949d25fb3up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:52:17image-primary373afcdb2-236d-496e-9163-3c6ff6e95e23up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:52:17image-primary46025c1db-ec31-4451-96a8-922baeed4ef6up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:52:17image-primary56de1d859-a420-4abe-953c-031047e292ecup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:52:17image-primary688271fa2-1e10-4868-befd-15807c269565up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:52:17image-primary7ecec213c-dc11-4547-88c6-ded8df679ff6up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:52:17image-primary8901b32e6-a0c7-4be3-ada0-5f0083a2314cup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:52:17image-primary943c84ab2-a21e-441e-82d9-32f1b27926a0up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:52:17' 2024-09-12T13:52:20.223 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:52:20.224 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary8'\'']/last_update' 2024-09-12T13:52:20.224 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041924231mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary1f6ac15bd-c3ff-4c34-86f6-e9bbc701f22fup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:52:17image-primary1002633bcb-38b0-4c95-9ca5-a988ab7c0dd0up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:52:17image-primary28399b5a6-24d7-4c4c-9667-28e949d25fb3up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:52:17image-primary373afcdb2-236d-496e-9163-3c6ff6e95e23up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:52:17image-primary46025c1db-ec31-4451-96a8-922baeed4ef6up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:52:17image-primary56de1d859-a420-4abe-953c-031047e292ecup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:52:17image-primary688271fa2-1e10-4868-befd-15807c269565up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:52:17image-primary7ecec213c-dc11-4547-88c6-ded8df679ff6up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:52:17image-primary8901b32e6-a0c7-4be3-ada0-5f0083a2314cup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:52:17image-primary943c84ab2-a21e-441e-82d9-32f1b27926a0up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:52:17' 2024-09-12T13:52:20.224 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:52:20.225 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary4'\'']/last_update' 2024-09-12T13:52:20.227 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:52:17' 2024-09-12T13:52:20.229 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:52:17' 2024-09-12T13:52:20.229 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary8'\'']/state' 2024-09-12T13:52:20.229 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary4'\'']/state' 2024-09-12T13:52:20.232 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+unknown 2024-09-12T13:52:20.233 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:52:20.233 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+unknown 2024-09-12T13:52:20.234 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+unknown 2024-09-12T13:52:20.234 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:52:20.234 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+unknown 2024-09-12T13:52:20.235 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:52:20.236 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:52:20.236 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:52:20.236 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ promote_image cluster2 mirror image-primary8 2024-09-12T13:52:20.236 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:52:20.236 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:52:20.237 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:52:20.237 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local force= 2024-09-12T13:52:20.237 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 mirror image promote mirror/image-primary8 2024-09-12T13:52:20.237 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:52:20.238 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:52:20.238 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:52:20.238 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ promote_image cluster2 mirror image-primary4 2024-09-12T13:52:20.238 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:52:20.238 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:52:20.238 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:52:20.238 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local force= 2024-09-12T13:52:20.238 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 mirror image promote mirror/image-primary4 2024-09-12T13:52:20.548 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Image promoted to primary 2024-09-12T13:52:20.548 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Image promoted to primary 2024-09-12T13:52:20.560 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \n\b\d ]] 2024-09-12T13:52:20.560 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \k\r\b\d ]] 2024-09-12T13:52:20.560 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \n\b\d ]] 2024-09-12T13:52:20.560 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \k\r\b\d ]] 2024-09-12T13:52:20.560 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo rbd --cluster cluster2 device map -t krbd mirror/image-primary8 2024-09-12T13:52:20.560 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo rbd --cluster cluster2 device map -t krbd mirror/image-primary4 2024-09-12T13:52:20.696 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ dev=/dev/rbd1 2024-09-12T13:52:20.697 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo md5sum /dev/rbd1 2024-09-12T13:52:20.697 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ dev=/dev/rbd0 2024-09-12T13:52:20.697 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ awk '{print $1}' 2024-09-12T13:52:20.698 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo md5sum /dev/rbd0 2024-09-12T13:52:20.698 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ awk '{print $1}' 2024-09-12T13:52:21.698 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary2 up+unknown '' '' 2024-09-12T13:52:21.698 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:52:21.698 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:52:21.698 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:52:21.698 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:52:21.698 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:52:21.698 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:52:21.698 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:52:21.699 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:52:21.699 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary2 2024-09-12T13:52:21.774 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary2: 2024-09-12T13:52:21.774 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 8399b5a6-24d7-4c4c-9667-28e949d25fb3 2024-09-12T13:52:21.774 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+unknown 2024-09-12T13:52:21.774 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: remote image is not primary 2024-09-12T13:52:21.774 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:52:21.774 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:52:17' 2024-09-12T13:52:21.775 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:52:21.777 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+unknown 2024-09-12T13:52:21.777 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:52:21.780 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: remote image is not primary 2024-09-12T13:52:21.780 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:52:21.781 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:52:21.781 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:52:21.783 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:52:21.784 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:52:21.786 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:52:21.787 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:52:21.788 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:52:21.792 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:52:17' 2024-09-12T13:52:21.792 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary2 up+unknown '2024-09-12 13:52:17' 2024-09-12T13:52:21.792 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:52:21.792 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:52:21.792 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:52:21.792 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:52:21.792 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:52:17' 2024-09-12T13:52:21.792 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:52:21.793 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:52:21.793 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-09-12T13:52:21.903 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041934232mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary1f6ac15bd-c3ff-4c34-86f6-e9bbc701f22fup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:52:17image-primary1002633bcb-38b0-4c95-9ca5-a988ab7c0dd0up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:52:17image-primary28399b5a6-24d7-4c4c-9667-28e949d25fb3up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:52:17image-primary373afcdb2-236d-496e-9163-3c6ff6e95e23up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:52:17image-primary46025c1db-ec31-4451-96a8-922baeed4ef6up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:52:17image-primary56de1d859-a420-4abe-953c-031047e292ecup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:52:17image-primary688271fa2-1e10-4868-befd-15807c269565up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:52:17image-primary7ecec213c-dc11-4547-88c6-ded8df679ff6up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:52:17image-primary8901b32e6-a0c7-4be3-ada0-5f0083a2314cup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:52:17image-primary943c84ab2-a21e-441e-82d9-32f1b27926a0up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:52:17' 2024-09-12T13:52:21.904 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:52:21.904 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary2'\'']/last_update' 2024-09-12T13:52:21.909 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:52:17' 2024-09-12T13:52:21.909 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary2'\'']/state' 2024-09-12T13:52:21.913 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+unknown 2024-09-12T13:52:21.914 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:52:21.915 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+unknown 2024-09-12T13:52:21.917 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:52:21.918 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:52:21.918 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:52:21.918 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster2 mirror image-primary2 up+unknown 2024-09-12T13:52:21.918 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:52:21.918 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:52:21.918 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:52:21.918 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:52:21.918 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:52:21.918 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:52:21.918 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:52:21.918 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:52:22.920 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster2 mirror image-primary2 up+unknown '' '' 2024-09-12T13:52:22.921 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:52:22.921 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:52:22.921 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:52:22.921 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:52:22.921 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:52:22.921 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:52:22.921 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:52:22.921 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:52:22.921 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster2 mirror image status mirror/image-primary2 2024-09-12T13:52:22.993 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary2: 2024-09-12T13:52:22.994 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 8399b5a6-24d7-4c4c-9667-28e949d25fb3 2024-09-12T13:52:22.994 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+unknown 2024-09-12T13:52:22.994 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: remote image is not primary 2024-09-12T13:52:22.994 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:52:22.994 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:52:17' 2024-09-12T13:52:22.994 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:52:22.996 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+unknown 2024-09-12T13:52:22.996 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:52:22.999 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: remote image is not primary 2024-09-12T13:52:22.999 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:52:23.000 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:52:23.000 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:52:23.002 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:52:23.003 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:52:23.005 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:52:23.005 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:52:23.006 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:52:23.010 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:52:17' 2024-09-12T13:52:23.010 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster2 mirror image-primary2 up+unknown '2024-09-12 13:52:17' 2024-09-12T13:52:23.010 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:52:23.010 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:52:23.011 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:52:23.011 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:52:23.011 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:52:17' 2024-09-12T13:52:23.011 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:52:23.011 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:52:23.011 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2024-09-12T13:52:23.129 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041924231mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary1f6ac15bd-c3ff-4c34-86f6-e9bbc701f22fup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:52:17image-primary1002633bcb-38b0-4c95-9ca5-a988ab7c0dd0up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:52:17image-primary28399b5a6-24d7-4c4c-9667-28e949d25fb3up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:52:17image-primary373afcdb2-236d-496e-9163-3c6ff6e95e23up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:52:17image-primary46025c1db-ec31-4451-96a8-922baeed4ef6up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:52:17image-primary56de1d859-a420-4abe-953c-031047e292ecup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:52:17image-primary688271fa2-1e10-4868-befd-15807c269565up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:52:17image-primary7ecec213c-dc11-4547-88c6-ded8df679ff6up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:52:17image-primary8901b32e6-a0c7-4be3-ada0-5f0083a2314cup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:52:17image-primary943c84ab2-a21e-441e-82d9-32f1b27926a0up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:52:17' 2024-09-12T13:52:23.129 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:52:23.130 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary2'\'']/last_update' 2024-09-12T13:52:23.133 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:52:17' 2024-09-12T13:52:23.134 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary2'\'']/state' 2024-09-12T13:52:23.138 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+unknown 2024-09-12T13:52:23.139 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:52:23.141 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+unknown 2024-09-12T13:52:23.142 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:52:23.143 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:52:23.143 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:52:23.143 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ promote_image cluster2 mirror image-primary2 2024-09-12T13:52:23.143 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:52:23.143 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:52:23.143 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:52:23.143 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local force= 2024-09-12T13:52:23.143 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 mirror image promote mirror/image-primary2 2024-09-12T13:52:23.174 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary7 up+unknown '' '' 2024-09-12T13:52:23.174 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:52:23.174 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:52:23.175 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:52:23.175 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:52:23.175 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:52:23.175 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:52:23.175 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:52:23.175 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:52:23.175 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary7 2024-09-12T13:52:23.192 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary6 up+unknown '' '' 2024-09-12T13:52:23.192 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:52:23.192 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:52:23.192 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:52:23.192 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:52:23.192 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:52:23.192 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:52:23.193 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:52:23.193 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:52:23.193 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary6 2024-09-12T13:52:23.258 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary7: 2024-09-12T13:52:23.259 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: ecec213c-dc11-4547-88c6-ded8df679ff6 2024-09-12T13:52:23.259 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+unknown 2024-09-12T13:52:23.259 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: remote image is not primary 2024-09-12T13:52:23.259 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:52:23.259 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:52:17' 2024-09-12T13:52:23.259 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:52:23.261 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+unknown 2024-09-12T13:52:23.261 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:52:23.264 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: remote image is not primary 2024-09-12T13:52:23.264 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:52:23.265 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:52:23.266 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:52:23.268 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:52:23.268 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:52:23.272 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:52:23.272 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:52:23.273 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:52:23.276 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary6: 2024-09-12T13:52:23.276 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 88271fa2-1e10-4868-befd-15807c269565 2024-09-12T13:52:23.276 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+unknown 2024-09-12T13:52:23.276 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: remote image is not primary 2024-09-12T13:52:23.276 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:52:23.276 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:52:17' 2024-09-12T13:52:23.276 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:52:23.277 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:52:17' 2024-09-12T13:52:23.277 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary7 up+unknown '2024-09-12 13:52:17' 2024-09-12T13:52:23.277 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:52:23.277 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:52:23.278 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:52:23.278 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:52:23.278 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:52:17' 2024-09-12T13:52:23.278 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:52:23.278 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:52:23.278 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-09-12T13:52:23.279 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+unknown 2024-09-12T13:52:23.279 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:52:23.281 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: remote image is not primary 2024-09-12T13:52:23.282 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:52:23.282 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:52:23.283 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:52:23.285 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:52:23.285 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:52:23.288 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:52:23.288 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:52:23.289 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:52:23.293 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:52:17' 2024-09-12T13:52:23.293 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary6 up+unknown '2024-09-12 13:52:17' 2024-09-12T13:52:23.294 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:52:23.294 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:52:23.294 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:52:23.294 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:52:23.294 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:52:17' 2024-09-12T13:52:23.294 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:52:23.294 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:52:23.295 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-09-12T13:52:23.414 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041934232mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary1f6ac15bd-c3ff-4c34-86f6-e9bbc701f22fup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:52:17image-primary1002633bcb-38b0-4c95-9ca5-a988ab7c0dd0up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:52:17image-primary28399b5a6-24d7-4c4c-9667-28e949d25fb3up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:52:17image-primary373afcdb2-236d-496e-9163-3c6ff6e95e23up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:52:17image-primary46025c1db-ec31-4451-96a8-922baeed4ef6up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:52:17image-primary56de1d859-a420-4abe-953c-031047e292ecup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:52:17image-primary688271fa2-1e10-4868-befd-15807c269565up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:52:17image-primary7ecec213c-dc11-4547-88c6-ded8df679ff6up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:52:17image-primary8901b32e6-a0c7-4be3-ada0-5f0083a2314cup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:52:17image-primary943c84ab2-a21e-441e-82d9-32f1b27926a0up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:52:17' 2024-09-12T13:52:23.414 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:52:23.414 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary7'\'']/last_update' 2024-09-12T13:52:23.419 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:52:17' 2024-09-12T13:52:23.420 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary7'\'']/state' 2024-09-12T13:52:23.422 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041934232mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary1f6ac15bd-c3ff-4c34-86f6-e9bbc701f22fup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:52:17image-primary1002633bcb-38b0-4c95-9ca5-a988ab7c0dd0up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:52:17image-primary28399b5a6-24d7-4c4c-9667-28e949d25fb3up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:52:17image-primary373afcdb2-236d-496e-9163-3c6ff6e95e23up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:52:17image-primary46025c1db-ec31-4451-96a8-922baeed4ef6up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:52:17image-primary56de1d859-a420-4abe-953c-031047e292ecup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:52:17image-primary688271fa2-1e10-4868-befd-15807c269565up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:52:17image-primary7ecec213c-dc11-4547-88c6-ded8df679ff6up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:52:17image-primary8901b32e6-a0c7-4be3-ada0-5f0083a2314cup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:52:17image-primary943c84ab2-a21e-441e-82d9-32f1b27926a0up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:52:17' 2024-09-12T13:52:23.422 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:52:23.423 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary6'\'']/last_update' 2024-09-12T13:52:23.424 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+unknown 2024-09-12T13:52:23.425 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:52:23.425 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+unknown 2024-09-12T13:52:23.427 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:52:17' 2024-09-12T13:52:23.428 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:52:23.429 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary6'\'']/state' 2024-09-12T13:52:23.429 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:52:23.429 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:52:23.429 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster2 mirror image-primary7 up+unknown 2024-09-12T13:52:23.429 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:52:23.429 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:52:23.429 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:52:23.429 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:52:23.430 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:52:23.430 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:52:23.430 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:52:23.430 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:52:23.432 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+unknown 2024-09-12T13:52:23.433 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:52:23.433 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+unknown 2024-09-12T13:52:23.435 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:52:23.436 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:52:23.436 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:52:23.436 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster2 mirror image-primary6 up+unknown 2024-09-12T13:52:23.436 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:52:23.436 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:52:23.436 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:52:23.437 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:52:23.437 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:52:23.437 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:52:23.437 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:52:23.437 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:52:23.558 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Image promoted to primary 2024-09-12T13:52:23.570 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \n\b\d ]] 2024-09-12T13:52:23.571 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \k\r\b\d ]] 2024-09-12T13:52:23.571 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo rbd --cluster cluster2 device map -t krbd mirror/image-primary2 2024-09-12T13:52:23.694 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ dev=/dev/rbd2 2024-09-12T13:52:23.695 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo md5sum /dev/rbd2 2024-09-12T13:52:23.695 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ awk '{print $1}' 2024-09-12T13:52:23.944 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary1 up+unknown '' '' 2024-09-12T13:52:23.945 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:52:23.945 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:52:23.945 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:52:23.945 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:52:23.945 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:52:23.945 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:52:23.945 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:52:23.945 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:52:23.945 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary1 2024-09-12T13:52:23.958 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary10 up+unknown '' '' 2024-09-12T13:52:23.958 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:52:23.959 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:52:23.959 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:52:23.959 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:52:23.959 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:52:23.959 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:52:23.959 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:52:23.959 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:52:23.959 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary10 2024-09-12T13:52:23.961 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary5 up+unknown '' '' 2024-09-12T13:52:23.962 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:52:23.962 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:52:23.962 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:52:23.962 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:52:23.962 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:52:23.962 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:52:23.962 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:52:23.963 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:52:23.963 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary5 2024-09-12T13:52:23.988 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary9 up+unknown '' '' 2024-09-12T13:52:23.988 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:52:23.988 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:52:23.988 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:52:23.988 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:52:23.989 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:52:23.989 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:52:23.989 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:52:23.989 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:52:23.989 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary9 2024-09-12T13:52:24.032 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary1: 2024-09-12T13:52:24.032 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: f6ac15bd-c3ff-4c34-86f6-e9bbc701f22f 2024-09-12T13:52:24.032 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+unknown 2024-09-12T13:52:24.032 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: remote image is not primary 2024-09-12T13:52:24.032 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:52:24.032 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:52:17' 2024-09-12T13:52:24.032 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:52:24.035 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+unknown 2024-09-12T13:52:24.036 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:52:24.040 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: remote image is not primary 2024-09-12T13:52:24.040 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:52:24.041 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:52:24.042 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:52:24.045 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:52:24.046 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:52:24.050 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:52:24.050 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:52:24.051 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:52:24.056 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:52:17' 2024-09-12T13:52:24.056 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary1 up+unknown '2024-09-12 13:52:17' 2024-09-12T13:52:24.056 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:52:24.056 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:52:24.056 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:52:24.056 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:52:24.056 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:52:17' 2024-09-12T13:52:24.056 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:52:24.056 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary5: 2024-09-12T13:52:24.056 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 6de1d859-a420-4abe-953c-031047e292ec 2024-09-12T13:52:24.056 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+unknown 2024-09-12T13:52:24.057 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: remote image is not primary 2024-09-12T13:52:24.057 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:52:24.057 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:52:17' 2024-09-12T13:52:24.057 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:52:24.057 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:52:24.057 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-09-12T13:52:24.060 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+unknown 2024-09-12T13:52:24.060 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:52:24.064 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: remote image is not primary 2024-09-12T13:52:24.064 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:52:24.065 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:52:24.065 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:52:24.067 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:52:24.068 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:52:24.071 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:52:24.071 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:52:24.073 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:52:24.073 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary10: 2024-09-12T13:52:24.073 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 02633bcb-38b0-4c95-9ca5-a988ab7c0dd0 2024-09-12T13:52:24.073 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+unknown 2024-09-12T13:52:24.073 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: remote image is not primary 2024-09-12T13:52:24.073 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:52:24.073 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:52:17' 2024-09-12T13:52:24.073 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:52:24.076 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+unknown 2024-09-12T13:52:24.076 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:52:24.079 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:52:17' 2024-09-12T13:52:24.079 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary5 up+unknown '2024-09-12 13:52:17' 2024-09-12T13:52:24.079 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:52:24.079 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:52:24.079 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:52:24.079 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:52:24.079 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:52:17' 2024-09-12T13:52:24.080 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:52:24.080 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: remote image is not primary 2024-09-12T13:52:24.080 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:52:24.080 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:52:24.080 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-09-12T13:52:24.082 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:52:24.082 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:52:24.085 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary9: 2024-09-12T13:52:24.085 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 43c84ab2-a21e-441e-82d9-32f1b27926a0 2024-09-12T13:52:24.085 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+unknown 2024-09-12T13:52:24.085 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: remote image is not primary 2024-09-12T13:52:24.085 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:52:24.085 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:52:17' 2024-09-12T13:52:24.085 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:52:24.086 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:52:24.086 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:52:24.088 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+unknown 2024-09-12T13:52:24.088 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:52:24.090 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:52:24.091 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:52:24.091 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: remote image is not primary 2024-09-12T13:52:24.092 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:52:24.092 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:52:24.093 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:52:24.093 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:52:24.095 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:52:24.096 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:52:24.099 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:52:24.099 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:52:17' 2024-09-12T13:52:24.099 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:52:24.100 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary10 up+unknown '2024-09-12 13:52:17' 2024-09-12T13:52:24.100 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:52:24.100 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:52:24.100 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:52:24.100 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:52:24.100 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:52:17' 2024-09-12T13:52:24.100 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:52:24.100 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:52:24.101 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-09-12T13:52:24.101 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:52:24.107 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:52:17' 2024-09-12T13:52:24.107 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary9 up+unknown '2024-09-12 13:52:17' 2024-09-12T13:52:24.107 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:52:24.107 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:52:24.108 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:52:24.108 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:52:24.108 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:52:17' 2024-09-12T13:52:24.108 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:52:24.108 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:52:24.108 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-09-12T13:52:24.220 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041934232mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary1f6ac15bd-c3ff-4c34-86f6-e9bbc701f22fup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:52:17image-primary1002633bcb-38b0-4c95-9ca5-a988ab7c0dd0up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:52:17image-primary28399b5a6-24d7-4c4c-9667-28e949d25fb3up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:52:17image-primary373afcdb2-236d-496e-9163-3c6ff6e95e23up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:52:17image-primary46025c1db-ec31-4451-96a8-922baeed4ef6up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:52:17image-primary56de1d859-a420-4abe-953c-031047e292ecup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:52:17image-primary688271fa2-1e10-4868-befd-15807c269565up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:52:17image-primary7ecec213c-dc11-4547-88c6-ded8df679ff6up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:52:17image-primary8901b32e6-a0c7-4be3-ada0-5f0083a2314cup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:52:17image-primary943c84ab2-a21e-441e-82d9-32f1b27926a0up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:52:17' 2024-09-12T13:52:24.220 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:52:24.221 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary1'\'']/last_update' 2024-09-12T13:52:24.226 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:52:17' 2024-09-12T13:52:24.227 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary1'\'']/state' 2024-09-12T13:52:24.232 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+unknown 2024-09-12T13:52:24.233 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:52:24.233 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+unknown 2024-09-12T13:52:24.236 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:52:24.237 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:52:24.237 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:52:24.237 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster2 mirror image-primary1 up+unknown 2024-09-12T13:52:24.237 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:52:24.237 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:52:24.237 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:52:24.238 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:52:24.238 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:52:24.238 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:52:24.238 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:52:24.238 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:52:24.257 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041934232mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary1f6ac15bd-c3ff-4c34-86f6-e9bbc701f22fup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:52:17image-primary1002633bcb-38b0-4c95-9ca5-a988ab7c0dd0up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:52:17image-primary28399b5a6-24d7-4c4c-9667-28e949d25fb3up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:52:17image-primary373afcdb2-236d-496e-9163-3c6ff6e95e23up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:52:17image-primary46025c1db-ec31-4451-96a8-922baeed4ef6up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:52:17image-primary56de1d859-a420-4abe-953c-031047e292ecup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:52:17image-primary688271fa2-1e10-4868-befd-15807c269565up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:52:17image-primary7ecec213c-dc11-4547-88c6-ded8df679ff6up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:52:17image-primary8901b32e6-a0c7-4be3-ada0-5f0083a2314cup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:52:17image-primary943c84ab2-a21e-441e-82d9-32f1b27926a0up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:52:17' 2024-09-12T13:52:24.257 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:52:24.258 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary5'\'']/last_update' 2024-09-12T13:52:24.263 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:52:17' 2024-09-12T13:52:24.264 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary5'\'']/state' 2024-09-12T13:52:24.268 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+unknown 2024-09-12T13:52:24.269 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:52:24.270 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+unknown 2024-09-12T13:52:24.273 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:52:24.274 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:52:24.274 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:52:24.275 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster2 mirror image-primary5 up+unknown 2024-09-12T13:52:24.275 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:52:24.275 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:52:24.275 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:52:24.275 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:52:24.275 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:52:24.275 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:52:24.275 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:52:24.275 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:52:24.277 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041934232mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary1f6ac15bd-c3ff-4c34-86f6-e9bbc701f22fup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:52:17image-primary1002633bcb-38b0-4c95-9ca5-a988ab7c0dd0up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:52:17image-primary28399b5a6-24d7-4c4c-9667-28e949d25fb3up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:52:17image-primary373afcdb2-236d-496e-9163-3c6ff6e95e23up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:52:17image-primary46025c1db-ec31-4451-96a8-922baeed4ef6up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:52:17image-primary56de1d859-a420-4abe-953c-031047e292ecup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:52:17image-primary688271fa2-1e10-4868-befd-15807c269565up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:52:17image-primary7ecec213c-dc11-4547-88c6-ded8df679ff6up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:52:17image-primary8901b32e6-a0c7-4be3-ada0-5f0083a2314cup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:52:17image-primary943c84ab2-a21e-441e-82d9-32f1b27926a0up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:52:17' 2024-09-12T13:52:24.278 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:52:24.278 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary10'\'']/last_update' 2024-09-12T13:52:24.285 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:52:17' 2024-09-12T13:52:24.286 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary10'\'']/state' 2024-09-12T13:52:24.290 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+unknown 2024-09-12T13:52:24.291 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:52:24.292 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+unknown 2024-09-12T13:52:24.294 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:52:24.295 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:52:24.295 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:52:24.295 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster2 mirror image-primary10 up+unknown 2024-09-12T13:52:24.295 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:52:24.295 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:52:24.295 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:52:24.296 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:52:24.296 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:52:24.296 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:52:24.296 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:52:24.296 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:52:24.296 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041934232mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary1f6ac15bd-c3ff-4c34-86f6-e9bbc701f22fup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:52:17image-primary1002633bcb-38b0-4c95-9ca5-a988ab7c0dd0up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:52:17image-primary28399b5a6-24d7-4c4c-9667-28e949d25fb3up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:52:17image-primary373afcdb2-236d-496e-9163-3c6ff6e95e23up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:52:17image-primary46025c1db-ec31-4451-96a8-922baeed4ef6up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:52:17image-primary56de1d859-a420-4abe-953c-031047e292ecup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:52:17image-primary688271fa2-1e10-4868-befd-15807c269565up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:52:17image-primary7ecec213c-dc11-4547-88c6-ded8df679ff6up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:52:17image-primary8901b32e6-a0c7-4be3-ada0-5f0083a2314cup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:52:17image-primary943c84ab2-a21e-441e-82d9-32f1b27926a0up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:52:17' 2024-09-12T13:52:24.296 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:52:24.297 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary9'\'']/last_update' 2024-09-12T13:52:24.301 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:52:17' 2024-09-12T13:52:24.302 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary9'\'']/state' 2024-09-12T13:52:24.306 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+unknown 2024-09-12T13:52:24.307 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:52:24.308 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+unknown 2024-09-12T13:52:24.310 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:52:24.311 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:52:24.311 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:52:24.312 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster2 mirror image-primary9 up+unknown 2024-09-12T13:52:24.312 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:52:24.312 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:52:24.312 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:52:24.312 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:52:24.312 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:52:24.312 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:52:24.312 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:52:24.312 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:52:24.431 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster2 mirror image-primary7 up+unknown '' '' 2024-09-12T13:52:24.431 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:52:24.431 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:52:24.431 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:52:24.431 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:52:24.432 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:52:24.432 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:52:24.432 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:52:24.432 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:52:24.432 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster2 mirror image status mirror/image-primary7 2024-09-12T13:52:24.438 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster2 mirror image-primary6 up+unknown '' '' 2024-09-12T13:52:24.438 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:52:24.438 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:52:24.439 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:52:24.439 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:52:24.439 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:52:24.439 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:52:24.439 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:52:24.439 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:52:24.439 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster2 mirror image status mirror/image-primary6 2024-09-12T13:52:24.510 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary7: 2024-09-12T13:52:24.510 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: ecec213c-dc11-4547-88c6-ded8df679ff6 2024-09-12T13:52:24.511 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+unknown 2024-09-12T13:52:24.511 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: remote image is not primary 2024-09-12T13:52:24.511 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:52:24.511 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:52:17' 2024-09-12T13:52:24.511 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:52:24.513 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+unknown 2024-09-12T13:52:24.514 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:52:24.518 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: remote image is not primary 2024-09-12T13:52:24.519 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:52:24.520 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:52:24.521 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:52:24.524 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:52:24.524 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:52:24.529 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:52:24.530 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:52:24.531 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:52:24.535 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary6: 2024-09-12T13:52:24.535 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 88271fa2-1e10-4868-befd-15807c269565 2024-09-12T13:52:24.535 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+unknown 2024-09-12T13:52:24.535 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: remote image is not primary 2024-09-12T13:52:24.535 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:52:24.535 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:52:17' 2024-09-12T13:52:24.536 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:52:24.536 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:52:17' 2024-09-12T13:52:24.536 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster2 mirror image-primary7 up+unknown '2024-09-12 13:52:17' 2024-09-12T13:52:24.536 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:52:24.536 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:52:24.536 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:52:24.536 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:52:24.536 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:52:17' 2024-09-12T13:52:24.536 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:52:24.537 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:52:24.537 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2024-09-12T13:52:24.539 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+unknown 2024-09-12T13:52:24.539 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:52:24.544 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: remote image is not primary 2024-09-12T13:52:24.544 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:52:24.549 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:52:24.550 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:52:24.550 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:52:24.550 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:52:24.554 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:52:24.555 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:52:24.556 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:52:24.563 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:52:17' 2024-09-12T13:52:24.563 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster2 mirror image-primary6 up+unknown '2024-09-12 13:52:17' 2024-09-12T13:52:24.563 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:52:24.563 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:52:24.563 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:52:24.563 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:52:24.563 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:52:17' 2024-09-12T13:52:24.563 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:52:24.564 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:52:24.564 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2024-09-12T13:52:24.728 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary3 up+unknown '' '' 2024-09-12T13:52:24.728 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:52:24.728 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:52:24.728 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:52:24.728 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:52:24.728 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:52:24.728 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:52:24.728 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:52:24.729 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:52:24.729 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary3 2024-09-12T13:52:24.733 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041924231mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary1f6ac15bd-c3ff-4c34-86f6-e9bbc701f22fup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:52:17image-primary1002633bcb-38b0-4c95-9ca5-a988ab7c0dd0up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:52:17image-primary28399b5a6-24d7-4c4c-9667-28e949d25fb3up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:52:17image-primary373afcdb2-236d-496e-9163-3c6ff6e95e23up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:52:17image-primary46025c1db-ec31-4451-96a8-922baeed4ef6up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:52:17image-primary56de1d859-a420-4abe-953c-031047e292ecup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:52:17image-primary688271fa2-1e10-4868-befd-15807c269565up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:52:17image-primary7ecec213c-dc11-4547-88c6-ded8df679ff6up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:52:17image-primary8901b32e6-a0c7-4be3-ada0-5f0083a2314cup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:52:17image-primary943c84ab2-a21e-441e-82d9-32f1b27926a0up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:52:17' 2024-09-12T13:52:24.733 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:52:24.734 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary7'\'']/last_update' 2024-09-12T13:52:24.743 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:52:17' 2024-09-12T13:52:24.743 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary7'\'']/state' 2024-09-12T13:52:24.745 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041924231mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary1f6ac15bd-c3ff-4c34-86f6-e9bbc701f22fup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:52:17image-primary1002633bcb-38b0-4c95-9ca5-a988ab7c0dd0up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:52:17image-primary28399b5a6-24d7-4c4c-9667-28e949d25fb3up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:52:17image-primary373afcdb2-236d-496e-9163-3c6ff6e95e23up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:52:17image-primary46025c1db-ec31-4451-96a8-922baeed4ef6up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:52:17image-primary56de1d859-a420-4abe-953c-031047e292ecup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:52:17image-primary688271fa2-1e10-4868-befd-15807c269565up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:52:17image-primary7ecec213c-dc11-4547-88c6-ded8df679ff6up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:52:17image-primary8901b32e6-a0c7-4be3-ada0-5f0083a2314cup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:52:17image-primary943c84ab2-a21e-441e-82d9-32f1b27926a0up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:52:17' 2024-09-12T13:52:24.746 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:52:24.746 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary6'\'']/last_update' 2024-09-12T13:52:24.753 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+unknown 2024-09-12T13:52:24.754 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:52:17' 2024-09-12T13:52:24.754 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:52:24.754 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary6'\'']/state' 2024-09-12T13:52:24.754 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+unknown 2024-09-12T13:52:24.755 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:52:24.756 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:52:24.756 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:52:24.756 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ promote_image cluster2 mirror image-primary7 2024-09-12T13:52:24.756 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:52:24.757 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:52:24.757 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:52:24.757 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local force= 2024-09-12T13:52:24.757 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 mirror image promote mirror/image-primary7 2024-09-12T13:52:24.757 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+unknown 2024-09-12T13:52:24.757 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:52:24.758 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+unknown 2024-09-12T13:52:24.761 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:52:24.762 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:52:24.762 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:52:24.762 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ promote_image cluster2 mirror image-primary6 2024-09-12T13:52:24.762 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:52:24.763 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:52:24.763 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:52:24.763 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local force= 2024-09-12T13:52:24.763 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 mirror image promote mirror/image-primary6 2024-09-12T13:52:24.860 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary3: 2024-09-12T13:52:24.860 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 73afcdb2-236d-496e-9163-3c6ff6e95e23 2024-09-12T13:52:24.860 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+unknown 2024-09-12T13:52:24.860 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: remote image is not primary 2024-09-12T13:52:24.861 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:52:24.861 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:52:17' 2024-09-12T13:52:24.861 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:52:24.864 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+unknown 2024-09-12T13:52:24.865 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:52:24.869 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: remote image is not primary 2024-09-12T13:52:24.869 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:52:24.871 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:52:24.871 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:52:24.873 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:52:24.874 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:52:24.878 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:52:24.878 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:52:24.879 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:52:24.883 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:52:17' 2024-09-12T13:52:24.883 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary3 up+unknown '2024-09-12 13:52:17' 2024-09-12T13:52:24.883 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:52:24.883 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:52:24.883 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:52:24.883 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:52:24.883 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:52:17' 2024-09-12T13:52:24.883 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:52:24.884 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:52:24.884 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-09-12T13:52:24.999 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041934232mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary1f6ac15bd-c3ff-4c34-86f6-e9bbc701f22fup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:52:17image-primary1002633bcb-38b0-4c95-9ca5-a988ab7c0dd0up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:52:17image-primary28399b5a6-24d7-4c4c-9667-28e949d25fb3up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:52:17image-primary373afcdb2-236d-496e-9163-3c6ff6e95e23up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:52:17image-primary46025c1db-ec31-4451-96a8-922baeed4ef6up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:52:17image-primary56de1d859-a420-4abe-953c-031047e292ecup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:52:17image-primary688271fa2-1e10-4868-befd-15807c269565up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:52:17image-primary7ecec213c-dc11-4547-88c6-ded8df679ff6up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:52:17image-primary8901b32e6-a0c7-4be3-ada0-5f0083a2314cup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:52:17image-primary943c84ab2-a21e-441e-82d9-32f1b27926a0up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 13:52:17' 2024-09-12T13:52:25.000 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:52:25.000 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary3'\'']/last_update' 2024-09-12T13:52:25.004 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:52:17' 2024-09-12T13:52:25.005 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary3'\'']/state' 2024-09-12T13:52:25.010 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+unknown 2024-09-12T13:52:25.011 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:52:25.011 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+unknown 2024-09-12T13:52:25.013 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:52:25.015 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:52:25.015 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:52:25.016 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster2 mirror image-primary3 up+unknown 2024-09-12T13:52:25.016 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:52:25.016 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:52:25.016 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:52:25.016 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:52:25.016 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:52:25.016 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:52:25.016 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:52:25.016 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:52:25.240 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster2 mirror image-primary1 up+unknown '' '' 2024-09-12T13:52:25.240 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:52:25.240 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:52:25.240 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:52:25.240 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:52:25.241 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:52:25.241 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:52:25.241 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:52:25.241 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:52:25.241 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster2 mirror image status mirror/image-primary1 2024-09-12T13:52:25.278 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster2 mirror image-primary5 up+unknown '' '' 2024-09-12T13:52:25.278 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:52:25.278 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:52:25.279 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:52:25.279 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:52:25.279 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:52:25.279 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:52:25.279 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:52:25.279 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:52:25.279 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster2 mirror image status mirror/image-primary5 2024-09-12T13:52:25.298 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster2 mirror image-primary10 up+unknown '' '' 2024-09-12T13:52:25.298 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:52:25.298 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:52:25.298 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:52:25.298 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:52:25.298 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:52:25.298 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:52:25.298 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:52:25.299 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:52:25.299 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster2 mirror image status mirror/image-primary10 2024-09-12T13:52:25.314 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster2 mirror image-primary9 up+unknown '' '' 2024-09-12T13:52:25.314 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:52:25.314 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:52:25.315 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:52:25.315 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:52:25.315 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:52:25.315 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:52:25.315 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:52:25.316 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:52:25.316 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster2 mirror image status mirror/image-primary9 2024-09-12T13:52:25.340 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary1: 2024-09-12T13:52:25.340 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: f6ac15bd-c3ff-4c34-86f6-e9bbc701f22f 2024-09-12T13:52:25.340 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+unknown 2024-09-12T13:52:25.341 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: remote image is not primary 2024-09-12T13:52:25.341 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:52:25.341 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:52:17' 2024-09-12T13:52:25.341 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:52:25.344 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+unknown 2024-09-12T13:52:25.345 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:52:25.348 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: remote image is not primary 2024-09-12T13:52:25.349 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:52:25.351 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:52:25.352 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:52:25.354 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:52:25.355 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:52:25.358 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:52:25.358 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:52:25.360 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:52:25.367 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:52:17' 2024-09-12T13:52:25.367 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster2 mirror image-primary1 up+unknown '2024-09-12 13:52:17' 2024-09-12T13:52:25.367 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:52:25.367 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:52:25.368 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:52:25.368 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:52:25.368 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:52:17' 2024-09-12T13:52:25.368 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:52:25.368 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:52:25.368 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2024-09-12T13:52:25.382 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary5: 2024-09-12T13:52:25.382 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 6de1d859-a420-4abe-953c-031047e292ec 2024-09-12T13:52:25.382 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+unknown 2024-09-12T13:52:25.382 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: remote image is not primary 2024-09-12T13:52:25.382 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:52:25.382 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:52:17' 2024-09-12T13:52:25.382 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:52:25.386 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+unknown 2024-09-12T13:52:25.388 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:52:25.391 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: remote image is not primary 2024-09-12T13:52:25.392 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:52:25.393 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:52:25.393 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:52:25.396 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:52:25.397 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:52:25.399 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:52:25.400 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:52:25.401 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:52:25.406 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:52:17' 2024-09-12T13:52:25.406 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster2 mirror image-primary5 up+unknown '2024-09-12 13:52:17' 2024-09-12T13:52:25.406 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:52:25.406 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:52:25.407 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:52:25.407 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:52:25.407 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:52:17' 2024-09-12T13:52:25.407 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:52:25.407 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:52:25.407 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2024-09-12T13:52:25.429 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary10: 2024-09-12T13:52:25.429 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 02633bcb-38b0-4c95-9ca5-a988ab7c0dd0 2024-09-12T13:52:25.429 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+unknown 2024-09-12T13:52:25.429 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: remote image is not primary 2024-09-12T13:52:25.429 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:52:25.429 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:52:17' 2024-09-12T13:52:25.429 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:52:25.434 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+unknown 2024-09-12T13:52:25.435 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:52:25.440 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: remote image is not primary 2024-09-12T13:52:25.440 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary9: 2024-09-12T13:52:25.440 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 43c84ab2-a21e-441e-82d9-32f1b27926a0 2024-09-12T13:52:25.440 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+unknown 2024-09-12T13:52:25.440 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: remote image is not primary 2024-09-12T13:52:25.441 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:52:25.441 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:52:17' 2024-09-12T13:52:25.441 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:52:25.441 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:52:25.442 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:52:25.443 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:52:25.445 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+unknown 2024-09-12T13:52:25.446 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:52:25.447 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:52:25.447 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:52:25.450 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: remote image is not primary 2024-09-12T13:52:25.451 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:52:25.451 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:52:25.452 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:52:25.453 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:52:25.453 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:52:25.454 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:52:25.459 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:52:25.460 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:52:25.460 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:52:17' 2024-09-12T13:52:25.460 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster2 mirror image-primary10 up+unknown '2024-09-12 13:52:17' 2024-09-12T13:52:25.460 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:52:25.460 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:52:25.461 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:52:25.461 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:52:25.461 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:52:17' 2024-09-12T13:52:25.461 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:52:25.461 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:52:25.461 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2024-09-12T13:52:25.464 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:52:25.465 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:52:25.466 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:52:25.473 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:52:17' 2024-09-12T13:52:25.475 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster2 mirror image-primary9 up+unknown '2024-09-12 13:52:17' 2024-09-12T13:52:25.475 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:52:25.475 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:52:25.475 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:52:25.475 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:52:25.475 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:52:17' 2024-09-12T13:52:25.475 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:52:25.476 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:52:25.476 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2024-09-12T13:52:25.589 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041924231mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary1f6ac15bd-c3ff-4c34-86f6-e9bbc701f22fup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:52:17image-primary1002633bcb-38b0-4c95-9ca5-a988ab7c0dd0up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:52:17image-primary28399b5a6-24d7-4c4c-9667-28e949d25fb3up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:52:17image-primary373afcdb2-236d-496e-9163-3c6ff6e95e23up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:52:17image-primary46025c1db-ec31-4451-96a8-922baeed4ef6up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:52:17image-primary56de1d859-a420-4abe-953c-031047e292ecup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:52:17image-primary688271fa2-1e10-4868-befd-15807c269565up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:52:17image-primary7ecec213c-dc11-4547-88c6-ded8df679ff6up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:52:17image-primary8901b32e6-a0c7-4be3-ada0-5f0083a2314cup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:52:17image-primary943c84ab2-a21e-441e-82d9-32f1b27926a0up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:52:17' 2024-09-12T13:52:25.590 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:52:25.591 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary1'\'']/last_update' 2024-09-12T13:52:25.608 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:52:17' 2024-09-12T13:52:25.610 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary1'\'']/state' 2024-09-12T13:52:25.629 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+unknown 2024-09-12T13:52:25.630 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:52:25.631 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+unknown 2024-09-12T13:52:25.638 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:52:25.641 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Image promoted to primary 2024-09-12T13:52:25.642 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:52:25.642 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:52:25.642 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ promote_image cluster2 mirror image-primary1 2024-09-12T13:52:25.642 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:52:25.642 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:52:25.642 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:52:25.642 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local force= 2024-09-12T13:52:25.643 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 mirror image promote mirror/image-primary1 2024-09-12T13:52:25.659 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Image promoted to primary 2024-09-12T13:52:25.722 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \n\b\d ]] 2024-09-12T13:52:25.723 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \k\r\b\d ]] 2024-09-12T13:52:25.728 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo rbd --cluster cluster2 device map -t krbd mirror/image-primary6 2024-09-12T13:52:25.738 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \n\b\d ]] 2024-09-12T13:52:25.739 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \k\r\b\d ]] 2024-09-12T13:52:25.742 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo rbd --cluster cluster2 device map -t krbd mirror/image-primary7 2024-09-12T13:52:25.742 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041924231mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary1f6ac15bd-c3ff-4c34-86f6-e9bbc701f22fup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:52:17image-primary1002633bcb-38b0-4c95-9ca5-a988ab7c0dd0up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:52:17image-primary28399b5a6-24d7-4c4c-9667-28e949d25fb3up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:52:17image-primary373afcdb2-236d-496e-9163-3c6ff6e95e23up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:52:17image-primary46025c1db-ec31-4451-96a8-922baeed4ef6up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:52:17image-primary56de1d859-a420-4abe-953c-031047e292ecup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:52:17image-primary688271fa2-1e10-4868-befd-15807c269565up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:52:17image-primary7ecec213c-dc11-4547-88c6-ded8df679ff6up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:52:17image-primary8901b32e6-a0c7-4be3-ada0-5f0083a2314cup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:52:17image-primary943c84ab2-a21e-441e-82d9-32f1b27926a0up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:52:17' 2024-09-12T13:52:25.742 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:52:25.745 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary5'\'']/last_update' 2024-09-12T13:52:25.762 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:52:17' 2024-09-12T13:52:25.763 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary5'\'']/state' 2024-09-12T13:52:25.773 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+unknown 2024-09-12T13:52:25.777 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:52:25.777 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+unknown 2024-09-12T13:52:25.782 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:52:25.785 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:52:25.785 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:52:25.785 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ promote_image cluster2 mirror image-primary5 2024-09-12T13:52:25.785 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:52:25.785 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:52:25.785 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:52:25.785 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local force= 2024-09-12T13:52:25.785 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 mirror image promote mirror/image-primary5 2024-09-12T13:52:25.806 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041924231mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary1f6ac15bd-c3ff-4c34-86f6-e9bbc701f22fup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:52:17image-primary1002633bcb-38b0-4c95-9ca5-a988ab7c0dd0up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:52:17image-primary28399b5a6-24d7-4c4c-9667-28e949d25fb3up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:52:17image-primary373afcdb2-236d-496e-9163-3c6ff6e95e23up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:52:17image-primary46025c1db-ec31-4451-96a8-922baeed4ef6up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:52:17image-primary56de1d859-a420-4abe-953c-031047e292ecup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:52:17image-primary688271fa2-1e10-4868-befd-15807c269565up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:52:17image-primary7ecec213c-dc11-4547-88c6-ded8df679ff6up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:52:17image-primary8901b32e6-a0c7-4be3-ada0-5f0083a2314cup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:52:17image-primary943c84ab2-a21e-441e-82d9-32f1b27926a0up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:52:17' 2024-09-12T13:52:25.806 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:52:25.807 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary9'\'']/last_update' 2024-09-12T13:52:25.814 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041924231mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary1f6ac15bd-c3ff-4c34-86f6-e9bbc701f22fup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:52:17image-primary1002633bcb-38b0-4c95-9ca5-a988ab7c0dd0up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:52:17image-primary28399b5a6-24d7-4c4c-9667-28e949d25fb3up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:52:17image-primary373afcdb2-236d-496e-9163-3c6ff6e95e23up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:52:17image-primary46025c1db-ec31-4451-96a8-922baeed4ef6up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:52:17image-primary56de1d859-a420-4abe-953c-031047e292ecup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:52:17image-primary688271fa2-1e10-4868-befd-15807c269565up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:52:17image-primary7ecec213c-dc11-4547-88c6-ded8df679ff6up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:52:17image-primary8901b32e6-a0c7-4be3-ada0-5f0083a2314cup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:52:17image-primary943c84ab2-a21e-441e-82d9-32f1b27926a0up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:52:17' 2024-09-12T13:52:25.814 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:52:25.814 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:52:17' 2024-09-12T13:52:25.816 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary10'\'']/last_update' 2024-09-12T13:52:25.816 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary9'\'']/state' 2024-09-12T13:52:25.823 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+unknown 2024-09-12T13:52:25.823 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:52:17' 2024-09-12T13:52:25.824 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:52:25.824 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary10'\'']/state' 2024-09-12T13:52:25.825 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+unknown 2024-09-12T13:52:25.828 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:52:25.830 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:52:25.830 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:52:25.830 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ promote_image cluster2 mirror image-primary9 2024-09-12T13:52:25.830 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:52:25.830 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:52:25.831 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:52:25.831 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local force= 2024-09-12T13:52:25.831 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 mirror image promote mirror/image-primary9 2024-09-12T13:52:25.831 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+unknown 2024-09-12T13:52:25.834 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:52:25.835 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+unknown 2024-09-12T13:52:25.838 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:52:25.840 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:52:25.840 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:52:25.841 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ promote_image cluster2 mirror image-primary10 2024-09-12T13:52:25.841 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:52:25.841 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:52:25.841 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:52:25.841 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local force= 2024-09-12T13:52:25.841 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 mirror image promote mirror/image-primary10 2024-09-12T13:52:25.984 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ dev=/dev/rbd3 2024-09-12T13:52:25.987 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo md5sum /dev/rbd3 2024-09-12T13:52:25.987 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ awk '{print $1}' 2024-09-12T13:52:26.000 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ dev=/dev/rbd4 2024-09-12T13:52:26.002 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo md5sum /dev/rbd4 2024-09-12T13:52:26.002 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ awk '{print $1}' 2024-09-12T13:52:26.017 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster2 mirror image-primary3 up+unknown '' '' 2024-09-12T13:52:26.018 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:52:26.018 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:52:26.018 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:52:26.018 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:52:26.018 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:52:26.018 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:52:26.018 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:52:26.019 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:52:26.019 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster2 mirror image status mirror/image-primary3 2024-09-12T13:52:26.128 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary3: 2024-09-12T13:52:26.128 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 73afcdb2-236d-496e-9163-3c6ff6e95e23 2024-09-12T13:52:26.129 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+unknown 2024-09-12T13:52:26.129 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: remote image is not primary 2024-09-12T13:52:26.129 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:52:26.129 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:52:17' 2024-09-12T13:52:26.129 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:52:26.133 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+unknown 2024-09-12T13:52:26.134 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:52:26.138 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: remote image is not primary 2024-09-12T13:52:26.139 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:52:26.140 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:52:26.141 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:52:26.144 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:52:26.145 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:52:26.149 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:52:26.150 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:52:26.151 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:52:26.158 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:52:17' 2024-09-12T13:52:26.158 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster2 mirror image-primary3 up+unknown '2024-09-12 13:52:17' 2024-09-12T13:52:26.158 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:52:26.159 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:52:26.159 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:52:26.159 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:52:26.159 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:52:17' 2024-09-12T13:52:26.159 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:52:26.159 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:52:26.159 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2024-09-12T13:52:26.362 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041924231mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary1f6ac15bd-c3ff-4c34-86f6-e9bbc701f22fup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:52:17image-primary1002633bcb-38b0-4c95-9ca5-a988ab7c0dd0up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:52:17image-primary28399b5a6-24d7-4c4c-9667-28e949d25fb3up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:52:17image-primary373afcdb2-236d-496e-9163-3c6ff6e95e23up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:52:17image-primary46025c1db-ec31-4451-96a8-922baeed4ef6up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:52:17image-primary56de1d859-a420-4abe-953c-031047e292ecup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:52:17image-primary688271fa2-1e10-4868-befd-15807c269565up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:52:17image-primary7ecec213c-dc11-4547-88c6-ded8df679ff6up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:52:17image-primary8901b32e6-a0c7-4be3-ada0-5f0083a2314cup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:52:17image-primary943c84ab2-a21e-441e-82d9-32f1b27926a0up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 13:52:17' 2024-09-12T13:52:26.362 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:52:26.362 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary3'\'']/last_update' 2024-09-12T13:52:26.367 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:52:17' 2024-09-12T13:52:26.368 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary3'\'']/state' 2024-09-12T13:52:26.372 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+unknown 2024-09-12T13:52:26.372 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T13:52:26.373 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+unknown 2024-09-12T13:52:26.375 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T13:52:26.377 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:52:26.377 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:52:26.377 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ promote_image cluster2 mirror image-primary3 2024-09-12T13:52:26.377 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:52:26.377 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:52:26.378 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:52:26.378 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local force= 2024-09-12T13:52:26.378 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 mirror image promote mirror/image-primary3 2024-09-12T13:52:26.701 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Image promoted to primary 2024-09-12T13:52:26.741 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Image promoted to primary 2024-09-12T13:52:26.741 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Image promoted to primary 2024-09-12T13:52:26.783 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Image promoted to primary 2024-09-12T13:52:26.807 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \n\b\d ]] 2024-09-12T13:52:26.807 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \k\r\b\d ]] 2024-09-12T13:52:26.808 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo rbd --cluster cluster2 device map -t krbd mirror/image-primary1 2024-09-12T13:52:26.817 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \n\b\d ]] 2024-09-12T13:52:26.817 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \k\r\b\d ]] 2024-09-12T13:52:26.819 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo rbd --cluster cluster2 device map -t krbd mirror/image-primary3 2024-09-12T13:52:26.820 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \n\b\d ]] 2024-09-12T13:52:26.820 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \k\r\b\d ]] 2024-09-12T13:52:26.821 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo rbd --cluster cluster2 device map -t krbd mirror/image-primary5 2024-09-12T13:52:26.835 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Image promoted to primary 2024-09-12T13:52:26.861 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \n\b\d ]] 2024-09-12T13:52:26.861 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \k\r\b\d ]] 2024-09-12T13:52:26.866 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo rbd --cluster cluster2 device map -t krbd mirror/image-primary10 2024-09-12T13:52:26.931 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \n\b\d ]] 2024-09-12T13:52:26.931 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \k\r\b\d ]] 2024-09-12T13:52:26.934 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo rbd --cluster cluster2 device map -t krbd mirror/image-primary9 2024-09-12T13:52:27.210 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ dev=/dev/rbd6 2024-09-12T13:52:27.212 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo md5sum /dev/rbd6 2024-09-12T13:52:27.213 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ awk '{print $1}' 2024-09-12T13:52:27.243 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ dev=/dev/rbd5 2024-09-12T13:52:27.251 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo md5sum /dev/rbd5 2024-09-12T13:52:27.253 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ awk '{print $1}' 2024-09-12T13:52:27.257 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ dev=/dev/rbd7 2024-09-12T13:52:27.264 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ awk '{print $1}' 2024-09-12T13:52:27.266 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo md5sum /dev/rbd7 2024-09-12T13:52:27.275 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ dev=/dev/rbd8 2024-09-12T13:52:27.278 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo md5sum /dev/rbd8 2024-09-12T13:52:27.281 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ awk '{print $1}' 2024-09-12T13:52:27.367 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ dev=/dev/rbd9 2024-09-12T13:52:27.373 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ awk '{print $1}' 2024-09-12T13:52:27.373 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo md5sum /dev/rbd9 2024-09-12T13:53:10.098 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ promote_md5=92b616a2adf309704fb48b12236529cf 2024-09-12T13:53:10.099 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo rbd --cluster cluster2 device unmap -t krbd /dev/rbd1 2024-09-12T13:53:10.656 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ promote_md5=3e247a1608ce7b437a1d237414b07013 2024-09-12T13:53:10.656 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo rbd --cluster cluster2 device unmap -t krbd /dev/rbd0 2024-09-12T13:53:10.774 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ 92b616a2adf309704fb48b12236529cf != \9\2\b\6\1\6\a\2\a\d\f\3\0\9\7\0\4\f\b\4\8\b\1\2\2\3\6\5\2\9\c\f ]] 2024-09-12T13:53:10.940 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ 3e247a1608ce7b437a1d237414b07013 != \3\e\2\4\7\a\1\6\0\8\c\e\7\b\4\3\7\a\1\d\2\3\7\4\1\4\b\0\7\0\1\3 ]] 2024-09-12T13:53:18.409 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ promote_md5=04830618d390ed33d3109d44602a6361 2024-09-12T13:53:18.409 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo rbd --cluster cluster2 device unmap -t krbd /dev/rbd2 2024-09-12T13:53:18.593 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ 04830618d390ed33d3109d44602a6361 != \0\4\8\3\0\6\1\8\d\3\9\0\e\d\3\3\d\3\1\0\9\d\4\4\6\0\2\a\6\3\6\1 ]] 2024-09-12T13:53:19.079 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ promote_md5=ba8af2f22e67cc1ddd71ae354b21df71 2024-09-12T13:53:19.079 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo rbd --cluster cluster2 device unmap -t krbd /dev/rbd9 2024-09-12T13:53:19.086 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ promote_md5=1c914606982714846b5a2e92428803c2 2024-09-12T13:53:19.086 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo rbd --cluster cluster2 device unmap -t krbd /dev/rbd3 2024-09-12T13:53:19.163 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ promote_md5=7c7d2fca48d3311a4a8196f927c7b885 2024-09-12T13:53:19.163 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo rbd --cluster cluster2 device unmap -t krbd /dev/rbd7 2024-09-12T13:53:19.239 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ ba8af2f22e67cc1ddd71ae354b21df71 != \b\a\8\a\f\2\f\2\2\e\6\7\c\c\1\d\d\d\7\1\a\e\3\5\4\b\2\1\d\f\7\1 ]] 2024-09-12T13:53:19.287 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ 1c914606982714846b5a2e92428803c2 != \1\c\9\1\4\6\0\6\9\8\2\7\1\4\8\4\6\b\5\a\2\e\9\2\4\2\8\8\0\3\c\2 ]] 2024-09-12T13:53:19.321 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ 7c7d2fca48d3311a4a8196f927c7b885 != \7\c\7\d\2\f\c\a\4\8\d\3\3\1\1\a\4\a\8\1\9\6\f\9\2\7\c\7\b\8\8\5 ]] 2024-09-12T13:53:20.016 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ promote_md5=2c7fd5759f9be57aa5c65afcbc50f7a7 2024-09-12T13:53:20.016 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo rbd --cluster cluster2 device unmap -t krbd /dev/rbd6 2024-09-12T13:53:20.233 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ 2c7fd5759f9be57aa5c65afcbc50f7a7 != \2\c\7\f\d\5\7\5\9\f\9\b\e\5\7\a\a\5\c\6\5\a\f\c\b\c\5\0\f\7\a\7 ]] 2024-09-12T13:53:20.253 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ promote_md5=901ce58fda4db04b03205a885e099dc3 2024-09-12T13:53:20.253 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo rbd --cluster cluster2 device unmap -t krbd /dev/rbd4 2024-09-12T13:53:20.412 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ 901ce58fda4db04b03205a885e099dc3 != \9\0\1\c\e\5\8\f\d\a\4\d\b\0\4\b\0\3\2\0\5\a\8\8\5\e\0\9\9\d\c\3 ]] 2024-09-12T13:53:20.937 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ promote_md5=dd6e340287a75aa57b1278a1d1583ce1 2024-09-12T13:53:20.938 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo rbd --cluster cluster2 device unmap -t krbd /dev/rbd8 2024-09-12T13:53:21.026 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ dd6e340287a75aa57b1278a1d1583ce1 != \d\d\6\e\3\4\0\2\8\7\a\7\5\a\a\5\7\b\1\2\7\8\a\1\d\1\5\8\3\c\e\1 ]] 2024-09-12T13:53:21.291 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ promote_md5=25356f5bc6839635dcb6e534f32366c6 2024-09-12T13:53:21.291 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo rbd --cluster cluster2 device unmap -t krbd /dev/rbd5 2024-09-12T13:53:21.385 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ 25356f5bc6839635dcb6e534f32366c6 != \2\5\3\5\6\f\5\b\c\6\8\3\9\6\3\5\d\c\b\6\e\5\3\4\f\3\2\3\6\6\c\6 ]] 2024-09-12T13:53:21.385 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${COMPARE_PIDS[@]} 2024-09-12T13:53:21.385 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 113712 2024-09-12T13:53:21.385 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${COMPARE_PIDS[@]} 2024-09-12T13:53:21.385 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 113713 2024-09-12T13:53:21.386 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${COMPARE_PIDS[@]} 2024-09-12T13:53:21.386 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 113716 2024-09-12T13:53:21.386 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${COMPARE_PIDS[@]} 2024-09-12T13:53:21.386 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 113718 2024-09-12T13:53:21.386 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${COMPARE_PIDS[@]} 2024-09-12T13:53:21.386 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 113720 2024-09-12T13:53:21.386 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${COMPARE_PIDS[@]} 2024-09-12T13:53:21.386 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 113722 2024-09-12T13:53:21.386 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${COMPARE_PIDS[@]} 2024-09-12T13:53:21.386 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 113724 2024-09-12T13:53:21.386 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${COMPARE_PIDS[@]} 2024-09-12T13:53:21.386 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 113727 2024-09-12T13:53:21.386 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${COMPARE_PIDS[@]} 2024-09-12T13:53:21.387 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 113728 2024-09-12T13:53:21.387 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ (( RET != 0 )) 2024-09-12T13:53:21.387 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:53:21.387 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ IMG=image-primary1 2024-09-12T13:53:21.387 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_replaying_status_in_pool_dir cluster1 mirror image-primary1 2024-09-12T13:53:21.387 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:53:21.387 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:53:21.387 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:53:21.387 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' snapshot = journal ']' 2024-09-12T13:53:21.387 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary1 up+replaying 2024-09-12T13:53:21.387 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:53:21.387 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:53:21.387 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:53:21.388 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:53:21.388 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:53:21.388 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:53:21.388 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:53:21.388 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:53:22.396 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary1 up+replaying '' '' 2024-09-12T13:53:22.396 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:53:22.396 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:53:22.396 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:53:22.396 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:53:22.396 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:53:22.396 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:53:22.397 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:53:22.397 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:53:22.397 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary1 2024-09-12T13:53:22.476 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary1: 2024-09-12T13:53:22.476 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: f6ac15bd-c3ff-4c34-86f6-e9bbc701f22f 2024-09-12T13:53:22.476 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+replaying 2024-09-12T13:53:22.477 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: replaying, {"bytes_per_second":3607552.0,"bytes_per_snapshot":72151040.0,"last_snapshot_bytes":72151040,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726149146,"remote_snapshot_timestamp":1726149146,"replay_state":"idle"} 2024-09-12T13:53:22.477 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:53:22.477 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:53:17' 2024-09-12T13:53:22.477 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+replaying' 2024-09-12T13:53:22.479 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+replaying 2024-09-12T13:53:22.479 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:53:22.482 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: replaying, {"bytes_per_second":3607552.0,"bytes_per_snapshot":72151040.0,"last_snapshot_bytes":72151040,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726149146,"remote_snapshot_timestamp":1726149146,"replay_state":"idle"} 2024-09-12T13:53:22.482 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:53:22.483 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+replaying 2024-09-12T13:53:22.483 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:53:22.485 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+replaying 2024-09-12T13:53:22.485 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:53:22.488 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:53:22.488 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:53:22.489 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:53:22.493 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:53:17' 2024-09-12T13:53:22.493 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary1 up+replaying '2024-09-12 13:53:17' 2024-09-12T13:53:22.493 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:53:22.493 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:53:22.493 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:53:22.494 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:53:22.494 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:53:17' 2024-09-12T13:53:22.494 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:53:22.494 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:53:22.494 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-09-12T13:53:22.601 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041934232mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary1f6ac15bd-c3ff-4c34-86f6-e9bbc701f22fup+replayingreplaying, {"bytes_per_second":3607552.0,"bytes_per_snapshot":72151040.0,"last_snapshot_bytes":72151040,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726149146,"remote_snapshot_timestamp":1726149146,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:53:17image-primary1002633bcb-38b0-4c95-9ca5-a988ab7c0dd0up+replayingreplaying, {"bytes_per_second":3417292.8,"bytes_per_snapshot":68345856.0,"last_snapshot_bytes":68345856,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726149146,"remote_snapshot_timestamp":1726149146,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:53:17image-primary28399b5a6-24d7-4c4c-9667-28e949d25fb3up+replayingreplaying, {"bytes_per_second":3279462.4,"bytes_per_snapshot":65589248.0,"last_snapshot_bytes":65589248,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726149143,"remote_snapshot_timestamp":1726149143,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:53:17image-primary373afcdb2-236d-496e-9163-3c6ff6e95e23up+replayingreplaying, {"bytes_per_second":3099033.6,"bytes_per_snapshot":61980672.0,"last_snapshot_bytes":61980672,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726149146,"remote_snapshot_timestamp":1726149146,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:53:17image-primary46025c1db-ec31-4451-96a8-922baeed4ef6up+replayingreplaying, {"bytes_per_second":3309158.4,"bytes_per_snapshot":66183168.0,"last_snapshot_bytes":66183168,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726149140,"remote_snapshot_timestamp":1726149140,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:53:17image-primary56de1d859-a420-4abe-953c-031047e292ecup+replayingreplaying, {"bytes_per_second":3584000.0,"bytes_per_snapshot":71680000.0,"last_snapshot_bytes":71680000,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726149146,"remote_snapshot_timestamp":1726149146,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:53:17image-primary688271fa2-1e10-4868-befd-15807c269565up+replayingreplaying, {"bytes_per_second":3380224.0,"bytes_per_snapshot":67604480.0,"last_snapshot_bytes":67604480,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726149145,"remote_snapshot_timestamp":1726149145,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:53:17image-primary7ecec213c-dc11-4547-88c6-ded8df679ff6up+replayingreplaying, {"bytes_per_second":3395584.0,"bytes_per_snapshot":67911680.0,"last_snapshot_bytes":67911680,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726149145,"remote_snapshot_timestamp":1726149145,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:53:17image-primary8901b32e6-a0c7-4be3-ada0-5f0083a2314cup+replayingreplaying, {"bytes_per_second":3258368.0,"bytes_per_snapshot":65167360.0,"last_snapshot_bytes":65167360,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726149140,"remote_snapshot_timestamp":1726149140,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:53:17image-primary943c84ab2-a21e-441e-82d9-32f1b27926a0up+replayingreplaying, {"bytes_per_second":3307520.0,"bytes_per_snapshot":66150400.0,"last_snapshot_bytes":66150400,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726149146,"remote_snapshot_timestamp":1726149146,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:53:17' 2024-09-12T13:53:22.602 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:53:22.602 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary1'\'']/last_update' 2024-09-12T13:53:22.608 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:53:17' 2024-09-12T13:53:22.608 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary1'\'']/state' 2024-09-12T13:53:22.612 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+replaying 2024-09-12T13:53:22.613 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+replaying 2024-09-12T13:53:22.613 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+replaying 2024-09-12T13:53:22.615 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+replaying 2024-09-12T13:53:22.616 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:53:22.616 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:53:22.616 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ remove_image cluster2 mirror image-primary1 2024-09-12T13:53:22.616 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:53:22.617 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:53:22.617 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:53:22.617 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 snap purge mirror/image-primary1 2024-09-12T13:53:22.677 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 rm mirror/image-primary1 2024-09-12T13:53:22.820 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:53:22.820+0000 7f67e4800640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x56055b7ee000 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-09-12T13:53:22.827 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:53:22.820+0000 7f67eca00640 -1 rbd::mirror::ImageReplayer: 0x5605561acf00 [3/f6ac15bd-c3ff-4c34-86f6-e9bbc701f22f] operator(): start failed: (85) Interrupted system call should be restarted 2024-09-12T13:53:24.737 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-09-12T13:53:24.743 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_image_removal cluster1 mirror image-primary1 2024-09-12T13:53:24.743 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:53:24.743 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:53:24.743 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:53:24.743 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:53:24.743 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 ls mirror 2024-09-12T13:53:24.744 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep -wq image-primary1 2024-09-12T13:53:24.795 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:53:24.795 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rm -rf test-primary1 2024-09-12T13:53:24.810 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:53:24.810 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ IMG=image-primary2 2024-09-12T13:53:24.810 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_replaying_status_in_pool_dir cluster1 mirror image-primary2 2024-09-12T13:53:24.810 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:53:24.810 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:53:24.810 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:53:24.810 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' snapshot = journal ']' 2024-09-12T13:53:24.810 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary2 up+replaying 2024-09-12T13:53:24.810 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:53:24.810 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:53:24.811 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:53:24.811 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:53:24.811 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:53:24.811 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:53:24.811 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:53:24.811 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:53:25.812 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary2 up+replaying '' '' 2024-09-12T13:53:25.812 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:53:25.812 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:53:25.812 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:53:25.812 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:53:25.812 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:53:25.812 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:53:25.813 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:53:25.813 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:53:25.813 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary2 2024-09-12T13:53:25.895 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary2: 2024-09-12T13:53:25.895 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 8399b5a6-24d7-4c4c-9667-28e949d25fb3 2024-09-12T13:53:25.895 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+replaying 2024-09-12T13:53:25.895 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: replaying, {"bytes_per_second":3279462.4,"bytes_per_snapshot":65589248.0,"last_snapshot_bytes":65589248,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726149143,"remote_snapshot_timestamp":1726149143,"replay_state":"idle"} 2024-09-12T13:53:25.895 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:53:25.895 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:53:17' 2024-09-12T13:53:25.895 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+replaying' 2024-09-12T13:53:25.897 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+replaying 2024-09-12T13:53:25.898 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:53:25.901 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: replaying, {"bytes_per_second":3279462.4,"bytes_per_snapshot":65589248.0,"last_snapshot_bytes":65589248,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726149143,"remote_snapshot_timestamp":1726149143,"replay_state":"idle"} 2024-09-12T13:53:25.901 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:53:25.902 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+replaying 2024-09-12T13:53:25.902 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:53:25.904 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+replaying 2024-09-12T13:53:25.904 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:53:25.909 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:53:25.909 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:53:25.909 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:53:25.912 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:53:17' 2024-09-12T13:53:25.913 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary2 up+replaying '2024-09-12 13:53:17' 2024-09-12T13:53:25.913 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:53:25.913 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:53:25.913 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:53:25.913 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:53:25.913 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:53:17' 2024-09-12T13:53:25.913 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:53:25.914 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:53:25.914 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-09-12T13:53:26.024 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING941934232mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary1002633bcb-38b0-4c95-9ca5-a988ab7c0dd0up+replayingreplaying, {"bytes_per_second":3417292.8,"bytes_per_snapshot":68345856.0,"last_snapshot_bytes":68345856,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726149146,"remote_snapshot_timestamp":1726149146,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:53:17image-primary28399b5a6-24d7-4c4c-9667-28e949d25fb3up+replayingreplaying, {"bytes_per_second":3279462.4,"bytes_per_snapshot":65589248.0,"last_snapshot_bytes":65589248,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726149143,"remote_snapshot_timestamp":1726149143,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:53:17image-primary373afcdb2-236d-496e-9163-3c6ff6e95e23up+replayingreplaying, {"bytes_per_second":3099033.6,"bytes_per_snapshot":61980672.0,"last_snapshot_bytes":61980672,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726149146,"remote_snapshot_timestamp":1726149146,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:53:17image-primary46025c1db-ec31-4451-96a8-922baeed4ef6up+replayingreplaying, {"bytes_per_second":3309158.4,"bytes_per_snapshot":66183168.0,"last_snapshot_bytes":66183168,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726149140,"remote_snapshot_timestamp":1726149140,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:53:17image-primary56de1d859-a420-4abe-953c-031047e292ecup+replayingreplaying, {"bytes_per_second":3584000.0,"bytes_per_snapshot":71680000.0,"last_snapshot_bytes":71680000,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726149146,"remote_snapshot_timestamp":1726149146,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:53:17image-primary688271fa2-1e10-4868-befd-15807c269565up+replayingreplaying, {"bytes_per_second":3380224.0,"bytes_per_snapshot":67604480.0,"last_snapshot_bytes":67604480,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726149145,"remote_snapshot_timestamp":1726149145,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:53:17image-primary7ecec213c-dc11-4547-88c6-ded8df679ff6up+replayingreplaying, {"bytes_per_second":3395584.0,"bytes_per_snapshot":67911680.0,"last_snapshot_bytes":67911680,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726149145,"remote_snapshot_timestamp":1726149145,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:53:17image-primary8901b32e6-a0c7-4be3-ada0-5f0083a2314cup+replayingreplaying, {"bytes_per_second":3258368.0,"bytes_per_snapshot":65167360.0,"last_snapshot_bytes":65167360,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726149140,"remote_snapshot_timestamp":1726149140,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:53:17image-primary943c84ab2-a21e-441e-82d9-32f1b27926a0up+replayingreplaying, {"bytes_per_second":3307520.0,"bytes_per_snapshot":66150400.0,"last_snapshot_bytes":66150400,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726149146,"remote_snapshot_timestamp":1726149146,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:53:17' 2024-09-12T13:53:26.024 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:53:26.025 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary2'\'']/last_update' 2024-09-12T13:53:26.028 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:53:17' 2024-09-12T13:53:26.029 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary2'\'']/state' 2024-09-12T13:53:26.033 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+replaying 2024-09-12T13:53:26.034 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+replaying 2024-09-12T13:53:26.034 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+replaying 2024-09-12T13:53:26.036 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+replaying 2024-09-12T13:53:26.038 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:53:26.038 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:53:26.038 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ remove_image cluster2 mirror image-primary2 2024-09-12T13:53:26.038 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:53:26.038 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:53:26.038 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:53:26.038 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 snap purge mirror/image-primary2 2024-09-12T13:53:26.100 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 rm mirror/image-primary2 2024-09-12T13:53:26.247 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:53:26.247+0000 7f67e4800640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x56055b973a00 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-09-12T13:53:26.247 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:53:26.247+0000 7f67ed400640 -1 rbd::mirror::ImageReplayer: 0x560556cef180 [3/8399b5a6-24d7-4c4c-9667-28e949d25fb3] operator(): start failed: (85) Interrupted system call should be restarted 2024-09-12T13:53:27.744 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-09-12T13:53:27.750 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_image_removal cluster1 mirror image-primary2 2024-09-12T13:53:27.750 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:53:27.750 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:53:27.750 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:53:27.751 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:53:27.751 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 ls mirror 2024-09-12T13:53:27.751 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep -wq image-primary2 2024-09-12T13:53:27.801 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:53:27.801 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rm -rf test-primary2 2024-09-12T13:53:27.803 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:53:27.803 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ IMG=image-primary3 2024-09-12T13:53:27.804 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_replaying_status_in_pool_dir cluster1 mirror image-primary3 2024-09-12T13:53:27.804 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:53:27.804 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:53:27.804 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:53:27.804 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' snapshot = journal ']' 2024-09-12T13:53:27.804 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary3 up+replaying 2024-09-12T13:53:27.804 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:53:27.804 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:53:27.804 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:53:27.804 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:53:27.804 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:53:27.804 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:53:27.805 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:53:27.805 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:53:28.806 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary3 up+replaying '' '' 2024-09-12T13:53:28.806 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:53:28.806 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:53:28.806 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:53:28.806 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:53:28.806 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:53:28.806 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:53:28.806 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:53:28.807 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:53:28.807 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary3 2024-09-12T13:53:28.887 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary3: 2024-09-12T13:53:28.888 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 73afcdb2-236d-496e-9163-3c6ff6e95e23 2024-09-12T13:53:28.888 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+replaying 2024-09-12T13:53:28.888 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: replaying, {"bytes_per_second":3099033.6,"bytes_per_snapshot":61980672.0,"last_snapshot_bytes":61980672,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726149146,"remote_snapshot_timestamp":1726149146,"replay_state":"idle"} 2024-09-12T13:53:28.888 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:53:28.888 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:53:17' 2024-09-12T13:53:28.888 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+replaying' 2024-09-12T13:53:28.890 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+replaying 2024-09-12T13:53:28.890 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:53:28.893 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: replaying, {"bytes_per_second":3099033.6,"bytes_per_snapshot":61980672.0,"last_snapshot_bytes":61980672,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726149146,"remote_snapshot_timestamp":1726149146,"replay_state":"idle"} 2024-09-12T13:53:28.893 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:53:28.893 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+replaying 2024-09-12T13:53:28.894 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:53:28.896 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+replaying 2024-09-12T13:53:28.896 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:53:28.898 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:53:28.899 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:53:28.900 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:53:28.903 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:53:17' 2024-09-12T13:53:28.904 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary3 up+replaying '2024-09-12 13:53:17' 2024-09-12T13:53:28.904 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:53:28.904 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:53:28.904 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:53:28.904 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:53:28.904 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:53:17' 2024-09-12T13:53:28.904 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:53:28.904 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:53:28.905 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-09-12T13:53:29.012 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING841934232mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary1002633bcb-38b0-4c95-9ca5-a988ab7c0dd0up+replayingreplaying, {"bytes_per_second":3417292.8,"bytes_per_snapshot":68345856.0,"last_snapshot_bytes":68345856,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726149146,"remote_snapshot_timestamp":1726149146,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:53:17image-primary373afcdb2-236d-496e-9163-3c6ff6e95e23up+replayingreplaying, {"bytes_per_second":3099033.6,"bytes_per_snapshot":61980672.0,"last_snapshot_bytes":61980672,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726149146,"remote_snapshot_timestamp":1726149146,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:53:17image-primary46025c1db-ec31-4451-96a8-922baeed4ef6up+replayingreplaying, {"bytes_per_second":3309158.4,"bytes_per_snapshot":66183168.0,"last_snapshot_bytes":66183168,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726149140,"remote_snapshot_timestamp":1726149140,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:53:17image-primary56de1d859-a420-4abe-953c-031047e292ecup+replayingreplaying, {"bytes_per_second":3584000.0,"bytes_per_snapshot":71680000.0,"last_snapshot_bytes":71680000,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726149146,"remote_snapshot_timestamp":1726149146,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:53:17image-primary688271fa2-1e10-4868-befd-15807c269565up+replayingreplaying, {"bytes_per_second":3380224.0,"bytes_per_snapshot":67604480.0,"last_snapshot_bytes":67604480,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726149145,"remote_snapshot_timestamp":1726149145,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:53:17image-primary7ecec213c-dc11-4547-88c6-ded8df679ff6up+replayingreplaying, {"bytes_per_second":3395584.0,"bytes_per_snapshot":67911680.0,"last_snapshot_bytes":67911680,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726149145,"remote_snapshot_timestamp":1726149145,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:53:17image-primary8901b32e6-a0c7-4be3-ada0-5f0083a2314cup+replayingreplaying, {"bytes_per_second":3258368.0,"bytes_per_snapshot":65167360.0,"last_snapshot_bytes":65167360,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726149140,"remote_snapshot_timestamp":1726149140,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:53:17image-primary943c84ab2-a21e-441e-82d9-32f1b27926a0up+replayingreplaying, {"bytes_per_second":3307520.0,"bytes_per_snapshot":66150400.0,"last_snapshot_bytes":66150400,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726149146,"remote_snapshot_timestamp":1726149146,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:53:17' 2024-09-12T13:53:29.012 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:53:29.013 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary3'\'']/last_update' 2024-09-12T13:53:29.016 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:53:17' 2024-09-12T13:53:29.017 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary3'\'']/state' 2024-09-12T13:53:29.021 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+replaying 2024-09-12T13:53:29.022 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+replaying 2024-09-12T13:53:29.022 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+replaying 2024-09-12T13:53:29.024 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+replaying 2024-09-12T13:53:29.024 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:53:29.025 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:53:29.025 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ remove_image cluster2 mirror image-primary3 2024-09-12T13:53:29.025 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:53:29.025 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:53:29.025 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:53:29.025 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 snap purge mirror/image-primary3 2024-09-12T13:53:29.084 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 rm mirror/image-primary3 2024-09-12T13:53:29.227 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:53:29.227+0000 7f67e4800640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x56055be69520 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-09-12T13:53:29.227 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:53:29.227+0000 7f67ed400640 -1 rbd::mirror::ImageReplayer: 0x5605561ad680 [3/73afcdb2-236d-496e-9163-3c6ff6e95e23] operator(): start failed: (85) Interrupted system call should be restarted 2024-09-12T13:53:30.425 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-09-12T13:53:30.431 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_image_removal cluster1 mirror image-primary3 2024-09-12T13:53:30.431 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:53:30.431 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:53:30.431 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:53:30.431 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:53:30.432 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 ls mirror 2024-09-12T13:53:30.432 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep -wq image-primary3 2024-09-12T13:53:30.488 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:53:30.488 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rm -rf test-primary3 2024-09-12T13:53:30.491 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:53:30.491 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ IMG=image-primary4 2024-09-12T13:53:30.491 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_replaying_status_in_pool_dir cluster1 mirror image-primary4 2024-09-12T13:53:30.491 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:53:30.491 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:53:30.491 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:53:30.491 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' snapshot = journal ']' 2024-09-12T13:53:30.491 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary4 up+replaying 2024-09-12T13:53:30.491 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:53:30.492 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:53:30.492 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:53:30.492 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:53:30.492 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:53:30.492 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:53:30.492 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:53:30.492 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:53:31.493 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary4 up+replaying '' '' 2024-09-12T13:53:31.494 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:53:31.494 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:53:31.494 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:53:31.494 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:53:31.494 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:53:31.494 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:53:31.494 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:53:31.495 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:53:31.495 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary4 2024-09-12T13:53:31.563 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary4: 2024-09-12T13:53:31.563 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 6025c1db-ec31-4451-96a8-922baeed4ef6 2024-09-12T13:53:31.563 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+replaying 2024-09-12T13:53:31.563 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: replaying, {"bytes_per_second":3309158.4,"bytes_per_snapshot":66183168.0,"last_snapshot_bytes":66183168,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726149140,"remote_snapshot_timestamp":1726149140,"replay_state":"idle"} 2024-09-12T13:53:31.563 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:53:31.563 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:53:17' 2024-09-12T13:53:31.563 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+replaying' 2024-09-12T13:53:31.565 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+replaying 2024-09-12T13:53:31.565 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:53:31.568 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: replaying, {"bytes_per_second":3309158.4,"bytes_per_snapshot":66183168.0,"last_snapshot_bytes":66183168,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726149140,"remote_snapshot_timestamp":1726149140,"replay_state":"idle"} 2024-09-12T13:53:31.568 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:53:31.569 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+replaying 2024-09-12T13:53:31.569 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:53:31.571 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+replaying 2024-09-12T13:53:31.571 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:53:31.574 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:53:31.574 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:53:31.575 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:53:31.579 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:53:17' 2024-09-12T13:53:31.579 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary4 up+replaying '2024-09-12 13:53:17' 2024-09-12T13:53:31.579 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:53:31.579 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:53:31.579 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:53:31.579 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:53:31.579 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:53:17' 2024-09-12T13:53:31.579 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:53:31.580 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:53:31.580 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-09-12T13:53:31.677 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING741934232mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary1002633bcb-38b0-4c95-9ca5-a988ab7c0dd0up+replayingreplaying, {"bytes_per_second":3417292.8,"bytes_per_snapshot":68345856.0,"last_snapshot_bytes":68345856,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726149146,"remote_snapshot_timestamp":1726149146,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:53:17image-primary46025c1db-ec31-4451-96a8-922baeed4ef6up+replayingreplaying, {"bytes_per_second":3309158.4,"bytes_per_snapshot":66183168.0,"last_snapshot_bytes":66183168,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726149140,"remote_snapshot_timestamp":1726149140,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:53:17image-primary56de1d859-a420-4abe-953c-031047e292ecup+replayingreplaying, {"bytes_per_second":3584000.0,"bytes_per_snapshot":71680000.0,"last_snapshot_bytes":71680000,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726149146,"remote_snapshot_timestamp":1726149146,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:53:17image-primary688271fa2-1e10-4868-befd-15807c269565up+replayingreplaying, {"bytes_per_second":3380224.0,"bytes_per_snapshot":67604480.0,"last_snapshot_bytes":67604480,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726149145,"remote_snapshot_timestamp":1726149145,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:53:17image-primary7ecec213c-dc11-4547-88c6-ded8df679ff6up+replayingreplaying, {"bytes_per_second":3395584.0,"bytes_per_snapshot":67911680.0,"last_snapshot_bytes":67911680,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726149145,"remote_snapshot_timestamp":1726149145,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:53:17image-primary8901b32e6-a0c7-4be3-ada0-5f0083a2314cup+replayingreplaying, {"bytes_per_second":3258368.0,"bytes_per_snapshot":65167360.0,"last_snapshot_bytes":65167360,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726149140,"remote_snapshot_timestamp":1726149140,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:53:17image-primary943c84ab2-a21e-441e-82d9-32f1b27926a0up+replayingreplaying, {"bytes_per_second":3307520.0,"bytes_per_snapshot":66150400.0,"last_snapshot_bytes":66150400,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726149146,"remote_snapshot_timestamp":1726149146,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:53:17' 2024-09-12T13:53:31.677 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:53:31.678 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary4'\'']/last_update' 2024-09-12T13:53:31.682 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:53:17' 2024-09-12T13:53:31.682 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary4'\'']/state' 2024-09-12T13:53:31.686 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+replaying 2024-09-12T13:53:31.687 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+replaying 2024-09-12T13:53:31.687 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+replaying 2024-09-12T13:53:31.689 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+replaying 2024-09-12T13:53:31.690 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:53:31.690 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:53:31.690 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ remove_image cluster2 mirror image-primary4 2024-09-12T13:53:31.690 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:53:31.690 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:53:31.690 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:53:31.690 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 snap purge mirror/image-primary4 2024-09-12T13:53:31.752 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 rm mirror/image-primary4 2024-09-12T13:53:31.893 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:53:31.893+0000 7f67e5200640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x56055ba3e340 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-09-12T13:53:31.893 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:53:31.893+0000 7f67ed400640 -1 rbd::mirror::ImageReplayer: 0x5605561ac000 [3/6025c1db-ec31-4451-96a8-922baeed4ef6] operator(): start failed: (85) Interrupted system call should be restarted 2024-09-12T13:53:33.768 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-09-12T13:53:33.774 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_image_removal cluster1 mirror image-primary4 2024-09-12T13:53:33.774 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:53:33.774 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:53:33.774 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:53:33.774 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:53:33.775 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 ls mirror 2024-09-12T13:53:33.775 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep -wq image-primary4 2024-09-12T13:53:33.824 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:53:33.824 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rm -rf test-primary4 2024-09-12T13:53:33.827 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:53:33.827 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ IMG=image-primary5 2024-09-12T13:53:33.827 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_replaying_status_in_pool_dir cluster1 mirror image-primary5 2024-09-12T13:53:33.827 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:53:33.827 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:53:33.827 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:53:33.827 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' snapshot = journal ']' 2024-09-12T13:53:33.827 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary5 up+replaying 2024-09-12T13:53:33.827 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:53:33.827 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:53:33.828 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:53:33.828 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:53:33.828 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:53:33.828 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:53:33.828 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:53:33.828 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:53:34.829 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary5 up+replaying '' '' 2024-09-12T13:53:34.829 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:53:34.829 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:53:34.829 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:53:34.830 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:53:34.830 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:53:34.830 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:53:34.830 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:53:34.830 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:53:34.830 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary5 2024-09-12T13:53:34.911 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary5: 2024-09-12T13:53:34.912 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 6de1d859-a420-4abe-953c-031047e292ec 2024-09-12T13:53:34.912 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+replaying 2024-09-12T13:53:34.912 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: replaying, {"bytes_per_second":3584000.0,"bytes_per_snapshot":71680000.0,"last_snapshot_bytes":71680000,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726149146,"remote_snapshot_timestamp":1726149146,"replay_state":"idle"} 2024-09-12T13:53:34.912 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:53:34.912 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:53:17' 2024-09-12T13:53:34.912 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+replaying' 2024-09-12T13:53:34.913 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+replaying 2024-09-12T13:53:34.913 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:53:34.916 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: replaying, {"bytes_per_second":3584000.0,"bytes_per_snapshot":71680000.0,"last_snapshot_bytes":71680000,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726149146,"remote_snapshot_timestamp":1726149146,"replay_state":"idle"} 2024-09-12T13:53:34.916 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:53:34.917 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+replaying 2024-09-12T13:53:34.917 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:53:34.919 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+replaying 2024-09-12T13:53:34.919 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:53:34.922 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:53:34.923 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:53:34.923 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:53:34.927 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:53:17' 2024-09-12T13:53:34.927 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary5 up+replaying '2024-09-12 13:53:17' 2024-09-12T13:53:34.927 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:53:34.927 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:53:34.928 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:53:34.928 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:53:34.928 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:53:17' 2024-09-12T13:53:34.928 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:53:34.928 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:53:34.928 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-09-12T13:53:35.025 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING641934232mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary1002633bcb-38b0-4c95-9ca5-a988ab7c0dd0up+replayingreplaying, {"bytes_per_second":3417292.8,"bytes_per_snapshot":68345856.0,"last_snapshot_bytes":68345856,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726149146,"remote_snapshot_timestamp":1726149146,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:53:17image-primary56de1d859-a420-4abe-953c-031047e292ecup+replayingreplaying, {"bytes_per_second":3584000.0,"bytes_per_snapshot":71680000.0,"last_snapshot_bytes":71680000,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726149146,"remote_snapshot_timestamp":1726149146,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:53:17image-primary688271fa2-1e10-4868-befd-15807c269565up+replayingreplaying, {"bytes_per_second":3380224.0,"bytes_per_snapshot":67604480.0,"last_snapshot_bytes":67604480,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726149145,"remote_snapshot_timestamp":1726149145,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:53:17image-primary7ecec213c-dc11-4547-88c6-ded8df679ff6up+replayingreplaying, {"bytes_per_second":3395584.0,"bytes_per_snapshot":67911680.0,"last_snapshot_bytes":67911680,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726149145,"remote_snapshot_timestamp":1726149145,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:53:17image-primary8901b32e6-a0c7-4be3-ada0-5f0083a2314cup+replayingreplaying, {"bytes_per_second":3258368.0,"bytes_per_snapshot":65167360.0,"last_snapshot_bytes":65167360,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726149140,"remote_snapshot_timestamp":1726149140,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:53:17image-primary943c84ab2-a21e-441e-82d9-32f1b27926a0up+replayingreplaying, {"bytes_per_second":3307520.0,"bytes_per_snapshot":66150400.0,"last_snapshot_bytes":66150400,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726149146,"remote_snapshot_timestamp":1726149146,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:53:17' 2024-09-12T13:53:35.025 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:53:35.026 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary5'\'']/last_update' 2024-09-12T13:53:35.029 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:53:17' 2024-09-12T13:53:35.030 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary5'\'']/state' 2024-09-12T13:53:35.034 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+replaying 2024-09-12T13:53:35.034 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+replaying 2024-09-12T13:53:35.035 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+replaying 2024-09-12T13:53:35.037 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+replaying 2024-09-12T13:53:35.037 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:53:35.038 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:53:35.038 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ remove_image cluster2 mirror image-primary5 2024-09-12T13:53:35.038 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:53:35.038 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:53:35.038 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:53:35.038 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 snap purge mirror/image-primary5 2024-09-12T13:53:35.098 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 rm mirror/image-primary5 2024-09-12T13:53:35.265 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:53:35.264+0000 7f67e5200640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x56055ba3eea0 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-09-12T13:53:35.265 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:53:35.264+0000 7f67eca00640 -1 rbd::mirror::ImageReplayer: 0x56055c198500 [3/6de1d859-a420-4abe-953c-031047e292ec] operator(): start failed: (85) Interrupted system call should be restarted 2024-09-12T13:53:36.416 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-09-12T13:53:36.422 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_image_removal cluster1 mirror image-primary5 2024-09-12T13:53:36.423 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:53:36.423 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:53:36.423 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:53:36.423 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:53:36.423 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 ls mirror 2024-09-12T13:53:36.424 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep -wq image-primary5 2024-09-12T13:53:36.472 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:53:36.472 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rm -rf test-primary5 2024-09-12T13:53:36.475 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:53:36.475 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ IMG=image-primary6 2024-09-12T13:53:36.475 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_replaying_status_in_pool_dir cluster1 mirror image-primary6 2024-09-12T13:53:36.475 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:53:36.475 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:53:36.475 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:53:36.475 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' snapshot = journal ']' 2024-09-12T13:53:36.476 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary6 up+replaying 2024-09-12T13:53:36.476 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:53:36.476 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:53:36.476 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:53:36.476 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:53:36.476 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:53:36.476 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:53:36.476 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:53:36.476 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:53:37.477 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary6 up+replaying '' '' 2024-09-12T13:53:37.477 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:53:37.477 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:53:37.477 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:53:37.477 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:53:37.478 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:53:37.478 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:53:37.478 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:53:37.478 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:53:37.478 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary6 2024-09-12T13:53:37.557 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary6: 2024-09-12T13:53:37.558 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 88271fa2-1e10-4868-befd-15807c269565 2024-09-12T13:53:37.558 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+replaying 2024-09-12T13:53:37.558 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: replaying, {"bytes_per_second":3380224.0,"bytes_per_snapshot":67604480.0,"last_snapshot_bytes":67604480,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726149145,"remote_snapshot_timestamp":1726149145,"replay_state":"idle"} 2024-09-12T13:53:37.558 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:53:37.558 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:53:17' 2024-09-12T13:53:37.558 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+replaying' 2024-09-12T13:53:37.560 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+replaying 2024-09-12T13:53:37.560 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:53:37.563 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: replaying, {"bytes_per_second":3380224.0,"bytes_per_snapshot":67604480.0,"last_snapshot_bytes":67604480,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726149145,"remote_snapshot_timestamp":1726149145,"replay_state":"idle"} 2024-09-12T13:53:37.563 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:53:37.564 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+replaying 2024-09-12T13:53:37.564 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:53:37.566 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+replaying 2024-09-12T13:53:37.566 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:53:37.569 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:53:37.569 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:53:37.570 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:53:37.574 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:53:17' 2024-09-12T13:53:37.574 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary6 up+replaying '2024-09-12 13:53:17' 2024-09-12T13:53:37.574 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:53:37.574 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:53:37.574 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:53:37.574 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:53:37.575 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:53:17' 2024-09-12T13:53:37.575 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:53:37.575 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:53:37.575 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-09-12T13:53:37.668 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING541934232mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary1002633bcb-38b0-4c95-9ca5-a988ab7c0dd0up+replayingreplaying, {"bytes_per_second":3417292.8,"bytes_per_snapshot":68345856.0,"last_snapshot_bytes":68345856,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726149146,"remote_snapshot_timestamp":1726149146,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:53:17image-primary688271fa2-1e10-4868-befd-15807c269565up+replayingreplaying, {"bytes_per_second":3380224.0,"bytes_per_snapshot":67604480.0,"last_snapshot_bytes":67604480,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726149145,"remote_snapshot_timestamp":1726149145,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:53:17image-primary7ecec213c-dc11-4547-88c6-ded8df679ff6up+replayingreplaying, {"bytes_per_second":3395584.0,"bytes_per_snapshot":67911680.0,"last_snapshot_bytes":67911680,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726149145,"remote_snapshot_timestamp":1726149145,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:53:17image-primary8901b32e6-a0c7-4be3-ada0-5f0083a2314cup+replayingreplaying, {"bytes_per_second":3258368.0,"bytes_per_snapshot":65167360.0,"last_snapshot_bytes":65167360,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726149140,"remote_snapshot_timestamp":1726149140,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:53:17image-primary943c84ab2-a21e-441e-82d9-32f1b27926a0up+replayingreplaying, {"bytes_per_second":3307520.0,"bytes_per_snapshot":66150400.0,"last_snapshot_bytes":66150400,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726149146,"remote_snapshot_timestamp":1726149146,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:53:17' 2024-09-12T13:53:37.669 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:53:37.669 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary6'\'']/last_update' 2024-09-12T13:53:37.673 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:53:17' 2024-09-12T13:53:37.674 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary6'\'']/state' 2024-09-12T13:53:37.678 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+replaying 2024-09-12T13:53:37.678 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+replaying 2024-09-12T13:53:37.678 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+replaying 2024-09-12T13:53:37.680 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+replaying 2024-09-12T13:53:37.681 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:53:37.681 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:53:37.681 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ remove_image cluster2 mirror image-primary6 2024-09-12T13:53:37.681 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:53:37.681 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:53:37.681 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:53:37.681 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 snap purge mirror/image-primary6 2024-09-12T13:53:37.741 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 rm mirror/image-primary6 2024-09-12T13:53:37.888 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:53:37.888+0000 7f67e4800640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x560556ddd6c0 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-09-12T13:53:37.888 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:53:37.888+0000 7f67eca00640 -1 rbd::mirror::ImageReplayer: 0x56055c199180 [3/88271fa2-1e10-4868-befd-15807c269565] operator(): start failed: (85) Interrupted system call should be restarted 2024-09-12T13:53:39.409 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-09-12T13:53:39.415 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_image_removal cluster1 mirror image-primary6 2024-09-12T13:53:39.415 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:53:39.415 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:53:39.415 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:53:39.415 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:53:39.416 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 ls mirror 2024-09-12T13:53:39.416 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep -wq image-primary6 2024-09-12T13:53:39.471 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:53:39.471 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rm -rf test-primary6 2024-09-12T13:53:39.474 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:53:39.474 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ IMG=image-primary7 2024-09-12T13:53:39.474 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_replaying_status_in_pool_dir cluster1 mirror image-primary7 2024-09-12T13:53:39.474 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:53:39.474 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:53:39.474 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:53:39.475 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' snapshot = journal ']' 2024-09-12T13:53:39.475 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary7 up+replaying 2024-09-12T13:53:39.475 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:53:39.475 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:53:39.475 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:53:39.475 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:53:39.475 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:53:39.475 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:53:39.475 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:53:39.475 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:53:40.476 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary7 up+replaying '' '' 2024-09-12T13:53:40.477 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:53:40.477 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:53:40.477 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:53:40.477 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:53:40.477 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:53:40.477 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:53:40.477 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:53:40.477 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:53:40.478 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary7 2024-09-12T13:53:40.548 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary7: 2024-09-12T13:53:40.548 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: ecec213c-dc11-4547-88c6-ded8df679ff6 2024-09-12T13:53:40.548 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+replaying 2024-09-12T13:53:40.548 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: replaying, {"bytes_per_second":3395584.0,"bytes_per_snapshot":67911680.0,"last_snapshot_bytes":67911680,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726149145,"remote_snapshot_timestamp":1726149145,"replay_state":"idle"} 2024-09-12T13:53:40.548 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:53:40.548 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:53:17' 2024-09-12T13:53:40.548 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+replaying' 2024-09-12T13:53:40.550 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+replaying 2024-09-12T13:53:40.550 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:53:40.553 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: replaying, {"bytes_per_second":3395584.0,"bytes_per_snapshot":67911680.0,"last_snapshot_bytes":67911680,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726149145,"remote_snapshot_timestamp":1726149145,"replay_state":"idle"} 2024-09-12T13:53:40.553 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:53:40.554 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+replaying 2024-09-12T13:53:40.554 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:53:40.556 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+replaying 2024-09-12T13:53:40.556 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:53:40.559 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:53:40.559 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:53:40.560 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:53:40.564 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:53:17' 2024-09-12T13:53:40.564 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary7 up+replaying '2024-09-12 13:53:17' 2024-09-12T13:53:40.564 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:53:40.564 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:53:40.564 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:53:40.564 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:53:40.564 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:53:17' 2024-09-12T13:53:40.564 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:53:40.565 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:53:40.565 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-09-12T13:53:40.649 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING441934232mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary1002633bcb-38b0-4c95-9ca5-a988ab7c0dd0up+replayingreplaying, {"bytes_per_second":3417292.8,"bytes_per_snapshot":68345856.0,"last_snapshot_bytes":68345856,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726149146,"remote_snapshot_timestamp":1726149146,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:53:17image-primary7ecec213c-dc11-4547-88c6-ded8df679ff6up+replayingreplaying, {"bytes_per_second":3395584.0,"bytes_per_snapshot":67911680.0,"last_snapshot_bytes":67911680,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726149145,"remote_snapshot_timestamp":1726149145,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:53:17image-primary8901b32e6-a0c7-4be3-ada0-5f0083a2314cup+replayingreplaying, {"bytes_per_second":3258368.0,"bytes_per_snapshot":65167360.0,"last_snapshot_bytes":65167360,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726149140,"remote_snapshot_timestamp":1726149140,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:53:17image-primary943c84ab2-a21e-441e-82d9-32f1b27926a0up+replayingreplaying, {"bytes_per_second":3307520.0,"bytes_per_snapshot":66150400.0,"last_snapshot_bytes":66150400,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726149146,"remote_snapshot_timestamp":1726149146,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:53:17' 2024-09-12T13:53:40.649 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:53:40.650 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary7'\'']/last_update' 2024-09-12T13:53:40.654 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:53:17' 2024-09-12T13:53:40.654 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary7'\'']/state' 2024-09-12T13:53:40.658 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+replaying 2024-09-12T13:53:40.659 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+replaying 2024-09-12T13:53:40.659 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+replaying 2024-09-12T13:53:40.661 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+replaying 2024-09-12T13:53:40.662 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:53:40.662 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:53:40.662 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ remove_image cluster2 mirror image-primary7 2024-09-12T13:53:40.662 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:53:40.662 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:53:40.662 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:53:40.662 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 snap purge mirror/image-primary7 2024-09-12T13:53:40.729 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 rm mirror/image-primary7 2024-09-12T13:53:40.869 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:53:40.869+0000 7f67e4800640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x560556ddd380 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-09-12T13:53:40.870 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:53:40.869+0000 7f67eca00640 -1 rbd::mirror::ImageReplayer: 0x56055c198780 [3/ecec213c-dc11-4547-88c6-ded8df679ff6] operator(): start failed: (85) Interrupted system call should be restarted 2024-09-12T13:53:42.417 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-09-12T13:53:42.424 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_image_removal cluster1 mirror image-primary7 2024-09-12T13:53:42.424 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:53:42.424 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:53:42.424 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:53:42.424 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:53:42.424 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 ls mirror 2024-09-12T13:53:42.425 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep -wq image-primary7 2024-09-12T13:53:42.474 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:53:42.474 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rm -rf test-primary7 2024-09-12T13:53:42.476 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:53:42.476 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ IMG=image-primary8 2024-09-12T13:53:42.476 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_replaying_status_in_pool_dir cluster1 mirror image-primary8 2024-09-12T13:53:42.476 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:53:42.476 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:53:42.477 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:53:42.477 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' snapshot = journal ']' 2024-09-12T13:53:42.477 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary8 up+replaying 2024-09-12T13:53:42.477 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:53:42.477 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:53:42.477 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:53:42.477 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:53:42.477 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:53:42.477 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:53:42.477 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:53:42.477 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:53:43.478 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary8 up+replaying '' '' 2024-09-12T13:53:43.478 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:53:43.479 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:53:43.479 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:53:43.479 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:53:43.479 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:53:43.479 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:53:43.479 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:53:43.479 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:53:43.479 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary8 2024-09-12T13:53:43.558 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary8: 2024-09-12T13:53:43.558 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 901b32e6-a0c7-4be3-ada0-5f0083a2314c 2024-09-12T13:53:43.558 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+replaying 2024-09-12T13:53:43.558 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: replaying, {"bytes_per_second":3258368.0,"bytes_per_snapshot":65167360.0,"last_snapshot_bytes":65167360,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726149140,"remote_snapshot_timestamp":1726149140,"replay_state":"idle"} 2024-09-12T13:53:43.558 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:53:43.558 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:53:17' 2024-09-12T13:53:43.558 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+replaying' 2024-09-12T13:53:43.560 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+replaying 2024-09-12T13:53:43.560 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:53:43.563 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: replaying, {"bytes_per_second":3258368.0,"bytes_per_snapshot":65167360.0,"last_snapshot_bytes":65167360,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726149140,"remote_snapshot_timestamp":1726149140,"replay_state":"idle"} 2024-09-12T13:53:43.563 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:53:43.564 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+replaying 2024-09-12T13:53:43.564 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:53:43.566 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+replaying 2024-09-12T13:53:43.566 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:53:43.569 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:53:43.569 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:53:43.570 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:53:43.574 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:53:17' 2024-09-12T13:53:43.574 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary8 up+replaying '2024-09-12 13:53:17' 2024-09-12T13:53:43.574 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:53:43.574 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:53:43.574 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:53:43.574 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:53:43.574 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:53:17' 2024-09-12T13:53:43.574 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:53:43.575 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:53:43.575 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-09-12T13:53:43.662 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING341934232mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary1002633bcb-38b0-4c95-9ca5-a988ab7c0dd0up+replayingreplaying, {"bytes_per_second":3417292.8,"bytes_per_snapshot":68345856.0,"last_snapshot_bytes":68345856,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726149146,"remote_snapshot_timestamp":1726149146,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:53:17image-primary8901b32e6-a0c7-4be3-ada0-5f0083a2314cup+replayingreplaying, {"bytes_per_second":3258368.0,"bytes_per_snapshot":65167360.0,"last_snapshot_bytes":65167360,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726149140,"remote_snapshot_timestamp":1726149140,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:53:17image-primary943c84ab2-a21e-441e-82d9-32f1b27926a0up+replayingreplaying, {"bytes_per_second":3307520.0,"bytes_per_snapshot":66150400.0,"last_snapshot_bytes":66150400,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726149146,"remote_snapshot_timestamp":1726149146,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:53:17' 2024-09-12T13:53:43.663 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:53:43.663 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary8'\'']/last_update' 2024-09-12T13:53:43.667 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:53:17' 2024-09-12T13:53:43.668 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary8'\'']/state' 2024-09-12T13:53:43.672 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+replaying 2024-09-12T13:53:43.672 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+replaying 2024-09-12T13:53:43.672 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+replaying 2024-09-12T13:53:43.674 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+replaying 2024-09-12T13:53:43.675 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:53:43.675 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:53:43.675 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ remove_image cluster2 mirror image-primary8 2024-09-12T13:53:43.675 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:53:43.675 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:53:43.675 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:53:43.675 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 snap purge mirror/image-primary8 2024-09-12T13:53:43.735 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 rm mirror/image-primary8 2024-09-12T13:53:43.881 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:53:43.881+0000 7f67e5200640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x560556ddd520 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-09-12T13:53:43.881 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:53:43.881+0000 7f67eca00640 -1 rbd::mirror::ImageReplayer: 0x56055c199680 [3/901b32e6-a0c7-4be3-ada0-5f0083a2314c] operator(): start failed: (85) Interrupted system call should be restarted 2024-09-12T13:53:45.415 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-09-12T13:53:45.421 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_image_removal cluster1 mirror image-primary8 2024-09-12T13:53:45.421 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:53:45.421 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:53:45.422 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:53:45.422 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:53:45.422 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 ls mirror 2024-09-12T13:53:45.422 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep -wq image-primary8 2024-09-12T13:53:45.471 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:53:45.471 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rm -rf test-primary8 2024-09-12T13:53:45.474 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:53:45.474 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ IMG=image-primary9 2024-09-12T13:53:45.474 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_replaying_status_in_pool_dir cluster1 mirror image-primary9 2024-09-12T13:53:45.474 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:53:45.474 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:53:45.475 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:53:45.475 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' snapshot = journal ']' 2024-09-12T13:53:45.475 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary9 up+replaying 2024-09-12T13:53:45.475 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:53:45.475 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:53:45.475 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:53:45.475 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:53:45.475 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:53:45.475 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:53:45.475 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:53:45.476 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:53:46.476 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary9 up+replaying '' '' 2024-09-12T13:53:46.476 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:53:46.477 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:53:46.477 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:53:46.477 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:53:46.477 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:53:46.477 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:53:46.477 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:53:46.477 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:53:46.477 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary9 2024-09-12T13:53:46.556 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary9: 2024-09-12T13:53:46.556 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 43c84ab2-a21e-441e-82d9-32f1b27926a0 2024-09-12T13:53:46.556 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+replaying 2024-09-12T13:53:46.556 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: replaying, {"bytes_per_second":3307520.0,"bytes_per_snapshot":66150400.0,"last_snapshot_bytes":66150400,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726149146,"remote_snapshot_timestamp":1726149146,"replay_state":"idle"} 2024-09-12T13:53:46.556 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:53:46.557 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:53:17' 2024-09-12T13:53:46.557 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+replaying' 2024-09-12T13:53:46.559 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+replaying 2024-09-12T13:53:46.559 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:53:46.561 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: replaying, {"bytes_per_second":3307520.0,"bytes_per_snapshot":66150400.0,"last_snapshot_bytes":66150400,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726149146,"remote_snapshot_timestamp":1726149146,"replay_state":"idle"} 2024-09-12T13:53:46.562 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:53:46.562 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+replaying 2024-09-12T13:53:46.563 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:53:46.565 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+replaying 2024-09-12T13:53:46.565 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:53:46.568 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:53:46.568 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:53:46.569 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:53:46.573 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:53:17' 2024-09-12T13:53:46.573 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary9 up+replaying '2024-09-12 13:53:17' 2024-09-12T13:53:46.573 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:53:46.573 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:53:46.573 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:53:46.573 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:53:46.573 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:53:17' 2024-09-12T13:53:46.573 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:53:46.574 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:53:46.574 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-09-12T13:53:46.659 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING241934232mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary1002633bcb-38b0-4c95-9ca5-a988ab7c0dd0up+replayingreplaying, {"bytes_per_second":3417292.8,"bytes_per_snapshot":68345856.0,"last_snapshot_bytes":68345856,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726149146,"remote_snapshot_timestamp":1726149146,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:53:17image-primary943c84ab2-a21e-441e-82d9-32f1b27926a0up+replayingreplaying, {"bytes_per_second":3307520.0,"bytes_per_snapshot":66150400.0,"last_snapshot_bytes":66150400,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726149146,"remote_snapshot_timestamp":1726149146,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:53:17' 2024-09-12T13:53:46.659 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:53:46.659 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary9'\'']/last_update' 2024-09-12T13:53:46.663 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:53:17' 2024-09-12T13:53:46.664 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary9'\'']/state' 2024-09-12T13:53:46.667 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+replaying 2024-09-12T13:53:46.668 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+replaying 2024-09-12T13:53:46.668 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+replaying 2024-09-12T13:53:46.670 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+replaying 2024-09-12T13:53:46.671 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:53:46.671 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:53:46.671 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ remove_image cluster2 mirror image-primary9 2024-09-12T13:53:46.671 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:53:46.672 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:53:46.672 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:53:46.672 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 snap purge mirror/image-primary9 2024-09-12T13:53:46.732 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 rm mirror/image-primary9 2024-09-12T13:53:46.880 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:53:46.879+0000 7f67e4800640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x56055627a820 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-09-12T13:53:46.880 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:53:46.879+0000 7f67eca00640 -1 rbd::mirror::ImageReplayer: 0x56055c198c80 [3/43c84ab2-a21e-441e-82d9-32f1b27926a0] operator(): start failed: (85) Interrupted system call should be restarted 2024-09-12T13:53:48.715 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-09-12T13:53:48.721 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_image_removal cluster1 mirror image-primary9 2024-09-12T13:53:48.721 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:53:48.721 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:53:48.721 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:53:48.721 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:53:48.722 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 ls mirror 2024-09-12T13:53:48.722 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep -wq image-primary9 2024-09-12T13:53:48.781 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:53:48.781 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rm -rf test-primary9 2024-09-12T13:53:48.784 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:53:48.784 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ IMG=image-primary10 2024-09-12T13:53:48.784 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_replaying_status_in_pool_dir cluster1 mirror image-primary10 2024-09-12T13:53:48.784 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:53:48.784 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:53:48.784 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:53:48.784 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' snapshot = journal ']' 2024-09-12T13:53:48.784 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary10 up+replaying 2024-09-12T13:53:48.785 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:53:48.785 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:53:48.785 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:53:48.785 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:53:48.785 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:53:48.785 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:53:48.785 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:53:48.785 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:53:49.786 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary10 up+replaying '' '' 2024-09-12T13:53:49.786 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:53:49.786 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:53:49.786 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:53:49.786 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:53:49.786 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:53:49.786 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:53:49.787 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:53:49.787 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:53:49.787 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary10 2024-09-12T13:53:49.861 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary10: 2024-09-12T13:53:49.861 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 02633bcb-38b0-4c95-9ca5-a988ab7c0dd0 2024-09-12T13:53:49.861 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+replaying 2024-09-12T13:53:49.861 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: replaying, {"bytes_per_second":0.0,"bytes_per_snapshot":68345856.0,"last_snapshot_bytes":68345856,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726149146,"remote_snapshot_timestamp":1726149146,"replay_state":"idle"} 2024-09-12T13:53:49.861 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:53:49.861 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:53:47' 2024-09-12T13:53:49.861 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+replaying' 2024-09-12T13:53:49.863 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+replaying 2024-09-12T13:53:49.863 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T13:53:49.866 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: replaying, {"bytes_per_second":0.0,"bytes_per_snapshot":68345856.0,"last_snapshot_bytes":68345856,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726149146,"remote_snapshot_timestamp":1726149146,"replay_state":"idle"} 2024-09-12T13:53:49.866 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T13:53:49.867 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+replaying 2024-09-12T13:53:49.867 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T13:53:49.869 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+replaying 2024-09-12T13:53:49.869 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T13:53:49.872 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T13:53:49.872 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T13:53:49.873 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T13:53:49.877 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:53:47' 2024-09-12T13:53:49.877 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary10 up+replaying '2024-09-12 13:53:47' 2024-09-12T13:53:49.877 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:53:49.878 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:53:49.878 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:53:49.878 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T13:53:49.878 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 13:53:47' 2024-09-12T13:53:49.878 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:53:49.878 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:53:49.878 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-09-12T13:53:49.958 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING141934232mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary1002633bcb-38b0-4c95-9ca5-a988ab7c0dd0up+replayingreplaying, {"bytes_per_second":0.0,"bytes_per_snapshot":68345856.0,"last_snapshot_bytes":68345856,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726149146,"remote_snapshot_timestamp":1726149146,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 13:53:47' 2024-09-12T13:53:49.959 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T13:53:49.959 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary10'\'']/last_update' 2024-09-12T13:53:49.963 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 13:53:47' 2024-09-12T13:53:49.964 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary10'\'']/state' 2024-09-12T13:53:49.967 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+replaying 2024-09-12T13:53:49.968 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+replaying 2024-09-12T13:53:49.968 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+replaying 2024-09-12T13:53:49.970 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+replaying 2024-09-12T13:53:49.971 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:53:49.971 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:53:49.971 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ remove_image cluster2 mirror image-primary10 2024-09-12T13:53:49.971 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T13:53:49.971 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:53:49.971 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:53:49.971 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 snap purge mirror/image-primary10 2024-09-12T13:53:50.033 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 rm mirror/image-primary10 2024-09-12T13:53:50.172 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:53:50.172+0000 7f67e4800640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x56055b865860 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-09-12T13:53:50.172 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:53:50.172+0000 7f67eca00640 -1 rbd::mirror::ImageReplayer: 0x5605561ac280 [3/02633bcb-38b0-4c95-9ca5-a988ab7c0dd0] operator(): start failed: (85) Interrupted system call should be restarted 2024-09-12T13:53:51.469 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-09-12T13:53:51.475 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_image_removal cluster1 mirror image-primary10 2024-09-12T13:53:51.475 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:53:51.475 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:53:51.475 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:53:51.475 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:53:51.476 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 ls mirror 2024-09-12T13:53:51.476 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep -wq image-primary10 2024-09-12T13:53:51.525 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:53:51.525 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rm -rf test-primary10 2024-09-12T13:53:51.529 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..10} 2024-09-12T13:53:51.530 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEVS=() 2024-09-12T13:53:51.530 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ SNAP_PIDS=() 2024-09-12T13:53:51.530 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ COMPARE_PIDS=() 2024-09-12T13:53:51.530 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ WORKLOAD_PIDS=() 2024-09-12T13:53:51.530 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ RET=0 2024-09-12T13:53:51.530 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:53:51.530 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ IMG=image-primary1 2024-09-12T13:53:51.530 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ MNTPT=test-primary1 2024-09-12T13:53:51.530 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ create_image_and_enable_mirror cluster1 mirror image-primary1 snapshot 10G 2024-09-12T13:53:51.530 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:53:51.531 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:53:51.531 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:53:51.531 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:53:51.531 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:53:51.531 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:53:51.531 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mode=snapshot 2024-09-12T13:53:51.531 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n snapshot ']' 2024-09-12T13:53:51.531 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:53:51.531 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ create_image cluster1 mirror image-primary1 10G 2024-09-12T13:53:51.531 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:53:51.531 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:53:51.532 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:53:51.532 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:53:51.532 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:53:51.532 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:53:51.532 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local size=128 2024-09-12T13:53:51.532 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n 10G ']' 2024-09-12T13:53:51.532 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ size=10G 2024-09-12T13:53:51.532 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:53:51.532 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 create --size 10G --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image-primary1 2024-09-12T13:53:51.603 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ is_pool_mirror_mode_image mirror 2024-09-12T13:53:51.604 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:53:51.604 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' image = image ']' 2024-09-12T13:53:51.604 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:53:51.604 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ enable_mirror cluster1 mirror image-primary1 snapshot 2024-09-12T13:53:51.604 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:53:51.604 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:53:51.604 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:53:51.604 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mode=snapshot 2024-09-12T13:53:51.604 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 mirror image enable mirror/image-primary1 snapshot 2024-09-12T13:53:52.382 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Mirroring enabled 2024-09-12T13:53:52.397 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 info mirror/image-primary1 2024-09-12T13:53:52.455 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:rbd image 'image-primary1': 2024-09-12T13:53:52.456 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: size 10 GiB in 2560 objects 2024-09-12T13:53:52.456 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: order 22 (4 MiB objects) 2024-09-12T13:53:52.456 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: snapshot_count: 1 2024-09-12T13:53:52.456 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: id: 3338640fb1b8 2024-09-12T13:53:52.456 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: block_name_prefix: rbd_data.3338640fb1b8 2024-09-12T13:53:52.456 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: format: 2 2024-09-12T13:53:52.456 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: features: layering, exclusive-lock, object-map, fast-diff 2024-09-12T13:53:52.456 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: op_features: 2024-09-12T13:53:52.456 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: flags: 2024-09-12T13:53:52.456 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: create_timestamp: Thu Sep 12 13:53:51 2024 2024-09-12T13:53:52.456 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: access_timestamp: Thu Sep 12 13:53:51 2024 2024-09-12T13:53:52.456 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: modify_timestamp: Thu Sep 12 13:53:51 2024 2024-09-12T13:53:52.457 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring state: enabled 2024-09-12T13:53:52.457 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring mode: snapshot 2024-09-12T13:53:52.457 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring global id: a862d41a-8d34-4c63-bea6-4195101bf11a 2024-09-12T13:53:52.457 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring primary: true 2024-09-12T13:53:52.461 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \n\b\d ]] 2024-09-12T13:53:52.461 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \k\r\b\d ]] 2024-09-12T13:53:52.462 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo rbd --cluster cluster1 device map -t krbd mirror/image-primary1 2024-09-12T13:53:52.604 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEV=/dev/rbd0 2024-09-12T13:53:52.604 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEVS+=($DEV) 2024-09-12T13:53:52.604 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo mkfs.ext4 /dev/rbd0 2024-09-12T13:53:52.673 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:mke2fs 1.46.5 (30-Dec-2021) 2024-09-12T13:53:52.684 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Discarding device blocks: 0/2621440 done 2024-09-12T13:53:52.684 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Creating filesystem with 2621440 4k blocks and 655360 inodes 2024-09-12T13:53:52.684 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Filesystem UUID: 192d7624-09b3-44ff-bdac-6dc60e7dcbc0 2024-09-12T13:53:52.684 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Superblock backups stored on blocks: 2024-09-12T13:53:52.684 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 2024-09-12T13:53:52.685 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 2024-09-12T13:53:52.685 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Allocating group tables: 0/80 done 2024-09-12T13:53:52.685 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Writing inode tables: 0/80 done 2024-09-12T13:53:52.689 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Creating journal (16384 blocks): done 2024-09-12T13:53:52.754 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Writing superblocks and filesystem accounting information: 0/80 done 2024-09-12T13:53:52.754 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 2024-09-12T13:53:52.756 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mkdir test-primary1 2024-09-12T13:53:52.773 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo mount /dev/rbd0 test-primary1 2024-09-12T13:53:52.831 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ whoami 2024-09-12T13:53:52.842 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo chown ubuntu test-primary1 2024-09-12T13:53:52.872 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ SNAP_PIDS+=($!) 2024-09-12T13:53:52.873 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ take_mirror_snapshots cluster1 mirror image-primary1 2024-09-12T13:53:52.873 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:53:52.873 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:53:52.873 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:53:52.873 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:53:52.873 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:53:52.873 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:53:52.873 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:53:52.873 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ WORKLOAD_PIDS+=($!) 2024-09-12T13:53:52.873 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:53:52.873 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:53:52.874 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:53:52.874 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ IMG=image-primary2 2024-09-12T13:53:52.874 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ MNTPT=test-primary2 2024-09-12T13:53:52.875 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ create_image_and_enable_mirror cluster1 mirror image-primary2 snapshot 10G 2024-09-12T13:53:52.875 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ slow_untar_workload test-primary1 2024-09-12T13:53:52.875 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:53:52.875 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:53:52.875 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:53:52.876 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:53:52.876 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:53:52.876 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:53:52.876 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mode=snapshot 2024-09-12T13:53:52.876 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n snapshot ']' 2024-09-12T13:53:52.876 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:53:52.876 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ create_image cluster1 mirror image-primary2 10G 2024-09-12T13:53:52.876 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:53:52.876 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mountpt=test-primary1 2024-09-12T13:53:52.876 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:53:52.876 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:53:52.876 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:53:52.876 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:53:52.877 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:53:52.877 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local size=128 2024-09-12T13:53:52.877 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ cp linux-5.4.tar.gz test-primary1 2024-09-12T13:53:52.877 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n 10G ']' 2024-09-12T13:53:52.877 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ size=10G 2024-09-12T13:53:52.877 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:53:52.877 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 create --size 10G --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image-primary2 2024-09-12T13:53:52.951 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ is_pool_mirror_mode_image mirror 2024-09-12T13:53:52.951 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:53:52.951 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' image = image ']' 2024-09-12T13:53:52.951 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:53:52.952 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ enable_mirror cluster1 mirror image-primary2 snapshot 2024-09-12T13:53:52.952 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:53:52.952 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:53:52.952 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:53:52.952 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mode=snapshot 2024-09-12T13:53:52.952 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 mirror image enable mirror/image-primary2 snapshot 2024-09-12T13:53:53.168 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local ret=0 2024-09-12T13:53:53.168 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ timeout 5m bash -c 'zcat test-primary1/linux-5.4.tar.gz | pv -L 256K | tar xf - -C test-primary1' 2024-09-12T13:53:53.341 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Mirroring enabled 2024-09-12T13:53:53.342 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2314 2024-09-12T13:53:53.353 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:53:53.356 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 info mirror/image-primary2 2024-09-12T13:53:53.412 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:rbd image 'image-primary2': 2024-09-12T13:53:53.412 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: size 10 GiB in 2560 objects 2024-09-12T13:53:53.412 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: order 22 (4 MiB objects) 2024-09-12T13:53:53.413 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: snapshot_count: 1 2024-09-12T13:53:53.413 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: id: 334850e1974b 2024-09-12T13:53:53.413 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: block_name_prefix: rbd_data.334850e1974b 2024-09-12T13:53:53.413 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: format: 2 2024-09-12T13:53:53.413 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: features: layering, exclusive-lock, object-map, fast-diff 2024-09-12T13:53:53.413 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: op_features: 2024-09-12T13:53:53.413 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: flags: 2024-09-12T13:53:53.413 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: create_timestamp: Thu Sep 12 13:53:52 2024 2024-09-12T13:53:53.413 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: access_timestamp: Thu Sep 12 13:53:52 2024 2024-09-12T13:53:53.413 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: modify_timestamp: Thu Sep 12 13:53:52 2024 2024-09-12T13:53:53.413 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring state: enabled 2024-09-12T13:53:53.413 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring mode: snapshot 2024-09-12T13:53:53.413 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring global id: fef52d8d-af21-4d3d-97e9-021eac35b36a 2024-09-12T13:53:53.414 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring primary: true 2024-09-12T13:53:53.417 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \n\b\d ]] 2024-09-12T13:53:53.418 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \k\r\b\d ]] 2024-09-12T13:53:53.418 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo rbd --cluster cluster1 device map -t krbd mirror/image-primary2 2024-09-12T13:53:53.536 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEV=/dev/rbd1 2024-09-12T13:53:53.536 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEVS+=($DEV) 2024-09-12T13:53:53.536 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo mkfs.ext4 /dev/rbd1 2024-09-12T13:53:53.557 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:mke2fs 1.46.5 (30-Dec-2021) 2024-09-12T13:53:53.571 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Discarding device blocks: 0/2621440 done 2024-09-12T13:53:53.571 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Creating filesystem with 2621440 4k blocks and 655360 inodes 2024-09-12T13:53:53.571 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Filesystem UUID: e9bef2ca-ca42-4a20-8b41-7571ca2626eb 2024-09-12T13:53:53.571 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Superblock backups stored on blocks: 2024-09-12T13:53:53.571 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 2024-09-12T13:53:53.571 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 2024-09-12T13:53:53.571 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Allocating group tables: 0/80 done 2024-09-12T13:53:53.571 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Writing inode tables: 0/80 done 2024-09-12T13:53:53.576 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Creating journal (16384 blocks): done 2024-09-12T13:53:53.633 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Writing superblocks and filesystem accounting information: 0/80 done 2024-09-12T13:53:53.634 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 2024-09-12T13:53:53.636 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mkdir test-primary2 2024-09-12T13:53:53.639 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo mount /dev/rbd1 test-primary2 2024-09-12T13:53:53.694 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ whoami 2024-09-12T13:53:53.695 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo chown ubuntu test-primary2 2024-09-12T13:53:53.721 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ SNAP_PIDS+=($!) 2024-09-12T13:53:53.721 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ take_mirror_snapshots cluster1 mirror image-primary2 2024-09-12T13:53:53.721 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:53:53.721 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:53:53.721 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:53:53.721 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:53:53.721 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:53:53.721 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:53:53.721 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:53:53.721 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:53:53.721 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ WORKLOAD_PIDS+=($!) 2024-09-12T13:53:53.722 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:53:53.722 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:53:53.722 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ IMG=image-primary3 2024-09-12T13:53:53.722 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ slow_untar_workload test-primary2 2024-09-12T13:53:53.722 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ MNTPT=test-primary3 2024-09-12T13:53:53.722 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ create_image_and_enable_mirror cluster1 mirror image-primary3 snapshot 10G 2024-09-12T13:53:53.722 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:53:53.722 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:53:53.722 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:53:53.722 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:53:53.722 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:53:53.722 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:53:53.722 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mode=snapshot 2024-09-12T13:53:53.722 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mountpt=test-primary2 2024-09-12T13:53:53.723 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n snapshot ']' 2024-09-12T13:53:53.723 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:53:53.723 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ create_image cluster1 mirror image-primary3 10G 2024-09-12T13:53:53.723 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ cp linux-5.4.tar.gz test-primary2 2024-09-12T13:53:53.723 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:53:53.723 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:53:53.723 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:53:53.723 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:53:53.723 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:53:53.723 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:53:53.723 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local size=128 2024-09-12T13:53:53.723 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n 10G ']' 2024-09-12T13:53:53.723 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ size=10G 2024-09-12T13:53:53.723 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:53:53.724 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 create --size 10G --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image-primary3 2024-09-12T13:53:53.793 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ is_pool_mirror_mode_image mirror 2024-09-12T13:53:53.794 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:53:53.794 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' image = image ']' 2024-09-12T13:53:53.794 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:53:53.794 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ enable_mirror cluster1 mirror image-primary3 snapshot 2024-09-12T13:53:53.794 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:53:53.794 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:53:53.794 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:53:53.794 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mode=snapshot 2024-09-12T13:53:53.794 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 mirror image enable mirror/image-primary3 snapshot 2024-09-12T13:53:54.006 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local ret=0 2024-09-12T13:53:54.006 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ timeout 5m bash -c 'zcat test-primary2/linux-5.4.tar.gz | pv -L 256K | tar xf - -C test-primary2' 2024-09-12T13:53:54.399 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Mirroring enabled 2024-09-12T13:53:54.406 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2316 2024-09-12T13:53:54.416 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 info mirror/image-primary3 2024-09-12T13:53:54.417 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:53:54.481 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:rbd image 'image-primary3': 2024-09-12T13:53:54.481 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: size 10 GiB in 2560 objects 2024-09-12T13:53:54.481 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: order 22 (4 MiB objects) 2024-09-12T13:53:54.481 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: snapshot_count: 1 2024-09-12T13:53:54.481 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: id: 33577276b18b 2024-09-12T13:53:54.481 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: block_name_prefix: rbd_data.33577276b18b 2024-09-12T13:53:54.482 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: format: 2 2024-09-12T13:53:54.482 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: features: layering, exclusive-lock, object-map, fast-diff 2024-09-12T13:53:54.482 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: op_features: 2024-09-12T13:53:54.482 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: flags: 2024-09-12T13:53:54.482 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: create_timestamp: Thu Sep 12 13:53:53 2024 2024-09-12T13:53:54.482 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: access_timestamp: Thu Sep 12 13:53:53 2024 2024-09-12T13:53:54.482 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: modify_timestamp: Thu Sep 12 13:53:53 2024 2024-09-12T13:53:54.482 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring state: enabled 2024-09-12T13:53:54.482 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring mode: snapshot 2024-09-12T13:53:54.482 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring global id: a8c53447-94fc-4fe3-b559-d953f6f94c54 2024-09-12T13:53:54.482 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring primary: true 2024-09-12T13:53:54.486 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \n\b\d ]] 2024-09-12T13:53:54.486 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \k\r\b\d ]] 2024-09-12T13:53:54.487 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo rbd --cluster cluster1 device map -t krbd mirror/image-primary3 2024-09-12T13:53:54.615 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEV=/dev/rbd2 2024-09-12T13:53:54.615 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEVS+=($DEV) 2024-09-12T13:53:54.615 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo mkfs.ext4 /dev/rbd2 2024-09-12T13:53:54.636 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:mke2fs 1.46.5 (30-Dec-2021) 2024-09-12T13:53:54.649 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Discarding device blocks: 0/2621440 done 2024-09-12T13:53:54.649 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Creating filesystem with 2621440 4k blocks and 655360 inodes 2024-09-12T13:53:54.649 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Filesystem UUID: 1de2e35e-097c-4304-b6d4-db3d4f1e40f0 2024-09-12T13:53:54.649 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Superblock backups stored on blocks: 2024-09-12T13:53:54.649 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 2024-09-12T13:53:54.649 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 2024-09-12T13:53:54.649 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Allocating group tables: 0/80 done 2024-09-12T13:53:54.650 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Writing inode tables: 0/80 done 2024-09-12T13:53:54.654 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Creating journal (16384 blocks): done 2024-09-12T13:53:54.712 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Writing superblocks and filesystem accounting information: 0/80 done 2024-09-12T13:53:54.712 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 2024-09-12T13:53:54.715 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mkdir test-primary3 2024-09-12T13:53:54.717 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo mount /dev/rbd2 test-primary3 2024-09-12T13:53:54.774 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ whoami 2024-09-12T13:53:54.776 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo chown ubuntu test-primary3 2024-09-12T13:53:54.800 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ SNAP_PIDS+=($!) 2024-09-12T13:53:54.800 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ take_mirror_snapshots cluster1 mirror image-primary3 2024-09-12T13:53:54.800 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:53:54.800 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:53:54.801 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:53:54.801 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:53:54.801 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:53:54.801 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:53:54.801 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:53:54.801 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:53:54.801 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:53:54.801 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ WORKLOAD_PIDS+=($!) 2024-09-12T13:53:54.801 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:53:54.801 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ IMG=image-primary4 2024-09-12T13:53:54.803 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ MNTPT=test-primary4 2024-09-12T13:53:54.803 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ create_image_and_enable_mirror cluster1 mirror image-primary4 snapshot 10G 2024-09-12T13:53:54.803 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ slow_untar_workload test-primary3 2024-09-12T13:53:54.803 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:53:54.803 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:53:54.803 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:53:54.804 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:53:54.804 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:53:54.804 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:53:54.804 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mode=snapshot 2024-09-12T13:53:54.804 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n snapshot ']' 2024-09-12T13:53:54.804 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:53:54.804 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ create_image cluster1 mirror image-primary4 10G 2024-09-12T13:53:54.804 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:53:54.804 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:53:54.804 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:53:54.804 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:53:54.804 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:53:54.805 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:53:54.805 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local size=128 2024-09-12T13:53:54.805 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n 10G ']' 2024-09-12T13:53:54.805 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ size=10G 2024-09-12T13:53:54.805 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:53:54.805 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mountpt=test-primary3 2024-09-12T13:53:54.805 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 create --size 10G --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image-primary4 2024-09-12T13:53:54.805 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ cp linux-5.4.tar.gz test-primary3 2024-09-12T13:53:54.873 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ is_pool_mirror_mode_image mirror 2024-09-12T13:53:54.874 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:53:54.874 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' image = image ']' 2024-09-12T13:53:54.874 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:53:54.874 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ enable_mirror cluster1 mirror image-primary4 snapshot 2024-09-12T13:53:54.874 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:53:54.874 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:53:54.874 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:53:54.874 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mode=snapshot 2024-09-12T13:53:54.874 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 mirror image enable mirror/image-primary4 snapshot 2024-09-12T13:53:55.085 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local ret=0 2024-09-12T13:53:55.085 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ timeout 5m bash -c 'zcat test-primary3/linux-5.4.tar.gz | pv -L 256K | tar xf - -C test-primary3' 2024-09-12T13:53:55.408 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Mirroring enabled 2024-09-12T13:53:55.409 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2318 2024-09-12T13:53:55.418 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:53:55.421 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 info mirror/image-primary4 2024-09-12T13:53:55.477 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:rbd image 'image-primary4': 2024-09-12T13:53:55.477 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: size 10 GiB in 2560 objects 2024-09-12T13:53:55.477 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: order 22 (4 MiB objects) 2024-09-12T13:53:55.478 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: snapshot_count: 1 2024-09-12T13:53:55.478 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: id: 33666ceb22c9 2024-09-12T13:53:55.478 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: block_name_prefix: rbd_data.33666ceb22c9 2024-09-12T13:53:55.478 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: format: 2 2024-09-12T13:53:55.478 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: features: layering, exclusive-lock, object-map, fast-diff 2024-09-12T13:53:55.478 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: op_features: 2024-09-12T13:53:55.478 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: flags: 2024-09-12T13:53:55.478 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: create_timestamp: Thu Sep 12 13:53:54 2024 2024-09-12T13:53:55.478 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: access_timestamp: Thu Sep 12 13:53:54 2024 2024-09-12T13:53:55.478 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: modify_timestamp: Thu Sep 12 13:53:54 2024 2024-09-12T13:53:55.478 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring state: enabled 2024-09-12T13:53:55.478 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring mode: snapshot 2024-09-12T13:53:55.478 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring global id: 8face4e5-1ee3-427e-b5bc-7b05be9d6e87 2024-09-12T13:53:55.478 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring primary: true 2024-09-12T13:53:55.483 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \n\b\d ]] 2024-09-12T13:53:55.483 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \k\r\b\d ]] 2024-09-12T13:53:55.484 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo rbd --cluster cluster1 device map -t krbd mirror/image-primary4 2024-09-12T13:53:55.604 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEV=/dev/rbd3 2024-09-12T13:53:55.604 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEVS+=($DEV) 2024-09-12T13:53:55.604 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo mkfs.ext4 /dev/rbd3 2024-09-12T13:53:55.624 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:mke2fs 1.46.5 (30-Dec-2021) 2024-09-12T13:53:55.637 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Discarding device blocks: 0/2621440 done 2024-09-12T13:53:55.637 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Creating filesystem with 2621440 4k blocks and 655360 inodes 2024-09-12T13:53:55.637 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Filesystem UUID: a8ce5794-b9f0-4ef2-b302-6c49e0ce8415 2024-09-12T13:53:55.637 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Superblock backups stored on blocks: 2024-09-12T13:53:55.637 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 2024-09-12T13:53:55.637 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 2024-09-12T13:53:55.637 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Allocating group tables: 0/80 done 2024-09-12T13:53:55.637 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Writing inode tables: 0/80 done 2024-09-12T13:53:55.642 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Creating journal (16384 blocks): done 2024-09-12T13:53:55.700 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Writing superblocks and filesystem accounting information: 0/80 done 2024-09-12T13:53:55.700 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 2024-09-12T13:53:55.703 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mkdir test-primary4 2024-09-12T13:53:55.705 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo mount /dev/rbd3 test-primary4 2024-09-12T13:53:55.762 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ whoami 2024-09-12T13:53:55.764 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo chown ubuntu test-primary4 2024-09-12T13:53:55.789 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ SNAP_PIDS+=($!) 2024-09-12T13:53:55.790 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ take_mirror_snapshots cluster1 mirror image-primary4 2024-09-12T13:53:55.790 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:53:55.790 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:53:55.790 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:53:55.790 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:53:55.790 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:53:55.790 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:53:55.790 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:53:55.790 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:53:55.790 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ WORKLOAD_PIDS+=($!) 2024-09-12T13:53:55.790 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:53:55.790 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:53:55.791 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ IMG=image-primary5 2024-09-12T13:53:55.792 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ MNTPT=test-primary5 2024-09-12T13:53:55.792 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ slow_untar_workload test-primary4 2024-09-12T13:53:55.792 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ create_image_and_enable_mirror cluster1 mirror image-primary5 snapshot 10G 2024-09-12T13:53:55.792 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:53:55.792 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:53:55.792 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:53:55.792 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:53:55.792 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:53:55.793 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:53:55.793 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mode=snapshot 2024-09-12T13:53:55.793 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n snapshot ']' 2024-09-12T13:53:55.793 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:53:55.793 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ create_image cluster1 mirror image-primary5 10G 2024-09-12T13:53:55.793 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mountpt=test-primary4 2024-09-12T13:53:55.793 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ cp linux-5.4.tar.gz test-primary4 2024-09-12T13:53:55.793 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:53:55.793 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:53:55.793 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:53:55.793 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:53:55.793 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:53:55.793 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:53:55.794 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local size=128 2024-09-12T13:53:55.794 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n 10G ']' 2024-09-12T13:53:55.794 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ size=10G 2024-09-12T13:53:55.794 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:53:55.794 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 create --size 10G --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image-primary5 2024-09-12T13:53:55.863 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ is_pool_mirror_mode_image mirror 2024-09-12T13:53:55.863 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:53:55.863 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' image = image ']' 2024-09-12T13:53:55.864 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:53:55.864 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ enable_mirror cluster1 mirror image-primary5 snapshot 2024-09-12T13:53:55.864 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:53:55.864 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:53:55.864 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:53:55.864 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mode=snapshot 2024-09-12T13:53:55.864 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 mirror image enable mirror/image-primary5 snapshot 2024-09-12T13:53:56.080 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local ret=0 2024-09-12T13:53:56.080 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ timeout 5m bash -c 'zcat test-primary4/linux-5.4.tar.gz | pv -L 256K | tar xf - -C test-primary4' 2024-09-12T13:53:56.355 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:53:56.356 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:53:56.356 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:53:56.356 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:53:56.356 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:53:56.356 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:53:56.425 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Mirroring enabled 2024-09-12T13:53:56.433 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2320 2024-09-12T13:53:56.451 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:53:56.461 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 info mirror/image-primary5 2024-09-12T13:53:56.550 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:rbd image 'image-primary5': 2024-09-12T13:53:56.550 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: size 10 GiB in 2560 objects 2024-09-12T13:53:56.550 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: order 22 (4 MiB objects) 2024-09-12T13:53:56.551 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: snapshot_count: 1 2024-09-12T13:53:56.551 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: id: 337540e0f894 2024-09-12T13:53:56.551 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: block_name_prefix: rbd_data.337540e0f894 2024-09-12T13:53:56.551 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: format: 2 2024-09-12T13:53:56.551 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: features: layering, exclusive-lock, object-map, fast-diff 2024-09-12T13:53:56.551 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: op_features: 2024-09-12T13:53:56.551 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: flags: 2024-09-12T13:53:56.551 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: create_timestamp: Thu Sep 12 13:53:55 2024 2024-09-12T13:53:56.551 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: access_timestamp: Thu Sep 12 13:53:55 2024 2024-09-12T13:53:56.551 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: modify_timestamp: Thu Sep 12 13:53:55 2024 2024-09-12T13:53:56.551 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring state: enabled 2024-09-12T13:53:56.551 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring mode: snapshot 2024-09-12T13:53:56.551 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring global id: fb69fceb-7a10-4ab7-ad45-ec4124a4621f 2024-09-12T13:53:56.551 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring primary: true 2024-09-12T13:53:56.556 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \n\b\d ]] 2024-09-12T13:53:56.556 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \k\r\b\d ]] 2024-09-12T13:53:56.557 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo rbd --cluster cluster1 device map -t krbd mirror/image-primary5 2024-09-12T13:53:56.716 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEV=/dev/rbd4 2024-09-12T13:53:56.716 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEVS+=($DEV) 2024-09-12T13:53:56.716 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo mkfs.ext4 /dev/rbd4 2024-09-12T13:53:56.737 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:mke2fs 1.46.5 (30-Dec-2021) 2024-09-12T13:53:56.750 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Discarding device blocks: 0/2621440 done 2024-09-12T13:53:56.750 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Creating filesystem with 2621440 4k blocks and 655360 inodes 2024-09-12T13:53:56.750 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Filesystem UUID: f65d3e3c-874c-4344-888b-e9bd5ca2170a 2024-09-12T13:53:56.750 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Superblock backups stored on blocks: 2024-09-12T13:53:56.750 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 2024-09-12T13:53:56.750 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 2024-09-12T13:53:56.750 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Allocating group tables: 0/80 done 2024-09-12T13:53:56.750 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Writing inode tables: 0/80 done 2024-09-12T13:53:56.755 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Creating journal (16384 blocks): done 2024-09-12T13:53:56.861 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Writing superblocks and filesystem accounting information: 0/80 done 2024-09-12T13:53:56.862 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 2024-09-12T13:53:56.864 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mkdir test-primary5 2024-09-12T13:53:56.867 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo mount /dev/rbd4 test-primary5 2024-09-12T13:53:56.960 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ whoami 2024-09-12T13:53:56.961 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo chown ubuntu test-primary5 2024-09-12T13:53:56.987 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ SNAP_PIDS+=($!) 2024-09-12T13:53:56.987 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ take_mirror_snapshots cluster1 mirror image-primary5 2024-09-12T13:53:56.988 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:53:56.988 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:53:56.988 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:53:56.988 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:53:56.988 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:53:56.988 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:53:56.988 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:53:56.988 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:53:56.988 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:53:56.988 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ WORKLOAD_PIDS+=($!) 2024-09-12T13:53:56.988 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:53:56.988 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ slow_untar_workload test-primary5 2024-09-12T13:53:56.988 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ IMG=image-primary6 2024-09-12T13:53:56.989 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ MNTPT=test-primary6 2024-09-12T13:53:56.989 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ create_image_and_enable_mirror cluster1 mirror image-primary6 snapshot 10G 2024-09-12T13:53:56.989 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mountpt=test-primary5 2024-09-12T13:53:56.989 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ cp linux-5.4.tar.gz test-primary5 2024-09-12T13:53:56.989 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:53:56.989 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:53:56.989 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:53:56.989 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:53:56.989 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:53:56.989 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:53:56.989 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mode=snapshot 2024-09-12T13:53:56.989 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n snapshot ']' 2024-09-12T13:53:56.989 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:53:56.990 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ create_image cluster1 mirror image-primary6 10G 2024-09-12T13:53:56.990 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:53:56.990 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:53:56.990 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:53:56.990 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:53:56.990 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:53:56.990 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:53:56.990 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local size=128 2024-09-12T13:53:56.990 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n 10G ']' 2024-09-12T13:53:56.990 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ size=10G 2024-09-12T13:53:56.990 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:53:56.990 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 create --size 10G --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image-primary6 2024-09-12T13:53:57.085 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ is_pool_mirror_mode_image mirror 2024-09-12T13:53:57.085 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:53:57.086 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' image = image ']' 2024-09-12T13:53:57.086 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:53:57.086 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ enable_mirror cluster1 mirror image-primary6 snapshot 2024-09-12T13:53:57.086 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:53:57.086 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:53:57.086 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:53:57.086 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mode=snapshot 2024-09-12T13:53:57.086 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 mirror image enable mirror/image-primary6 snapshot 2024-09-12T13:53:57.291 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local ret=0 2024-09-12T13:53:57.291 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ timeout 5m bash -c 'zcat test-primary5/linux-5.4.tar.gz | pv -L 256K | tar xf - -C test-primary5' 2024-09-12T13:53:57.421 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:53:57.421 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:53:57.421 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:53:57.421 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:53:57.421 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:53:57.421 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:53:57.448 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Mirroring enabled 2024-09-12T13:53:57.449 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2323 2024-09-12T13:53:57.458 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2322 2024-09-12T13:53:57.466 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:53:57.474 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:53:57.486 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 info mirror/image-primary6 2024-09-12T13:53:57.564 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:rbd image 'image-primary6': 2024-09-12T13:53:57.565 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: size 10 GiB in 2560 objects 2024-09-12T13:53:57.565 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: order 22 (4 MiB objects) 2024-09-12T13:53:57.565 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: snapshot_count: 1 2024-09-12T13:53:57.565 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: id: 33855113326 2024-09-12T13:53:57.565 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: block_name_prefix: rbd_data.33855113326 2024-09-12T13:53:57.565 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: format: 2 2024-09-12T13:53:57.565 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: features: layering, exclusive-lock, object-map, fast-diff 2024-09-12T13:53:57.565 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: op_features: 2024-09-12T13:53:57.565 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: flags: 2024-09-12T13:53:57.565 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: create_timestamp: Thu Sep 12 13:53:57 2024 2024-09-12T13:53:57.565 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: access_timestamp: Thu Sep 12 13:53:57 2024 2024-09-12T13:53:57.565 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: modify_timestamp: Thu Sep 12 13:53:57 2024 2024-09-12T13:53:57.566 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring state: enabled 2024-09-12T13:53:57.566 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring mode: snapshot 2024-09-12T13:53:57.566 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring global id: a63c238c-944e-44a1-be22-edc69436b860 2024-09-12T13:53:57.566 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring primary: true 2024-09-12T13:53:57.570 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \n\b\d ]] 2024-09-12T13:53:57.570 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \k\r\b\d ]] 2024-09-12T13:53:57.571 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo rbd --cluster cluster1 device map -t krbd mirror/image-primary6 2024-09-12T13:53:57.739 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEV=/dev/rbd5 2024-09-12T13:53:57.739 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEVS+=($DEV) 2024-09-12T13:53:57.739 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo mkfs.ext4 /dev/rbd5 2024-09-12T13:53:57.760 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:mke2fs 1.46.5 (30-Dec-2021) 2024-09-12T13:53:57.773 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Discarding device blocks: 0/2621440 done 2024-09-12T13:53:57.773 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Creating filesystem with 2621440 4k blocks and 655360 inodes 2024-09-12T13:53:57.773 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Filesystem UUID: 194acdd7-a5f1-470e-bf0c-26907691a641 2024-09-12T13:53:57.773 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Superblock backups stored on blocks: 2024-09-12T13:53:57.773 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 2024-09-12T13:53:57.773 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 2024-09-12T13:53:57.774 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Allocating group tables: 0/80 done 2024-09-12T13:53:57.774 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Writing inode tables: 0/80 done 2024-09-12T13:53:57.777 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Creating journal (16384 blocks): done 2024-09-12T13:53:57.871 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Writing superblocks and filesystem accounting information: 0/80 done 2024-09-12T13:53:57.871 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 2024-09-12T13:53:57.874 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mkdir test-primary6 2024-09-12T13:53:57.878 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo mount /dev/rbd5 test-primary6 2024-09-12T13:53:57.960 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ whoami 2024-09-12T13:53:57.961 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo chown ubuntu test-primary6 2024-09-12T13:53:57.987 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ SNAP_PIDS+=($!) 2024-09-12T13:53:57.987 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ take_mirror_snapshots cluster1 mirror image-primary6 2024-09-12T13:53:57.987 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:53:57.987 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:53:57.988 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:53:57.988 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ WORKLOAD_PIDS+=($!) 2024-09-12T13:53:57.988 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:53:57.988 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:53:57.988 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:53:57.988 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ IMG=image-primary7 2024-09-12T13:53:57.988 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:53:57.988 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ MNTPT=test-primary7 2024-09-12T13:53:57.988 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:53:57.988 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:53:57.988 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ slow_untar_workload test-primary6 2024-09-12T13:53:57.988 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ create_image_and_enable_mirror cluster1 mirror image-primary7 snapshot 10G 2024-09-12T13:53:57.988 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:53:57.989 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:53:57.989 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:53:57.989 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:53:57.989 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:53:57.989 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:53:57.989 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:53:57.989 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mode=snapshot 2024-09-12T13:53:57.989 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n snapshot ']' 2024-09-12T13:53:57.989 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:53:57.989 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ create_image cluster1 mirror image-primary7 10G 2024-09-12T13:53:57.989 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:53:57.989 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:53:57.990 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:53:57.990 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:53:57.990 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:53:57.990 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mountpt=test-primary6 2024-09-12T13:53:57.990 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:53:57.990 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local size=128 2024-09-12T13:53:57.990 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n 10G ']' 2024-09-12T13:53:57.990 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ size=10G 2024-09-12T13:53:57.990 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ cp linux-5.4.tar.gz test-primary6 2024-09-12T13:53:57.990 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:53:57.990 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 create --size 10G --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image-primary7 2024-09-12T13:53:58.130 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ is_pool_mirror_mode_image mirror 2024-09-12T13:53:58.130 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:53:58.130 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' image = image ']' 2024-09-12T13:53:58.130 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:53:58.130 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ enable_mirror cluster1 mirror image-primary7 snapshot 2024-09-12T13:53:58.130 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:53:58.130 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:53:58.130 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:53:58.130 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mode=snapshot 2024-09-12T13:53:58.130 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 mirror image enable mirror/image-primary7 snapshot 2024-09-12T13:53:58.319 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local ret=0 2024-09-12T13:53:58.319 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ timeout 5m bash -c 'zcat test-primary6/linux-5.4.tar.gz | pv -L 256K | tar xf - -C test-primary6' 2024-09-12T13:53:58.408 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Mirroring enabled 2024-09-12T13:53:58.421 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:53:58.421 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:53:58.421 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:53:58.421 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:53:58.421 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:53:58.421 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:53:58.423 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2325 2024-09-12T13:53:58.425 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2327 2024-09-12T13:53:58.438 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 info mirror/image-primary7 2024-09-12T13:53:58.445 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:53:58.447 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:53:58.521 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:rbd image 'image-primary7': 2024-09-12T13:53:58.521 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: size 10 GiB in 2560 objects 2024-09-12T13:53:58.522 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: order 22 (4 MiB objects) 2024-09-12T13:53:58.522 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: snapshot_count: 1 2024-09-12T13:53:58.522 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: id: 3393ad7fc398 2024-09-12T13:53:58.522 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: block_name_prefix: rbd_data.3393ad7fc398 2024-09-12T13:53:58.522 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: format: 2 2024-09-12T13:53:58.522 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: features: layering, exclusive-lock, object-map, fast-diff 2024-09-12T13:53:58.522 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: op_features: 2024-09-12T13:53:58.522 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: flags: 2024-09-12T13:53:58.522 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: create_timestamp: Thu Sep 12 13:53:58 2024 2024-09-12T13:53:58.522 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: access_timestamp: Thu Sep 12 13:53:58 2024 2024-09-12T13:53:58.522 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: modify_timestamp: Thu Sep 12 13:53:58 2024 2024-09-12T13:53:58.522 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring state: enabled 2024-09-12T13:53:58.522 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring mode: snapshot 2024-09-12T13:53:58.523 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring global id: 700ac9cd-0dca-415e-b998-79da95582617 2024-09-12T13:53:58.523 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring primary: true 2024-09-12T13:53:58.527 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \n\b\d ]] 2024-09-12T13:53:58.527 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \k\r\b\d ]] 2024-09-12T13:53:58.528 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo rbd --cluster cluster1 device map -t krbd mirror/image-primary7 2024-09-12T13:53:58.683 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEV=/dev/rbd6 2024-09-12T13:53:58.683 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEVS+=($DEV) 2024-09-12T13:53:58.683 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo mkfs.ext4 /dev/rbd6 2024-09-12T13:53:58.704 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:mke2fs 1.46.5 (30-Dec-2021) 2024-09-12T13:53:58.715 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Discarding device blocks: 0/2621440 done 2024-09-12T13:53:58.716 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Creating filesystem with 2621440 4k blocks and 655360 inodes 2024-09-12T13:53:58.716 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Filesystem UUID: cf5d1f96-d4bd-4550-9077-c3611e8fe279 2024-09-12T13:53:58.716 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Superblock backups stored on blocks: 2024-09-12T13:53:58.716 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 2024-09-12T13:53:58.716 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 2024-09-12T13:53:58.716 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Allocating group tables: 0/80 done 2024-09-12T13:53:58.716 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Writing inode tables: 0/80 done 2024-09-12T13:53:58.721 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Creating journal (16384 blocks): done 2024-09-12T13:53:58.835 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Writing superblocks and filesystem accounting information: 0/80 done 2024-09-12T13:53:58.836 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 2024-09-12T13:53:58.838 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mkdir test-primary7 2024-09-12T13:53:58.841 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo mount /dev/rbd6 test-primary7 2024-09-12T13:53:58.918 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ whoami 2024-09-12T13:53:58.920 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo chown ubuntu test-primary7 2024-09-12T13:53:58.946 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ SNAP_PIDS+=($!) 2024-09-12T13:53:58.947 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ take_mirror_snapshots cluster1 mirror image-primary7 2024-09-12T13:53:58.947 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:53:58.947 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:53:58.947 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:53:58.947 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:53:58.947 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:53:58.947 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:53:58.947 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ WORKLOAD_PIDS+=($!) 2024-09-12T13:53:58.947 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:53:58.947 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:53:58.947 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:53:58.948 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:53:58.948 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ IMG=image-primary8 2024-09-12T13:53:58.948 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ MNTPT=test-primary8 2024-09-12T13:53:58.948 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ create_image_and_enable_mirror cluster1 mirror image-primary8 snapshot 10G 2024-09-12T13:53:58.948 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ slow_untar_workload test-primary7 2024-09-12T13:53:58.948 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:53:58.948 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:53:58.948 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:53:58.948 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:53:58.948 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:53:58.948 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:53:58.948 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mode=snapshot 2024-09-12T13:53:58.949 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n snapshot ']' 2024-09-12T13:53:58.949 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:53:58.949 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ create_image cluster1 mirror image-primary8 10G 2024-09-12T13:53:58.949 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:53:58.949 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:53:58.949 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:53:58.949 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:53:58.949 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:53:58.949 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:53:58.949 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mountpt=test-primary7 2024-09-12T13:53:58.949 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local size=128 2024-09-12T13:53:58.950 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n 10G ']' 2024-09-12T13:53:58.950 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ size=10G 2024-09-12T13:53:58.950 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:53:58.950 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 create --size 10G --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image-primary8 2024-09-12T13:53:58.950 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ cp linux-5.4.tar.gz test-primary7 2024-09-12T13:53:59.021 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ is_pool_mirror_mode_image mirror 2024-09-12T13:53:59.021 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:53:59.021 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' image = image ']' 2024-09-12T13:53:59.021 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:53:59.021 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ enable_mirror cluster1 mirror image-primary8 snapshot 2024-09-12T13:53:59.021 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:53:59.021 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:53:59.021 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:53:59.021 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mode=snapshot 2024-09-12T13:53:59.021 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 mirror image enable mirror/image-primary8 snapshot 2024-09-12T13:53:59.243 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local ret=0 2024-09-12T13:53:59.243 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ timeout 5m bash -c 'zcat test-primary7/linux-5.4.tar.gz | pv -L 256K | tar xf - -C test-primary7' 2024-09-12T13:53:59.409 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Mirroring enabled 2024-09-12T13:53:59.415 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2328 2024-09-12T13:53:59.415 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2329 2024-09-12T13:53:59.433 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:53:59.435 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:53:59.435 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 info mirror/image-primary8 2024-09-12T13:53:59.454 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:53:59.456 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:53:59.456 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:53:59.456 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:53:59.456 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:53:59.457 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:53:59.538 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:rbd image 'image-primary8': 2024-09-12T13:53:59.538 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: size 10 GiB in 2560 objects 2024-09-12T13:53:59.538 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: order 22 (4 MiB objects) 2024-09-12T13:53:59.538 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: snapshot_count: 1 2024-09-12T13:53:59.538 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: id: 33a7315e15d5 2024-09-12T13:53:59.538 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: block_name_prefix: rbd_data.33a7315e15d5 2024-09-12T13:53:59.538 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: format: 2 2024-09-12T13:53:59.538 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: features: layering, exclusive-lock, object-map, fast-diff 2024-09-12T13:53:59.538 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: op_features: 2024-09-12T13:53:59.539 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: flags: 2024-09-12T13:53:59.539 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: create_timestamp: Thu Sep 12 13:53:59 2024 2024-09-12T13:53:59.539 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: access_timestamp: Thu Sep 12 13:53:59 2024 2024-09-12T13:53:59.539 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: modify_timestamp: Thu Sep 12 13:53:59 2024 2024-09-12T13:53:59.539 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring state: enabled 2024-09-12T13:53:59.539 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring mode: snapshot 2024-09-12T13:53:59.539 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring global id: 3b4d0b11-5c89-4212-85d3-673c05d34c4b 2024-09-12T13:53:59.539 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring primary: true 2024-09-12T13:53:59.549 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \n\b\d ]] 2024-09-12T13:53:59.549 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \k\r\b\d ]] 2024-09-12T13:53:59.551 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo rbd --cluster cluster1 device map -t krbd mirror/image-primary8 2024-09-12T13:53:59.685 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEV=/dev/rbd7 2024-09-12T13:53:59.685 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEVS+=($DEV) 2024-09-12T13:53:59.685 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo mkfs.ext4 /dev/rbd7 2024-09-12T13:53:59.706 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:mke2fs 1.46.5 (30-Dec-2021) 2024-09-12T13:53:59.718 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Discarding device blocks: 0/2621440 done 2024-09-12T13:53:59.718 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Creating filesystem with 2621440 4k blocks and 655360 inodes 2024-09-12T13:53:59.718 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Filesystem UUID: 125080ae-a367-481e-bc1d-47b61344f754 2024-09-12T13:53:59.718 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Superblock backups stored on blocks: 2024-09-12T13:53:59.718 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 2024-09-12T13:53:59.718 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 2024-09-12T13:53:59.718 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Allocating group tables: 0/80 done 2024-09-12T13:53:59.718 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Writing inode tables: 0/80 done 2024-09-12T13:53:59.723 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Creating journal (16384 blocks): done 2024-09-12T13:53:59.777 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Writing superblocks and filesystem accounting information: 0/80 done 2024-09-12T13:53:59.777 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 2024-09-12T13:53:59.780 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mkdir test-primary8 2024-09-12T13:53:59.783 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo mount /dev/rbd7 test-primary8 2024-09-12T13:53:59.838 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ whoami 2024-09-12T13:53:59.839 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo chown ubuntu test-primary8 2024-09-12T13:53:59.865 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ SNAP_PIDS+=($!) 2024-09-12T13:53:59.865 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ take_mirror_snapshots cluster1 mirror image-primary8 2024-09-12T13:53:59.865 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:53:59.865 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:53:59.865 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:53:59.865 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:53:59.865 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:53:59.865 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:53:59.866 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:53:59.866 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:53:59.866 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:53:59.866 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ WORKLOAD_PIDS+=($!) 2024-09-12T13:53:59.866 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:53:59.866 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ slow_untar_workload test-primary8 2024-09-12T13:53:59.867 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ IMG=image-primary9 2024-09-12T13:53:59.867 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ MNTPT=test-primary9 2024-09-12T13:53:59.867 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ create_image_and_enable_mirror cluster1 mirror image-primary9 snapshot 10G 2024-09-12T13:53:59.867 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mountpt=test-primary8 2024-09-12T13:53:59.868 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ cp linux-5.4.tar.gz test-primary8 2024-09-12T13:53:59.868 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:53:59.868 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:53:59.868 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:53:59.868 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:53:59.868 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:53:59.868 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:53:59.868 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mode=snapshot 2024-09-12T13:53:59.868 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n snapshot ']' 2024-09-12T13:53:59.869 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:53:59.869 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ create_image cluster1 mirror image-primary9 10G 2024-09-12T13:53:59.869 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:53:59.869 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:53:59.869 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:53:59.869 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:53:59.869 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:53:59.869 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:53:59.869 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local size=128 2024-09-12T13:53:59.869 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n 10G ']' 2024-09-12T13:53:59.869 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ size=10G 2024-09-12T13:53:59.869 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:53:59.869 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 create --size 10G --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image-primary9 2024-09-12T13:53:59.938 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ is_pool_mirror_mode_image mirror 2024-09-12T13:53:59.938 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:53:59.938 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' image = image ']' 2024-09-12T13:53:59.938 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:53:59.938 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ enable_mirror cluster1 mirror image-primary9 snapshot 2024-09-12T13:53:59.938 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:53:59.938 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:53:59.938 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:53:59.938 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mode=snapshot 2024-09-12T13:53:59.938 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 mirror image enable mirror/image-primary9 snapshot 2024-09-12T13:54:00.165 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local ret=0 2024-09-12T13:54:00.165 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ timeout 5m bash -c 'zcat test-primary8/linux-5.4.tar.gz | pv -L 256K | tar xf - -C test-primary8' 2024-09-12T13:54:00.411 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Mirroring enabled 2024-09-12T13:54:00.419 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2332 2024-09-12T13:54:00.426 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2331 2024-09-12T13:54:00.438 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:54:00.440 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 info mirror/image-primary9 2024-09-12T13:54:00.450 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:54:00.470 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:54:00.470 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:54:00.470 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:54:00.471 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:54:00.471 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:54:00.471 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:54:00.478 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:54:00.478 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:54:00.479 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:54:00.479 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:54:00.479 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:54:00.479 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:54:00.560 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:rbd image 'image-primary9': 2024-09-12T13:54:00.560 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: size 10 GiB in 2560 objects 2024-09-12T13:54:00.561 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: order 22 (4 MiB objects) 2024-09-12T13:54:00.561 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: snapshot_count: 1 2024-09-12T13:54:00.561 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: id: 33b9c7568e4a 2024-09-12T13:54:00.561 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: block_name_prefix: rbd_data.33b9c7568e4a 2024-09-12T13:54:00.561 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: format: 2 2024-09-12T13:54:00.561 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: features: layering, exclusive-lock, object-map, fast-diff 2024-09-12T13:54:00.561 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: op_features: 2024-09-12T13:54:00.561 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: flags: 2024-09-12T13:54:00.561 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: create_timestamp: Thu Sep 12 13:53:59 2024 2024-09-12T13:54:00.561 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: access_timestamp: Thu Sep 12 13:53:59 2024 2024-09-12T13:54:00.561 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: modify_timestamp: Thu Sep 12 13:53:59 2024 2024-09-12T13:54:00.561 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring state: enabled 2024-09-12T13:54:00.562 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring mode: snapshot 2024-09-12T13:54:00.562 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring global id: 41a87723-a798-494d-b064-df92256304f2 2024-09-12T13:54:00.562 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring primary: true 2024-09-12T13:54:00.568 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \n\b\d ]] 2024-09-12T13:54:00.568 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \k\r\b\d ]] 2024-09-12T13:54:00.569 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo rbd --cluster cluster1 device map -t krbd mirror/image-primary9 2024-09-12T13:54:00.696 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEV=/dev/rbd8 2024-09-12T13:54:00.696 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEVS+=($DEV) 2024-09-12T13:54:00.696 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo mkfs.ext4 /dev/rbd8 2024-09-12T13:54:00.717 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:mke2fs 1.46.5 (30-Dec-2021) 2024-09-12T13:54:00.729 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Discarding device blocks: 0/2621440 done 2024-09-12T13:54:00.729 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Creating filesystem with 2621440 4k blocks and 655360 inodes 2024-09-12T13:54:00.729 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Filesystem UUID: 73307011-c2d9-4560-b0e3-cfe348c85908 2024-09-12T13:54:00.729 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Superblock backups stored on blocks: 2024-09-12T13:54:00.730 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 2024-09-12T13:54:00.730 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 2024-09-12T13:54:00.730 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Allocating group tables: 0/80 done 2024-09-12T13:54:00.730 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Writing inode tables: 0/80 done 2024-09-12T13:54:00.734 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Creating journal (16384 blocks): done 2024-09-12T13:54:00.790 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Writing superblocks and filesystem accounting information: 0/80 done 2024-09-12T13:54:00.790 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 2024-09-12T13:54:00.792 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mkdir test-primary9 2024-09-12T13:54:00.794 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo mount /dev/rbd8 test-primary9 2024-09-12T13:54:00.851 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ whoami 2024-09-12T13:54:00.853 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo chown ubuntu test-primary9 2024-09-12T13:54:00.878 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ SNAP_PIDS+=($!) 2024-09-12T13:54:00.878 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ take_mirror_snapshots cluster1 mirror image-primary9 2024-09-12T13:54:00.878 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:54:00.878 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:54:00.878 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:54:00.878 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:54:00.878 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:54:00.878 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:54:00.879 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ WORKLOAD_PIDS+=($!) 2024-09-12T13:54:00.879 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:54:00.879 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:54:00.879 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:54:00.879 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:54:00.879 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ IMG=image-primary10 2024-09-12T13:54:00.879 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ MNTPT=test-primary10 2024-09-12T13:54:00.879 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ slow_untar_workload test-primary9 2024-09-12T13:54:00.879 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ create_image_and_enable_mirror cluster1 mirror image-primary10 snapshot 10G 2024-09-12T13:54:00.879 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:54:00.879 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:54:00.880 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:54:00.880 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:54:00.880 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:54:00.880 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:54:00.880 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mode=snapshot 2024-09-12T13:54:00.880 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n snapshot ']' 2024-09-12T13:54:00.880 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:54:00.880 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ create_image cluster1 mirror image-primary10 10G 2024-09-12T13:54:00.880 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:54:00.880 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:54:00.880 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mountpt=test-primary9 2024-09-12T13:54:00.880 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:54:00.881 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:54:00.881 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:54:00.881 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:54:00.881 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local size=128 2024-09-12T13:54:00.881 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n 10G ']' 2024-09-12T13:54:00.881 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ cp linux-5.4.tar.gz test-primary9 2024-09-12T13:54:00.881 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ size=10G 2024-09-12T13:54:00.881 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T13:54:00.881 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 create --size 10G --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image-primary10 2024-09-12T13:54:00.949 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ is_pool_mirror_mode_image mirror 2024-09-12T13:54:00.952 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:54:00.952 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' image = image ']' 2024-09-12T13:54:00.952 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T13:54:00.952 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ enable_mirror cluster1 mirror image-primary10 snapshot 2024-09-12T13:54:00.953 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:54:00.953 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:54:00.953 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:54:00.953 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mode=snapshot 2024-09-12T13:54:00.953 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 mirror image enable mirror/image-primary10 snapshot 2024-09-12T13:54:01.177 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local ret=0 2024-09-12T13:54:01.177 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ timeout 5m bash -c 'zcat test-primary9/linux-5.4.tar.gz | pv -L 256K | tar xf - -C test-primary9' 2024-09-12T13:54:01.432 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2334 2024-09-12T13:54:01.432 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2335 2024-09-12T13:54:01.432 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2336 2024-09-12T13:54:01.434 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Mirroring enabled 2024-09-12T13:54:01.450 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:54:01.451 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:54:01.451 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:54:01.451 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:54:01.451 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:54:01.451 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:54:01.451 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:54:01.451 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:54:01.451 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:54:01.451 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:54:01.451 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:54:01.451 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:54:01.455 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:54:01.457 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:54:01.484 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 info mirror/image-primary10 2024-09-12T13:54:01.493 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:54:01.620 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:rbd image 'image-primary10': 2024-09-12T13:54:01.620 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: size 10 GiB in 2560 objects 2024-09-12T13:54:01.620 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: order 22 (4 MiB objects) 2024-09-12T13:54:01.620 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: snapshot_count: 1 2024-09-12T13:54:01.620 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: id: 33cea897d895 2024-09-12T13:54:01.620 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: block_name_prefix: rbd_data.33cea897d895 2024-09-12T13:54:01.620 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: format: 2 2024-09-12T13:54:01.620 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: features: layering, exclusive-lock, object-map, fast-diff 2024-09-12T13:54:01.620 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: op_features: 2024-09-12T13:54:01.620 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: flags: 2024-09-12T13:54:01.620 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: create_timestamp: Thu Sep 12 13:54:00 2024 2024-09-12T13:54:01.620 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: access_timestamp: Thu Sep 12 13:54:00 2024 2024-09-12T13:54:01.621 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: modify_timestamp: Thu Sep 12 13:54:00 2024 2024-09-12T13:54:01.621 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring state: enabled 2024-09-12T13:54:01.621 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring mode: snapshot 2024-09-12T13:54:01.621 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring global id: 3238adfd-24b7-48b2-ab32-66aaa1c7b583 2024-09-12T13:54:01.621 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring primary: true 2024-09-12T13:54:01.627 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \n\b\d ]] 2024-09-12T13:54:01.627 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \k\r\b\d ]] 2024-09-12T13:54:01.627 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo rbd --cluster cluster1 device map -t krbd mirror/image-primary10 2024-09-12T13:54:01.761 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEV=/dev/rbd9 2024-09-12T13:54:01.761 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEVS+=($DEV) 2024-09-12T13:54:01.761 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo mkfs.ext4 /dev/rbd9 2024-09-12T13:54:01.782 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:mke2fs 1.46.5 (30-Dec-2021) 2024-09-12T13:54:01.797 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Discarding device blocks: 0/2621440 done 2024-09-12T13:54:01.797 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Creating filesystem with 2621440 4k blocks and 655360 inodes 2024-09-12T13:54:01.797 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Filesystem UUID: b391e850-116f-4f20-b61e-564a05006da1 2024-09-12T13:54:01.798 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Superblock backups stored on blocks: 2024-09-12T13:54:01.798 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 2024-09-12T13:54:01.798 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 2024-09-12T13:54:01.798 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Allocating group tables: 0/80 done 2024-09-12T13:54:01.798 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Writing inode tables: 0/80 done 2024-09-12T13:54:01.802 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Creating journal (16384 blocks): done 2024-09-12T13:54:01.860 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Writing superblocks and filesystem accounting information: 0/80 done 2024-09-12T13:54:01.860 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 2024-09-12T13:54:01.862 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mkdir test-primary10 2024-09-12T13:54:01.865 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo mount /dev/rbd9 test-primary10 2024-09-12T13:54:01.921 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ whoami 2024-09-12T13:54:01.922 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo chown ubuntu test-primary10 2024-09-12T13:54:01.947 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ SNAP_PIDS+=($!) 2024-09-12T13:54:01.947 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ take_mirror_snapshots cluster1 mirror image-primary10 2024-09-12T13:54:01.947 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:54:01.947 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:54:01.947 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:54:01.947 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:54:01.947 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:54:01.947 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ WORKLOAD_PIDS+=($!) 2024-09-12T13:54:01.947 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:54:01.947 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:54:01.947 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:54:01.947 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:54:01.948 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${SNAP_PIDS[@]} 2024-09-12T13:54:01.948 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 118259 2024-09-12T13:54:01.948 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ slow_untar_workload test-primary10 2024-09-12T13:54:01.948 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mountpt=test-primary10 2024-09-12T13:54:01.948 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ cp linux-5.4.tar.gz test-primary10 2024-09-12T13:54:02.255 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local ret=0 2024-09-12T13:54:02.255 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ timeout 5m bash -c 'zcat test-primary10/linux-5.4.tar.gz | pv -L 256K | tar xf - -C test-primary10' 2024-09-12T13:54:02.438 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:54:02.439 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:54:02.439 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:54:02.439 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:54:02.439 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:54:02.439 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:54:02.440 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:54:02.440 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:54:02.440 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:54:02.440 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:54:02.440 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:54:02.440 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:54:02.457 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2340 2024-09-12T13:54:02.496 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:54:02.525 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2338 2024-09-12T13:54:02.536 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2339 2024-09-12T13:54:02.578 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:54:02.615 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:54:03.345 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2341 2024-09-12T13:54:03.364 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2342 2024-09-12T13:54:03.370 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:54:03.381 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:54:03.442 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:54:03.442 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:54:03.442 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:54:03.442 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:54:03.442 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:54:03.443 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:54:03.456 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:54:03.456 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:54:03.456 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:54:03.456 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:54:03.456 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:54:03.456 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:54:04.368 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2344 2024-09-12T13:54:04.393 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2343 2024-09-12T13:54:04.416 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:54:04.420 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:54:04.462 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:54:04.462 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:54:04.462 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:54:04.463 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:54:04.463 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:54:04.463 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:54:04.472 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:54:04.472 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:54:04.473 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:54:04.473 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:54:04.473 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:54:04.473 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:54:04.498 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:54:04.498 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:54:04.498 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:54:04.498 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:54:04.498 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:54:04.498 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:54:05.480 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2347 2024-09-12T13:54:05.495 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2345 2024-09-12T13:54:05.502 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:54:05.502 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:54:05.502 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:54:05.502 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:54:05.502 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:54:05.502 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:54:05.502 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2346 2024-09-12T13:54:05.524 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:54:05.548 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:54:05.562 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:54:05.583 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:54:05.583 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:54:05.583 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:54:05.584 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:54:05.584 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:54:05.584 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:54:05.619 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:54:05.620 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:54:05.620 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:54:05.620 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:54:05.620 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:54:05.620 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:54:06.374 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:54:06.374 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:54:06.374 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:54:06.374 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:54:06.374 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:54:06.375 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:54:06.386 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:54:06.386 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:54:06.386 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:54:06.386 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:54:06.386 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:54:06.386 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:54:06.619 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2349 2024-09-12T13:54:06.626 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2350 2024-09-12T13:54:06.637 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2348 2024-09-12T13:54:06.670 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:54:06.673 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:54:06.679 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:54:07.423 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:54:07.424 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:54:07.424 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:54:07.424 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:54:07.424 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:54:07.424 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:54:07.426 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:54:07.426 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:54:07.426 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:54:07.426 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:54:07.426 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:54:07.426 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:54:07.604 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2351 2024-09-12T13:54:07.620 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2352 2024-09-12T13:54:07.633 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:54:07.658 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:54:08.440 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2353 2024-09-12T13:54:08.471 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2354 2024-09-12T13:54:08.481 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:54:08.512 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:54:08.534 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:54:08.534 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:54:08.534 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:54:08.534 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:54:08.534 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:54:08.534 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:54:08.559 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:54:08.559 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:54:08.559 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:54:08.559 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:54:08.559 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:54:08.559 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:54:08.568 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:54:08.568 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:54:08.569 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:54:08.570 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:54:08.570 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:54:08.570 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:54:09.423 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2356 2024-09-12T13:54:09.459 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2355 2024-09-12T13:54:09.471 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2357 2024-09-12T13:54:09.500 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:54:09.500 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:54:09.531 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:54:09.673 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:54:09.673 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:54:09.674 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:54:09.674 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:54:09.674 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:54:09.674 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:54:09.676 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:54:09.677 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:54:09.677 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:54:09.677 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:54:09.677 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:54:09.677 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:54:09.682 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:54:09.682 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:54:09.682 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:54:09.682 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:54:09.682 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:54:09.682 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:54:10.413 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2359 2024-09-12T13:54:10.483 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2358 2024-09-12T13:54:10.517 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:54:10.522 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2360 2024-09-12T13:54:10.545 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:54:10.569 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:54:10.637 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:54:10.637 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:54:10.637 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:54:10.637 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:54:10.637 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:54:10.638 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:54:10.666 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:54:10.667 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:54:10.667 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:54:10.667 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:54:10.667 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:54:10.667 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:54:11.349 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2361 2024-09-12T13:54:11.360 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2362 2024-09-12T13:54:11.375 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:54:11.382 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:54:11.485 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:54:11.485 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:54:11.486 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:54:11.486 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:54:11.486 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:54:11.486 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:54:11.520 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:54:11.521 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:54:11.521 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:54:11.521 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:54:11.521 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:54:11.521 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:54:12.352 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2363 2024-09-12T13:54:12.380 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:54:12.383 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2364 2024-09-12T13:54:12.413 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:54:12.507 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:54:12.507 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:54:12.507 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:54:12.507 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:54:12.507 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:54:12.507 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:54:12.509 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:54:12.509 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:54:12.509 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:54:12.509 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:54:12.509 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:54:12.509 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:54:12.540 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:54:12.540 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:54:12.540 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:54:12.540 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:54:12.541 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:54:12.541 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:54:13.366 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2366 2024-09-12T13:54:13.373 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2367 2024-09-12T13:54:13.383 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:54:13.427 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:54:13.524 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:54:13.524 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:54:13.524 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:54:13.524 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:54:13.524 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:54:13.524 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:54:13.549 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:54:13.549 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:54:13.549 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:54:13.549 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:54:13.550 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:54:13.550 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:54:13.573 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:54:13.574 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:54:13.574 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:54:13.574 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:54:13.574 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:54:13.574 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:54:14.379 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:54:14.379 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:54:14.379 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:54:14.379 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:54:14.379 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:54:14.379 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:54:14.385 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:54:14.385 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:54:14.385 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:54:14.385 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:54:14.386 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:54:14.386 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:54:14.436 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2368 2024-09-12T13:54:14.450 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2369 2024-09-12T13:54:14.511 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:54:14.515 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:54:15.370 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2365 2024-09-12T13:54:15.386 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:54:15.386 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:54:15.386 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:54:15.386 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:54:15.386 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:54:15.386 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:54:15.419 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:54:15.420 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:54:15.420 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:54:15.420 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:54:15.420 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:54:15.420 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:54:15.451 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:54:15.451+0000 7f67dc600640 -1 rbd::mirror::image_replayer::snapshot::Replayer: 0x56055c1a2400 handle_unlink_peer: failed to unlink local peer from remote image: (22) Invalid argument 2024-09-12T13:54:15.457 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:54:15.515 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2371 2024-09-12T13:54:15.568 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:54:16.363 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2370 2024-09-12T13:54:16.388 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:54:16.388 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:54:16.388 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:54:16.388 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:54:16.388 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:54:16.388 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:54:16.420 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2372 2024-09-12T13:54:16.431 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:54:16.431 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:54:16.431 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:54:16.431 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:54:16.431 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:54:16.431 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:54:16.435 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:54:16.470 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:54:16.504 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2373 2024-09-12T13:54:16.542 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:54:17.486 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2375 2024-09-12T13:54:17.514 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:54:17.514 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:54:17.514 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:54:17.514 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:54:17.514 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:54:17.515 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:54:17.520 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:54:17.520 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:54:17.520 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:54:17.520 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:54:17.520 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:54:17.521 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:54:17.521 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:54:18.327 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2374 2024-09-12T13:54:18.370 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:54:18.384 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2376 2024-09-12T13:54:18.393 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2377 2024-09-12T13:54:18.408 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:54:18.427 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:54:18.461 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:54:18.461 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:54:18.461 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:54:18.461 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:54:18.461 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:54:18.461 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:54:18.571 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:54:18.571 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:54:18.571 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:54:18.571 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:54:18.572 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:54:18.572 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:54:19.438 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:54:19.438 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:54:19.438 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:54:19.438 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:54:19.438 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:54:19.438 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:54:19.439 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2379 2024-09-12T13:54:19.453 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:54:19.473 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:54:19.473 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:54:19.473 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:54:19.473 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:54:19.473 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:54:19.473 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:54:19.544 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:54:19.545 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:54:19.545 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:54:19.545 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:54:19.545 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:54:19.545 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:54:20.369 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2378 2024-09-12T13:54:20.421 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:54:20.460 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2381 2024-09-12T13:54:20.481 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:54:20.522 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:54:20.522 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:54:20.522 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:54:20.522 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:54:20.522 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:54:20.522 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:54:21.369 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2380 2024-09-12T13:54:21.374 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:54:21.375 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:54:21.375 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:54:21.375 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:54:21.375 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:54:21.375 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:54:21.414 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:54:21.414 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:54:21.414 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:54:21.414 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:54:21.415 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:54:21.415 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:54:21.428 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:54:21.430 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:54:21.430 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:54:21.431 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:54:21.431 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:54:21.431 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:54:21.431 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:54:21.482 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:54:21.482+0000 7f67dc600640 -1 rbd::mirror::image_replayer::snapshot::Replayer: 0x56055ba32400 handle_unlink_peer: failed to unlink local peer from remote image: (22) Invalid argument 2024-09-12T13:54:22.380 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2383 2024-09-12T13:54:22.388 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2382 2024-09-12T13:54:22.437 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:54:22.439 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:54:22.456 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:54:22.456 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:54:22.456 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:54:22.456 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:54:22.456 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:54:22.456 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:54:22.475 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:54:22.475+0000 7f67dc600640 -1 rbd::mirror::image_replayer::snapshot::Replayer: 0x56055c2d1000 handle_unlink_peer: failed to unlink local peer from remote image: (22) Invalid argument 2024-09-12T13:54:22.514 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2385 2024-09-12T13:54:22.533 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:54:23.312 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2384 2024-09-12T13:54:23.353 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:54:23.414 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2386 2024-09-12T13:54:23.425 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:54:23.425 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:54:23.425 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:54:23.425 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:54:23.426 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:54:23.426 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:54:23.426 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2387 2024-09-12T13:54:23.457 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:54:23.461 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:54:23.484 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:54:23.484 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:54:23.484 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:54:23.484 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:54:23.484 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:54:23.484 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:54:24.434 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:54:24.434 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:54:24.434 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:54:24.435 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:54:24.435 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:54:24.435 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:54:24.440 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2388 2024-09-12T13:54:24.487 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:54:24.548 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2389 2024-09-12T13:54:24.618 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:54:25.440 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:54:25.440 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:54:25.440 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:54:25.440 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:54:25.440 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:54:25.440 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:54:25.441 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:54:25.441 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:54:25.441 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:54:25.442 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:54:25.442 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:54:25.442 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:54:25.536 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:54:25.536 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:54:25.536 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:54:25.536 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:54:25.536 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:54:25.536 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:54:25.729 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2391 2024-09-12T13:54:25.776 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:54:26.356 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:54:26.356 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:54:26.356 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:54:26.356 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:54:26.357 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:54:26.357 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:54:26.403 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2390 2024-09-12T13:54:26.457 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:54:26.460 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:54:26.460 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:54:26.460 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:54:26.460 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:54:26.460 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:54:26.460 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:54:26.463 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:54:26.463 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:54:26.464 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:54:26.464 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:54:26.464 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:54:26.464 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:54:26.516 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2392 2024-09-12T13:54:26.571 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2394 2024-09-12T13:54:26.592 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:54:26.603 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:54:27.477 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2393 2024-09-12T13:54:27.490 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:54:27.490 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:54:27.491 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:54:27.491 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:54:27.491 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:54:27.491 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:54:27.515 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:54:27.537 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2395 2024-09-12T13:54:27.551 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:54:27.620 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:54:27.620 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:54:27.621 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:54:27.621 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:54:27.621 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:54:27.621 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:54:28.378 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2396 2024-09-12T13:54:28.405 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2397 2024-09-12T13:54:28.411 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:54:28.437 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:54:28.779 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:54:28.779 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:54:28.779 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:54:28.779 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:54:28.779 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:54:28.779 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:54:29.461 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:54:29.461 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:54:29.461 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:54:29.461 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:54:29.461 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:54:29.461 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:54:29.480 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2398 2024-09-12T13:54:29.510 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2399 2024-09-12T13:54:29.542 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:54:29.550 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:54:29.594 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:54:29.594 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:54:29.594 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:54:29.594 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:54:29.594 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:54:29.595 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:54:29.606 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:54:29.606 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:54:29.606 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:54:29.606 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:54:29.606 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:54:29.606 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:54:30.458 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2400 2024-09-12T13:54:30.471 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:54:30.477 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2402 2024-09-12T13:54:30.481 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2401 2024-09-12T13:54:30.498 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:54:30.498 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:54:30.518 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:54:30.518 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:54:30.518 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:54:30.518 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:54:30.518 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:54:30.518 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:54:30.554 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:54:30.554 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:54:30.554 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:54:30.554 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:54:30.554 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:54:30.554 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:54:31.416 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:54:31.416 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:54:31.416 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:54:31.416 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:54:31.416 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:54:31.416 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:54:31.444 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:54:31.444 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:54:31.444 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:54:31.444 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:54:31.444 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:54:31.444 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:54:31.491 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2403 2024-09-12T13:54:31.549 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2405 2024-09-12T13:54:31.552 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:54:31.579 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:54:32.545 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:54:32.545 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:54:32.545 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:54:32.545 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:54:32.545 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:54:32.546 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:54:32.561 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:54:32.561 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:54:32.561 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:54:32.561 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:54:32.561 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:54:32.561 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:54:33.315 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2404 2024-09-12T13:54:33.399 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:54:33.403 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2407 2024-09-12T13:54:33.433 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2406 2024-09-12T13:54:33.462 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:54:33.475 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:54:33.475 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:54:33.476 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:54:33.476 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:54:33.476 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:54:33.476 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:54:33.477 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:54:33.500 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:54:33.501 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:54:33.501 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:54:33.501 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:54:33.501 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:54:33.501 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:54:33.501 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:54:33.501 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:54:33.501 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:54:33.501 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:54:33.502 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:54:33.502 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:54:34.441 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2409 2024-09-12T13:54:34.465 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:54:34.510 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2411 2024-09-12T13:54:34.561 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:54:34.561 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:54:34.561 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:54:34.561 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:54:34.561 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:54:34.561 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:54:34.570 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:54:34.581 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:54:34.582 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:54:34.582 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:54:34.582 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:54:34.582 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:54:34.582 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:54:35.439 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2408 2024-09-12T13:54:35.505 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:54:35.594 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2414 2024-09-12T13:54:35.627 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:54:36.403 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:54:36.403 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:54:36.403 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:54:36.403 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:54:36.403 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:54:36.403 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:54:36.465 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2412 2024-09-12T13:54:36.466 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:54:36.466 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:54:36.466 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:54:36.467 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:54:36.467 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:54:36.467 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:54:36.471 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2410 2024-09-12T13:54:36.482 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:54:36.482 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:54:36.482 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:54:36.482 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:54:36.482 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:54:36.482 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:54:36.515 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:54:36.558 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:54:37.435 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2413 2024-09-12T13:54:37.468 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:54:37.468 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:54:37.469 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:54:37.469 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:54:37.469 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:54:37.469 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:54:37.480 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:54:37.513 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2415 2024-09-12T13:54:37.531 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:54:37.573 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:54:37.574 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:54:37.574 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:54:37.574 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:54:37.574 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:54:37.574 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:54:38.388 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2416 2024-09-12T13:54:38.396 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2417 2024-09-12T13:54:38.410 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:54:38.416 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:54:38.439 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2419 2024-09-12T13:54:38.456 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:54:38.508 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:54:38.508 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:54:38.508 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:54:38.508 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:54:38.508 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:54:38.509 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:54:38.630 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:54:38.630 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:54:38.630 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:54:38.630 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:54:38.630 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:54:38.631 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:54:39.380 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2418 2024-09-12T13:54:39.387 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2420 2024-09-12T13:54:39.390 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2421 2024-09-12T13:54:39.395 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:54:39.404 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:54:39.408 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:54:39.517 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:54:39.517 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:54:39.517 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:54:39.517 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:54:39.518 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:54:39.518 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:54:39.560 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:54:39.560 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:54:39.560 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:54:39.561 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:54:39.561 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:54:39.561 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:54:40.381 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2423 2024-09-12T13:54:40.387 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2422 2024-09-12T13:54:40.397 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:54:40.405 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:54:40.483 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:54:40.483 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:54:40.484 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:54:40.484 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:54:40.484 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:54:40.484 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:54:40.534 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:54:40.534 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:54:40.534 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:54:40.535 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:54:40.535 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:54:40.535 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:54:41.394 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2424 2024-09-12T13:54:41.406 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:54:41.413 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:54:41.413 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:54:41.413 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:54:41.414 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:54:41.414 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:54:41.414 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:54:41.419 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:54:41.419 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:54:41.419 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:54:41.419 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:54:41.419 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:54:41.419 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:54:41.458 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:54:41.459 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:54:41.459 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:54:41.459 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:54:41.459 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:54:41.459 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:54:42.398 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:54:42.399 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:54:42.399 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:54:42.399 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:54:42.399 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:54:42.399 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:54:42.406 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:54:42.406 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:54:42.406 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:54:42.406 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:54:42.406 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:54:42.407 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:54:42.411 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:54:42.411 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:54:42.411 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:54:42.411 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:54:42.411 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:54:42.411 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:54:42.413 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2425 2024-09-12T13:54:42.427 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:54:43.340 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2428 2024-09-12T13:54:43.390 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2426 2024-09-12T13:54:43.401 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:54:43.401 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:54:43.402 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:54:43.402 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:54:43.402 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:54:43.402 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:54:43.402 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:54:43.408 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:54:43.409 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:54:43.409 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:54:43.409 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:54:43.409 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:54:43.409 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:54:43.427 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:54:43.432 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2427 2024-09-12T13:54:43.454 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:54:44.356 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2431 2024-09-12T13:54:44.409 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:54:44.409 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:54:44.410 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:54:44.410 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:54:44.410 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:54:44.410 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:54:44.419 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:54:44.448 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2430 2024-09-12T13:54:44.520 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:54:45.381 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2429 2024-09-12T13:54:45.429 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:54:45.430 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:54:45.430 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:54:45.430 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:54:45.430 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:54:45.430 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:54:45.444 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:54:46.345 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2432 2024-09-12T13:54:46.352 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2433 2024-09-12T13:54:46.402 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:54:46.405 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:54:46.405 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:54:46.405 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:54:46.405 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:54:46.405 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:54:46.405 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:54:46.410 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:54:46.425 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2434 2024-09-12T13:54:46.431 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:54:46.431 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:54:46.431 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:54:46.431 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:54:46.431 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:54:46.431 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:54:46.440 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:54:46.459 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:54:46.459 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:54:46.459 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:54:46.459 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:54:46.459 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:54:46.459 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:54:47.422 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:54:47.422 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:54:47.422 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:54:47.422 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:54:47.422 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:54:47.423 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:54:47.523 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:54:47.523 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:54:47.523 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:54:47.523 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:54:47.523 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:54:47.523 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:54:48.353 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2435 2024-09-12T13:54:48.360 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2436 2024-09-12T13:54:48.396 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2437 2024-09-12T13:54:48.405 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:54:48.437 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2438 2024-09-12T13:54:48.444 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:54:48.449 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:54:48.449 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:54:48.450 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:54:48.450 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:54:48.450 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:54:48.450 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:54:48.456 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:54:48.456 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:54:49.362 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2439 2024-09-12T13:54:49.405 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:54:49.405 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:54:49.405 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:54:49.405 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:54:49.405 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:54:49.405 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:54:49.413 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:54:49.413 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:54:49.413 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:54:49.414 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:54:49.414 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:54:49.414 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:54:49.431 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:54:49.448 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:54:49.448 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:54:49.448 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:54:49.448 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:54:49.448 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:54:49.448 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:54:49.454 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2440 2024-09-12T13:54:49.484 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:54:49.513 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2441 2024-09-12T13:54:49.544 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:54:50.436 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2444 2024-09-12T13:54:50.438 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2442 2024-09-12T13:54:50.448 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:54:50.452 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:54:51.411 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:54:51.412 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:54:51.412 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:54:51.412 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:54:51.412 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:54:51.412 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:54:51.448 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:54:51.448 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:54:51.448 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:54:51.449 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:54:51.449 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:54:51.449 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:54:51.458 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2443 2024-09-12T13:54:51.463 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:54:51.464 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:54:51.464 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:54:51.464 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:54:51.464 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:54:51.464 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:54:51.467 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:54:51.467 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:54:51.468 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:54:51.468 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:54:51.468 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:54:51.468 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:54:51.509 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:54:52.435 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:54:52.435 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:54:52.435 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:54:52.436 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:54:52.436 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:54:52.436 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:54:52.485 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2446 2024-09-12T13:54:52.487 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:54:52.488 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:54:52.488 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:54:52.489 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:54:52.489 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:54:52.489 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:54:52.539 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:54:52.549 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:54:52.549 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:54:52.549 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:54:52.549 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:54:52.549 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:54:52.549 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:54:53.393 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2447 2024-09-12T13:54:53.451 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:54:53.451 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:54:53.451 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:54:53.451 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:54:53.452 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:54:53.452 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:54:53.455 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:54:53.455 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:54:53.455 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:54:53.455 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:54:53.455 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:54:53.455 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:54:53.487 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:54:53.492 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2448 2024-09-12T13:54:53.520 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:54:54.518 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:54:54.518 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:54:54.519 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:54:54.519 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:54:54.519 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:54:54.519 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:54:54.548 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2450 2024-09-12T13:54:54.550 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2451 2024-09-12T13:54:54.575 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:54:54.641 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:54:55.542 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:54:55.542 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:54:55.543 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:54:55.543 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:54:55.543 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:54:55.543 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:54:55.546 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:54:55.547+0000 7f67dd000640 -1 rbd::mirror::image_replayer::snapshot::Replayer: 0x56055ba32c00 handle_unlink_peer: failed to unlink local peer from remote image: (22) Invalid argument 2024-09-12T13:54:56.382 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2453 2024-09-12T13:54:56.406 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2452 2024-09-12T13:54:56.423 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2445 2024-09-12T13:54:56.461 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:54:56.487 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:54:56.491 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:54:56.491 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:54:56.491 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:54:56.491 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:54:56.492 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:54:56.492 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:54:56.496 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2454 2024-09-12T13:54:56.517 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:54:56.534 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:54:56.534 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:54:56.534 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:54:56.534 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:54:56.534 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:54:56.534 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:54:56.537 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:54:57.473 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2449 2024-09-12T13:54:57.483 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2455 2024-09-12T13:54:57.488 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:54:57.511 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:54:57.578 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:54:57.578 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:54:57.578 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:54:57.578 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:54:57.578 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:54:57.578 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:54:57.644 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:54:57.644 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:54:57.644 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:54:57.644 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:54:57.644 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:54:57.645 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:54:58.364 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2457 2024-09-12T13:54:58.383 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2456 2024-09-12T13:54:58.390 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:54:58.404 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:54:59.469 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:54:59.469 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:54:59.469 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:54:59.470 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:54:59.470 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:54:59.470 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:54:59.481 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2458 2024-09-12T13:54:59.490 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:54:59.490 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:54:59.490 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:54:59.490 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:54:59.490 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:54:59.491 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:54:59.497 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2459 2024-09-12T13:54:59.512 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:54:59.518 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:54:59.521 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:54:59.521 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:54:59.521 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:54:59.521 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:54:59.521 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:54:59.521 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:54:59.546 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:54:59.546 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:54:59.546 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:54:59.546 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:54:59.546 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:54:59.546 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:55:00.450 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2462 2024-09-12T13:55:00.451 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2460 2024-09-12T13:55:00.462 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:55:00.469 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2461 2024-09-12T13:55:00.478 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:55:00.480 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:55:00.491 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:55:00.491 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:55:00.491 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:55:00.491 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:55:00.491 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:55:00.492 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:55:00.515 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:55:00.515 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:55:00.515 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:55:00.515 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:55:00.515 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:55:00.515 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:55:01.394 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:55:01.395 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:55:01.395 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:55:01.395 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:55:01.395 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:55:01.395 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:55:01.407 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:55:01.408 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:55:01.408 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:55:01.408 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:55:01.408 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:55:01.408 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:55:02.427 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2463 2024-09-12T13:55:02.492 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:55:02.519 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:55:02.519 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:55:02.519 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:55:02.519 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:55:02.519 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:55:02.520 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:55:02.522 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:55:02.522 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:55:02.522 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:55:02.522 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:55:02.522 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:55:02.523 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:55:03.337 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2464 2024-09-12T13:55:03.346 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2465 2024-09-12T13:55:03.396 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:55:03.407 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2467 2024-09-12T13:55:03.430 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:55:03.452 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:55:03.457 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2466 2024-09-12T13:55:03.464 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:55:03.464 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:55:03.464 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:55:03.464 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:55:03.465 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:55:03.465 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:55:03.468 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:55:03.482 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:55:03.482 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:55:03.483 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:55:03.483 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:55:03.483 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:55:03.483 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:55:03.483 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:55:03.483 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:55:03.483 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:55:03.483 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:55:03.484 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:55:03.484 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:55:04.530 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2472 2024-09-12T13:55:04.538 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:55:05.427 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2468 2024-09-12T13:55:05.435 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2469 2024-09-12T13:55:05.466 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:55:05.494 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:55:05.494 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:55:05.494 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:55:05.495 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:55:05.495 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:55:05.495 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:55:05.496 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:55:06.400 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:55:06.400 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:55:06.400 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:55:06.400 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:55:06.400 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:55:06.400 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:55:06.429 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:55:06.429 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:55:06.429 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:55:06.429 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:55:06.430 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:55:06.430 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:55:06.434 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2471 2024-09-12T13:55:06.438 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2470 2024-09-12T13:55:06.455 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:55:06.455 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:55:06.455 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:55:06.455 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:55:06.455 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:55:06.455 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:55:06.468 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:55:06.471 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:55:06.471 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:55:06.471 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:55:06.471 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:55:06.471 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:55:06.472 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:55:06.479 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:55:06.494 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2473 2024-09-12T13:55:06.512 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:55:07.462 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2475 2024-09-12T13:55:07.468 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2474 2024-09-12T13:55:07.482 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:55:07.492 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:55:07.541 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:55:07.541 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:55:07.541 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:55:07.542 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:55:07.542 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:55:07.542 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:55:08.379 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2476 2024-09-12T13:55:08.388 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2477 2024-09-12T13:55:08.400 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:55:08.410 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:55:08.410 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2478 2024-09-12T13:55:08.436 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:55:08.469 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:55:08.469 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:55:08.469 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:55:08.469 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:55:08.469 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:55:08.469 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:55:08.499 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:55:08.499 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:55:08.499 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:55:08.500 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:55:08.500 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:55:08.500 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:55:09.470 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2479 2024-09-12T13:55:09.471 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:55:09.471 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:55:09.471 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:55:09.471 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:55:09.472 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:55:09.472 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:55:09.475 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2480 2024-09-12T13:55:09.479 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:55:09.482 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:55:09.482 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:55:09.482 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:55:09.482 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:55:09.483 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:55:09.483 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:55:09.483 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:55:09.515 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:55:09.515 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:55:09.515 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:55:09.515 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:55:09.515 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:55:09.515 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:55:10.468 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2482 2024-09-12T13:55:10.479 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2481 2024-09-12T13:55:10.485 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:55:10.485 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:55:10.485 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:55:10.485 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:55:10.485 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:55:10.485 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:55:10.487 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:55:10.490 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:55:10.495 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:55:10.495 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:55:10.495 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:55:10.495 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:55:10.495 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:55:10.495 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:55:11.403 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:55:11.403 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:55:11.403 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:55:11.403 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:55:11.403 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:55:11.404 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:55:11.413 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:55:11.413 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:55:11.413 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:55:11.413 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:55:11.413 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:55:11.413 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:55:11.439 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:55:11.440 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:55:11.440 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:55:11.440 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:55:11.440 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:55:11.440 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:55:12.452 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2483 2024-09-12T13:55:12.482 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:55:12.482 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:55:12.482 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:55:12.483 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:55:12.483 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:55:12.483 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:55:12.486 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:55:12.486 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:55:12.486 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:55:12.486 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:55:12.486 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:55:12.486 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:55:12.508 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:55:13.344 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2485 2024-09-12T13:55:13.361 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2488 2024-09-12T13:55:13.369 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2484 2024-09-12T13:55:13.407 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:55:13.412 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:55:13.429 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:55:13.475 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:55:13.475+0000 7f67dc600640 -1 rbd::mirror::image_replayer::snapshot::Replayer: 0x56055c10f800 handle_unlink_peer: failed to unlink local peer from remote image: (22) Invalid argument 2024-09-12T13:55:13.490 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:55:13.490 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:55:13.490 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:55:13.490 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:55:13.490 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:55:13.491 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:55:13.493 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:55:13.493 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:55:13.493 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:55:13.493 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:55:13.493 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:55:13.493 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:55:14.467 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2487 2024-09-12T13:55:14.474 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2486 2024-09-12T13:55:14.530 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:55:14.532 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:55:15.455 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2489 2024-09-12T13:55:15.463 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2490 2024-09-12T13:55:15.495 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:55:15.511 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:55:15.511 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:55:15.511 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:55:15.511 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:55:15.511 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:55:15.511 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:55:15.517 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:55:16.411 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:55:16.411 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:55:16.411 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:55:16.411 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:55:16.411 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:55:16.411 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:55:16.415 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:55:16.415 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:55:16.415 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:55:16.415 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:55:16.415 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:55:16.415 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:55:16.432 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:55:16.432 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:55:16.432 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:55:16.432 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:55:16.432 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:55:16.433 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:55:16.472 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2492 2024-09-12T13:55:16.478 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2491 2024-09-12T13:55:16.507 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:55:16.514 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:55:16.525 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2493 2024-09-12T13:55:16.540 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:55:17.533 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:55:17.533 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:55:17.533 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:55:17.533 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:55:17.533 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:55:17.533 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:55:17.535 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:55:17.535 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:55:17.535 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:55:17.535 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:55:17.535 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:55:17.535 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:55:17.536 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2496 2024-09-12T13:55:17.542 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2494 2024-09-12T13:55:17.549 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:55:17.558 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:55:18.338 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2495 2024-09-12T13:55:18.374 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:55:18.453 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2497 2024-09-12T13:55:18.455 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2498 2024-09-12T13:55:18.498 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:55:18.498 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:55:18.498 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:55:18.498 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:55:18.499 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:55:18.499 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:55:18.520 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:55:18.520 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:55:18.520 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:55:18.520 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:55:18.520 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:55:18.520 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:55:18.658 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:55:18.659 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:55:19.491 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2500 2024-09-12T13:55:19.494 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2499 2024-09-12T13:55:19.506 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:55:19.510 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:55:19.510 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:55:19.510 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:55:19.510 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:55:19.510 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:55:19.510 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:55:19.510 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:55:19.516 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:55:19.516 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:55:19.516 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:55:19.516 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:55:19.517 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:55:19.517 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:55:19.543 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:55:19.543 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:55:19.543 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:55:19.543 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:55:19.543 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:55:19.543 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:55:20.512 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2502 2024-09-12T13:55:20.516 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2501 2024-09-12T13:55:20.550 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:55:20.551 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:55:20.551 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:55:20.551 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:55:20.551 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:55:20.551 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:55:20.561 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:55:20.561 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:55:20.561 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:55:20.562 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:55:20.562 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:55:20.562 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:55:20.625 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:55:20.625 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:55:21.438 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:55:21.439 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:55:21.439 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:55:21.439 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:55:21.439 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:55:21.439 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:55:21.533 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:55:21.533+0000 7f67dc600640 -1 rbd::mirror::image_replayer::snapshot::Replayer: 0x56055c2f8000 handle_unlink_peer: failed to unlink local peer from remote image: (22) Invalid argument 2024-09-12T13:55:21.661 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:55:21.661 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:55:21.661 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:55:21.661 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:55:21.661 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:55:21.661 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:55:21.662 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:55:21.662 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:55:21.662 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:55:21.662 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:55:21.662 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:55:21.662 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:55:22.487 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2503 2024-09-12T13:55:22.508 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:55:22.508 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:55:22.508 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:55:22.509 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:55:22.509 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:55:22.509 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:55:22.512 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:55:22.512 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:55:22.512 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:55:22.512 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:55:22.512 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:55:22.512 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:55:22.538 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:55:22.538+0000 7f67dc600640 -1 rbd::mirror::image_replayer::snapshot::Replayer: 0x56055c381400 handle_unlink_peer: failed to unlink local peer from remote image: (22) Invalid argument 2024-09-12T13:55:22.552 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:55:23.326 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2505 2024-09-12T13:55:23.357 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2504 2024-09-12T13:55:23.401 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:55:23.401 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:55:23.407 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2508 2024-09-12T13:55:23.428 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2507 2024-09-12T13:55:23.472 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:55:23.493 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:55:23.627 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:55:23.627 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:55:23.628 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:55:23.628 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:55:23.628 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:55:23.628 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:55:23.628 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:55:23.628 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:55:23.628 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:55:23.628 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:55:23.628 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:55:23.628 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:55:24.403 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2510 2024-09-12T13:55:24.410 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2509 2024-09-12T13:55:24.431 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:55:24.434 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:55:25.348 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2506 2024-09-12T13:55:25.382 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2512 2024-09-12T13:55:25.385 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:55:25.420 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:55:25.556 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:55:25.556 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:55:25.556 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:55:25.556 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:55:25.557 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:55:25.557 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:55:26.404 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:55:26.405 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:55:26.405 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:55:26.405 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:55:26.405 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:55:26.405 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:55:26.406 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:55:26.406 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:55:26.406 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:55:26.407 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:55:26.407 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:55:26.408 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:55:26.479 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:55:26.479 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:55:26.479 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:55:26.479 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:55:26.479 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:55:26.480 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:55:26.498 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:55:26.498 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:55:26.498 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:55:26.498 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:55:26.498 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:55:26.498 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:55:26.499 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2511 2024-09-12T13:55:26.522 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:55:26.542 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2513 2024-09-12T13:55:26.558 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:55:27.435 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:55:27.435 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:55:27.436 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:55:27.436 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:55:27.436 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:55:27.436 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:55:27.438 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:55:27.438 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:55:27.438 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:55:27.438 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:55:27.438 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:55:27.438 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:55:27.529 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2515 2024-09-12T13:55:27.532 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2514 2024-09-12T13:55:27.556 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:55:27.556 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:55:28.374 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2517 2024-09-12T13:55:28.384 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2516 2024-09-12T13:55:28.388 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:55:28.388 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:55:28.388 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:55:28.388 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:55:28.388 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:55:28.388 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:55:28.410 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:55:28.417 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:55:28.424 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:55:28.425 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:55:28.425 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:55:28.425 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:55:28.425 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:55:28.425 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:55:29.387 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2518 2024-09-12T13:55:29.392 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2519 2024-09-12T13:55:29.416 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:55:29.419 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:55:29.525 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:55:29.525 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:55:29.525 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:55:29.525 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:55:29.525 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:55:29.525 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:55:29.561 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:55:29.561 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:55:29.561 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:55:29.561 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:55:29.561 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:55:29.561 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:55:30.405 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:55:30.405+0000 7f67dd000640 -1 rbd::mirror::image_replayer::snapshot::Replayer: 0x560556aac400 handle_unlink_peer: failed to unlink local peer from remote image: (22) Invalid argument 2024-09-12T13:55:30.414 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2523 2024-09-12T13:55:30.428 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2522 2024-09-12T13:55:30.430 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:55:30.436 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:55:30.559 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:55:30.559 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:55:30.559 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:55:30.559 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:55:30.559 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:55:30.559 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:55:30.559 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:55:30.559 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:55:30.559 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:55:30.560 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:55:30.560 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:55:30.560 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:55:31.353 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2521 2024-09-12T13:55:31.396 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:55:31.413 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2524 2024-09-12T13:55:31.415 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:55:31.415 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:55:31.415 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:55:31.415 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:55:31.415 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:55:31.415 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:55:31.421 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:55:31.421 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:55:31.421 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:55:31.421 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:55:31.421 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:55:31.421 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:55:31.436 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:55:32.345 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2520 2024-09-12T13:55:32.384 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:55:32.393 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2525 2024-09-12T13:55:32.421 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:55:32.422 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:55:32.422 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:55:32.422 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:55:32.422 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:55:32.422 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:55:32.423 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:55:32.423 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:55:32.423 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:55:32.424 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:55:32.424 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:55:32.424 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:55:32.424 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:55:33.369 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2526 2024-09-12T13:55:33.379 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2527 2024-09-12T13:55:33.398 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:55:33.400 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:55:33.434 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:55:33.434 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:55:33.434 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:55:33.434 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:55:33.434 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:55:33.434 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:55:33.439 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:55:33.439 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:55:33.439 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:55:33.439 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:55:33.439 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:55:33.439 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:55:34.397 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2529 2024-09-12T13:55:34.401 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:55:34.401 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:55:34.401 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:55:34.401 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:55:34.401 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:55:34.401 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:55:34.405 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2528 2024-09-12T13:55:34.433 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:55:34.440 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:55:34.440 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:55:34.440 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:55:34.440 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:55:34.440 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:55:34.440 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:55:34.440 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2530 2024-09-12T13:55:34.441 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:55:34.468 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:55:35.385 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:55:35.385+0000 7f67dd000640 -1 rbd::mirror::image_replayer::snapshot::Replayer: 0x56055bfdf800 handle_unlink_peer: failed to unlink local peer from remote image: (22) Invalid argument 2024-09-12T13:55:35.388 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:55:35.388 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:55:35.388 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:55:35.388 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:55:35.388 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:55:35.388 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:55:35.406 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2533 2024-09-12T13:55:35.416 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2532 2024-09-12T13:55:35.422 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:55:35.426 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:55:35.426 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:55:35.426 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:55:35.426 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:55:35.426 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:55:35.426 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:55:35.432 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:55:36.374 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2531 2024-09-12T13:55:36.402 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:55:36.402 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:55:36.402 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:55:36.402 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:55:36.402 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:55:36.402 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:55:36.402 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:55:36.405 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:55:36.406 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:55:36.406 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:55:36.406 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:55:36.406 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:55:36.406 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:55:36.429 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2534 2024-09-12T13:55:36.438 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:55:37.440 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:55:37.440 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:55:37.440 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:55:37.440 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:55:37.440 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:55:37.440 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:55:37.446 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:55:37.446 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:55:37.446 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:55:37.447 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:55:37.447 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:55:37.447 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:55:37.473 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:55:37.475 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:55:37.475 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:55:37.475 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:55:37.475 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:55:37.475 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:55:38.364 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2535 2024-09-12T13:55:38.403 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:55:38.424 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:55:38.425 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:55:38.425 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:55:38.425 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:55:38.425 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:55:38.425 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:55:38.435 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:55:38.435 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:55:38.435 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:55:38.435 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:55:38.435 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:55:38.435 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:55:38.442 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2540 2024-09-12T13:55:38.451 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:55:39.391 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2537 2024-09-12T13:55:39.404 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:55:39.404 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:55:39.405 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:55:39.405 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:55:39.405 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:55:39.405 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:55:39.406 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2536 2024-09-12T13:55:39.441 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:55:39.442 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:55:39.442 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:55:39.442 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:55:39.442 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:55:39.442 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:55:39.449 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:55:39.506 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:55:39.575 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2542 2024-09-12T13:55:39.611 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2541 2024-09-12T13:55:39.623 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:55:39.623 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:55:40.382 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2539 2024-09-12T13:55:40.390 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2538 2024-09-12T13:55:40.420 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:55:40.431 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:55:40.492 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2543 2024-09-12T13:55:40.513 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:55:41.406 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:55:41.406 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:55:41.406 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:55:41.406 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:55:41.406 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:55:41.406 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:55:41.453 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:55:41.453 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:55:41.454 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:55:41.454 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:55:41.454 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:55:41.454 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:55:42.385 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2544 2024-09-12T13:55:42.415 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:55:42.433 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2545 2024-09-12T13:55:42.449 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:55:42.451 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:55:42.452 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:55:42.452 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:55:42.452 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:55:42.452 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:55:42.452 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:55:42.510 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:55:42.510 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:55:42.510 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:55:42.510 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:55:42.510 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:55:42.510 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:55:42.625 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:55:42.626 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:55:42.626 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:55:42.626 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:55:42.626 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:55:42.626 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:55:42.626 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:55:42.626 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:55:42.626 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:55:42.626 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:55:42.626 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:55:42.626 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:55:43.346 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2546 2024-09-12T13:55:43.379 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:55:43.422 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:55:43.423 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:55:43.423 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:55:43.423 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:55:43.423 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:55:43.423 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:55:43.435 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:55:43.435 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:55:43.436 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:55:43.436 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:55:43.436 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:55:43.436 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:55:43.498 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2550 2024-09-12T13:55:43.508 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2548 2024-09-12T13:55:43.513 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2547 2024-09-12T13:55:43.516 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:55:43.516 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:55:43.516 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:55:43.516 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:55:43.516 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:55:43.516 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:55:43.531 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:55:43.532 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:55:43.555 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:55:44.388 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2549 2024-09-12T13:55:44.431 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:55:44.454 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2552 2024-09-12T13:55:44.457 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2551 2024-09-12T13:55:44.460 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2553 2024-09-12T13:55:44.470 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:55:44.473 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:55:44.477 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:55:45.419 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:55:45.419 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:55:45.419 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:55:45.419 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:55:45.419 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:55:45.420 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:55:45.452 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:55:45.453 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:55:45.453 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:55:45.453 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:55:45.453 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:55:45.453 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:55:46.382 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:55:46.382 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:55:46.382 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:55:46.383 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:55:46.383 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:55:46.383 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:55:46.429 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2554 2024-09-12T13:55:46.445 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:55:46.535 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:55:46.535 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:55:46.535 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:55:46.535 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:55:46.535 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:55:46.535 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:55:46.535 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:55:46.535 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:55:46.536 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:55:46.536 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:55:46.536 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:55:46.536 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:55:46.557 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:55:46.558 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:55:46.558 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:55:46.558 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:55:46.558 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:55:46.558 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:55:47.434 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:55:47.435 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:55:47.435 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:55:47.435 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:55:47.435 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:55:47.435 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:55:47.473 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:55:47.473 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:55:47.473 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:55:47.473 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:55:47.473 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:55:47.473 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:55:47.476 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:55:47.476 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:55:47.476 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:55:47.476 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:55:47.476 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:55:47.477 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:55:47.478 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:55:47.479 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:55:47.479 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:55:47.479 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:55:47.479 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:55:47.479 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:55:47.479 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2557 2024-09-12T13:55:47.491 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:55:48.344 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2556 2024-09-12T13:55:48.354 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2555 2024-09-12T13:55:48.433 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:55:48.436 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:55:48.450 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2558 2024-09-12T13:55:48.451 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2559 2024-09-12T13:55:48.473 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:55:48.493 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:55:48.566 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2562 2024-09-12T13:55:48.587 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:55:49.419 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2560 2024-09-12T13:55:49.448 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:55:49.448 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:55:49.448 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:55:49.449 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:55:49.449 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:55:49.449 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:55:49.452 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:55:49.474 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2563 2024-09-12T13:55:49.513 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:55:50.410 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2561 2024-09-12T13:55:50.440 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:55:50.494 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:55:50.494 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:55:50.494 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:55:50.494 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:55:50.494 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:55:50.494 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:55:51.438 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:55:51.439 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:55:51.439 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:55:51.439 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:55:51.439 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:55:51.439 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:55:51.441 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:55:51.441 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:55:51.441 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:55:51.441 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:55:51.441 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:55:51.441 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:55:51.457 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:55:51.458+0000 7f67dc600640 -1 rbd::mirror::image_replayer::snapshot::Replayer: 0x560555c66000 handle_unlink_peer: failed to unlink local peer from remote image: (22) Invalid argument 2024-09-12T13:55:51.477 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:55:51.478 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:55:51.478 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:55:51.478 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:55:51.478 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:55:51.478 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:55:51.497 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:55:51.497 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:55:51.497 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:55:51.497 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:55:51.497 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:55:51.497 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:55:51.590 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:55:51.590 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:55:51.590 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:55:51.590 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:55:51.590 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:55:51.590 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:55:52.440 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2564 2024-09-12T13:55:52.456 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:55:52.456 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:55:52.456 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:55:52.456 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:55:52.456 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:55:52.456 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:55:52.497 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:55:52.517 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:55:52.517 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:55:52.517 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:55:52.517 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:55:52.517 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:55:52.518 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:55:52.545 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2567 2024-09-12T13:55:52.568 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:55:53.393 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2568 2024-09-12T13:55:53.425 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2565 2024-09-12T13:55:53.434 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2569 2024-09-12T13:55:53.442 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:55:53.442 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:55:53.442 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:55:53.443 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:55:53.443 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:55:53.443 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:55:53.466 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:55:53.504 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:55:53.526 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:55:54.544 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2573 2024-09-12T13:55:54.565 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:55:55.435 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2572 2024-09-12T13:55:55.443 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2570 2024-09-12T13:55:55.483 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:55:55.495 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:55:55.499 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:55:55.499 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:55:55.500 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:55:55.500 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:55:55.500 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:55:55.500 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:55:55.522 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:55:55.521+0000 7f67dc600640 -1 rbd::mirror::image_replayer::snapshot::Replayer: 0x56055bb4e000 handle_unlink_peer: failed to unlink local peer from remote image: (22) Invalid argument 2024-09-12T13:55:55.570 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:55:55.571 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:55:55.571 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:55:55.571 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:55:55.571 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:55:55.571 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:55:56.436 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2566 2024-09-12T13:55:56.469 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:55:56.470 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:55:56.470 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:55:56.470 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:55:56.470 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:55:56.470 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:55:56.480 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:55:56.509 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:55:56.509 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:55:56.509 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:55:56.509 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:55:56.509 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:55:56.509 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:55:56.522 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2574 2024-09-12T13:55:56.529 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:55:56.529 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:55:56.529 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:55:56.529 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:55:56.529 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:55:56.529 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:55:56.538 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:55:57.567 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:55:57.567 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:55:57.567 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:55:57.568 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:55:57.568 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:55:57.568 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:55:58.351 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2571 2024-09-12T13:55:58.364 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2575 2024-09-12T13:55:58.397 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2576 2024-09-12T13:55:58.410 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:55:58.422 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:55:58.435 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2577 2024-09-12T13:55:58.442 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:55:58.476 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:55:58.486 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:55:58.486 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:55:58.486 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:55:58.486 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:55:58.486 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:55:58.486 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:55:58.498 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:55:58.498 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:55:58.498 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:55:58.498 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:55:58.498 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:55:58.499 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:55:59.482 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:55:59.483 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:55:59.483 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:55:59.483 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:55:59.483 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:55:59.483 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:55:59.526 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2580 2024-09-12T13:55:59.542 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:55:59.542 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:55:59.542 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:55:59.542 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:55:59.542 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:55:59.542 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:55:59.542 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:56:00.471 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2579 2024-09-12T13:56:00.519 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:56:00.547 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2582 2024-09-12T13:56:00.555 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2583 2024-09-12T13:56:00.566 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:56:00.572 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:56:01.413 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:56:01.413 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:56:01.413 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:56:01.413 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:56:01.413 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:56:01.413 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:56:01.426 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:56:01.426 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:56:01.426 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:56:01.426 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:56:01.427 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:56:01.427 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:56:01.444 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:56:01.444 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:56:01.445 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:56:01.445 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:56:01.445 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:56:01.445 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:56:01.456 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2578 2024-09-12T13:56:01.465 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2581 2024-09-12T13:56:01.478 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:56:01.478 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:56:01.478 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:56:01.479 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:56:01.479 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:56:01.479 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:56:01.494 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:56:01.512 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:56:02.507 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2584 2024-09-12T13:56:02.508 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2585 2024-09-12T13:56:02.524 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:56:02.525 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:56:02.544 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:56:02.544 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:56:02.544 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:56:02.544 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:56:02.544 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:56:02.545 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:56:03.388 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2586 2024-09-12T13:56:03.402 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2587 2024-09-12T13:56:03.410 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:56:03.417 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:56:03.521 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:56:03.521 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:56:03.521 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:56:03.521 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:56:03.521 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:56:03.522 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:56:03.569 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:56:03.569 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:56:03.569 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:56:03.570 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:56:03.570 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:56:03.570 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:56:03.574 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:56:03.574 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:56:03.574 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:56:03.574 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:56:03.575 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:56:03.575 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:56:04.496 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:56:04.496 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:56:04.496 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:56:04.496 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:56:04.496 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:56:04.497 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:56:04.515 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:56:04.515 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:56:04.515 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:56:04.515 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:56:04.515 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:56:04.515 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:56:04.516 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2589 2024-09-12T13:56:04.522 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2591 2024-09-12T13:56:04.527 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2590 2024-09-12T13:56:04.537 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:56:04.537 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:56:04.541 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:56:05.476 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2588 2024-09-12T13:56:05.500 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:56:05.517 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2593 2024-09-12T13:56:05.521 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2592 2024-09-12T13:56:05.527 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:56:05.527 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:56:05.527 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:56:05.527 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:56:05.528 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:56:05.528 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:56:05.528 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:56:05.528 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:56:05.529 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:56:05.529 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:56:05.529 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:56:05.529 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:56:05.533 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:56:05.539 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:56:06.414 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:56:06.414 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:56:06.414 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:56:06.414 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:56:06.415 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:56:06.415 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:56:06.424 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:56:06.424 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:56:06.424 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:56:06.424 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:56:06.424 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:56:06.424 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:56:07.511 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T13:56:07.511+0000 7f67dc600640 -1 rbd::mirror::image_replayer::snapshot::Replayer: 0x560556aad800 handle_unlink_peer: failed to unlink local peer from remote image: (22) Invalid argument 2024-09-12T13:56:07.540 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:56:07.540 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:56:07.540 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:56:07.540 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:56:07.540 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:56:07.540 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:56:07.541 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:56:07.541 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:56:07.541 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:56:07.541 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:56:07.541 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:56:07.541 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:56:07.544 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:56:07.544 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:56:07.544 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:56:07.544 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:56:07.544 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:56:07.544 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:56:08.353 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2595 2024-09-12T13:56:08.376 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2594 2024-09-12T13:56:08.410 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:56:08.418 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2596 2024-09-12T13:56:08.426 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2597 2024-09-12T13:56:08.435 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:56:08.477 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:56:08.479 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:56:08.485 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2600 2024-09-12T13:56:08.490 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2599 2024-09-12T13:56:08.503 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:56:08.503 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:56:08.503 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:56:08.503 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:56:08.504 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:56:08.504 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:56:08.504 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:56:08.509 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:56:08.537 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:56:08.537 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:56:08.537 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:56:08.538 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:56:08.538 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:56:08.538 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:56:08.542 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:56:08.542 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:56:08.543 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:56:08.543 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:56:08.543 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:56:08.543 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:56:09.547 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2601 2024-09-12T13:56:09.569 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:56:10.488 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2598 2024-09-12T13:56:10.540 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:56:11.414 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:56:11.414 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:56:11.415 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:56:11.415 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:56:11.415 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:56:11.415 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:56:11.439 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:56:11.440 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:56:11.440 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:56:11.440 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:56:11.440 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:56:11.440 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:56:11.481 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:56:11.481 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:56:11.481 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:56:11.481 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:56:11.481 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:56:11.481 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:56:11.482 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:56:11.483 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:56:11.483 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:56:11.483 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:56:11.483 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:56:11.483 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:56:11.498 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2602 2024-09-12T13:56:11.505 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2603 2024-09-12T13:56:11.507 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:56:11.507 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:56:11.507 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:56:11.507 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:56:11.507 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:56:11.507 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:56:11.512 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:56:11.512 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T13:56:11.512 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:56:11.512 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:56:11.512 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:56:11.513 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T13:56:11.547 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:56:11.549 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:56:12.548 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2604 2024-09-12T13:56:12.566 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2605 2024-09-12T13:56:12.570 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:56:12.572 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:56:12.572 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:56:12.572 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:56:12.572 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:56:12.572 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:56:12.572 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:56:12.584 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:56:13.352 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2609 2024-09-12T13:56:13.356 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2606 2024-09-12T13:56:13.395 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2607 2024-09-12T13:56:13.399 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:56:13.406 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:56:13.415 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2608 2024-09-12T13:56:13.434 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:56:13.471 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:56:13.543 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:56:13.543 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T13:56:13.543 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:56:13.543 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:56:13.543 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:56:13.544 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T13:56:14.370 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2610 2024-09-12T13:56:14.385 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2611 2024-09-12T13:56:14.386 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:56:14.405 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:56:14.552 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:56:14.552 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:56:14.552 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:56:14.552 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:56:14.552 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:56:14.552 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:56:14.552 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:56:14.552 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T13:56:14.552 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:56:14.552 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:56:14.552 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:56:14.553 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T13:56:15.536 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2612 2024-09-12T13:56:15.553 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:56:15.569 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2613 2024-09-12T13:56:15.573 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:56:15.573 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:56:15.573 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:56:15.574 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:56:15.574 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:56:15.574 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:56:15.579 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:56:15.587 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:56:15.587 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T13:56:15.587 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:56:15.587 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:56:15.587 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:56:15.587 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T13:56:16.410 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:56:16.411 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T13:56:16.411 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:56:16.411 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:56:16.411 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:56:16.411 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T13:56:16.439 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:56:16.439 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:56:16.439 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:56:16.439 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:56:16.439 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:56:16.439 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:56:16.476 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:56:16.477 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T13:56:16.477 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:56:16.477 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:56:16.477 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T13:56:16.477 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T13:56:16.541 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2615 2024-09-12T13:56:16.558 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:56:17.389 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:56:17.389 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T13:56:17.389 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:56:17.389 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:56:17.390 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:56:17.390 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T13:56:17.412 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${SNAP_PIDS[@]} 2024-09-12T13:56:17.412 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 118391 2024-09-12T13:56:18.341 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2614 2024-09-12T13:56:18.366 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2616 2024-09-12T13:56:18.383 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:56:18.395 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2617 2024-09-12T13:56:18.415 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2618 2024-09-12T13:56:18.424 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:56:18.441 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:56:18.445 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:56:18.556 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${SNAP_PIDS[@]} 2024-09-12T13:56:18.556 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 118523 2024-09-12T13:56:18.556 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${SNAP_PIDS[@]} 2024-09-12T13:56:18.556 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 118655 2024-09-12T13:56:18.582 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:56:18.582 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T13:56:18.582 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:56:18.582 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:56:18.582 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:56:18.582 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T13:56:19.415 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2619 2024-09-12T13:56:19.441 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:56:19.561 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${SNAP_PIDS[@]} 2024-09-12T13:56:19.561 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 118807 2024-09-12T13:56:21.361 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2620 2024-09-12T13:56:21.381 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:56:21.386 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:56:21.386 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T13:56:21.386 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:56:21.386 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:56:21.386 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:56:21.386 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T13:56:21.446 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T13:56:21.446 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T13:56:21.446 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:56:21.446 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:56:21.446 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:56:21.446 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T13:56:21.453 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${SNAP_PIDS[@]} 2024-09-12T13:56:21.453 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 118960 2024-09-12T13:56:22.381 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2621 2024-09-12T13:56:22.398 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:56:22.445 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${SNAP_PIDS[@]} 2024-09-12T13:56:22.445 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 119113 2024-09-12T13:56:23.387 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2622 2024-09-12T13:56:23.413 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T13:56:24.385 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${SNAP_PIDS[@]} 2024-09-12T13:56:24.385 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 119266 2024-09-12T13:56:24.385 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${SNAP_PIDS[@]} 2024-09-12T13:56:24.385 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 119439 2024-09-12T13:56:25.400 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${SNAP_PIDS[@]} 2024-09-12T13:56:25.401 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 119613 2024-09-12T13:56:26.416 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ (( RET != 0 )) 2024-09-12T13:56:26.416 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${WORKLOAD_PIDS[@]} 2024-09-12T13:56:26.417 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 118260 2024-09-12T13:58:53.231 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ ret=124 2024-09-12T13:58:53.232 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ (( ret != 124 )) 2024-09-12T13:58:53.232 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${WORKLOAD_PIDS[@]} 2024-09-12T13:58:53.232 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 118392 2024-09-12T13:58:54.009 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ ret=124 2024-09-12T13:58:54.010 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ (( ret != 124 )) 2024-09-12T13:58:54.010 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${WORKLOAD_PIDS[@]} 2024-09-12T13:58:54.010 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 118524 2024-09-12T13:58:55.088 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ ret=124 2024-09-12T13:58:55.088 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ (( ret != 124 )) 2024-09-12T13:58:55.089 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${WORKLOAD_PIDS[@]} 2024-09-12T13:58:55.089 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 118656 2024-09-12T13:58:56.083 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ ret=124 2024-09-12T13:58:56.083 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ (( ret != 124 )) 2024-09-12T13:58:56.084 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${WORKLOAD_PIDS[@]} 2024-09-12T13:58:56.084 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 118808 2024-09-12T13:58:57.294 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ ret=124 2024-09-12T13:58:57.294 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ (( ret != 124 )) 2024-09-12T13:58:57.295 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${WORKLOAD_PIDS[@]} 2024-09-12T13:58:57.295 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 118961 2024-09-12T13:58:58.324 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ ret=124 2024-09-12T13:58:58.324 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ (( ret != 124 )) 2024-09-12T13:58:58.324 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${WORKLOAD_PIDS[@]} 2024-09-12T13:58:58.324 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 119114 2024-09-12T13:58:59.247 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ ret=124 2024-09-12T13:58:59.247 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ (( ret != 124 )) 2024-09-12T13:58:59.247 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${WORKLOAD_PIDS[@]} 2024-09-12T13:58:59.248 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 119267 2024-09-12T13:59:00.168 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ ret=124 2024-09-12T13:59:00.169 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ (( ret != 124 )) 2024-09-12T13:59:00.169 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${WORKLOAD_PIDS[@]} 2024-09-12T13:59:00.169 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 119440 2024-09-12T13:59:01.180 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ ret=124 2024-09-12T13:59:01.180 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ (( ret != 124 )) 2024-09-12T13:59:01.180 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${WORKLOAD_PIDS[@]} 2024-09-12T13:59:01.180 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 119614 2024-09-12T13:59:02.258 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ ret=124 2024-09-12T13:59:02.259 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ (( ret != 124 )) 2024-09-12T13:59:02.259 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ (( RET != 0 )) 2024-09-12T13:59:02.259 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:59:02.260 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ COMPARE_PIDS+=($!) 2024-09-12T13:59:02.260 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:59:02.260 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ compare_demoted_promoted_image 1 2024-09-12T13:59:02.261 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local dev=/dev/rbd0 2024-09-12T13:59:02.261 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local img=image-primary1 2024-09-12T13:59:02.261 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mntpt=test-primary1 2024-09-12T13:59:02.261 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local demote_md5 promote_md5 2024-09-12T13:59:02.261 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo umount test-primary1 2024-09-12T13:59:02.261 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ COMPARE_PIDS+=($!) 2024-09-12T13:59:02.261 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:59:02.261 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ compare_demoted_promoted_image 2 2024-09-12T13:59:02.263 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local dev=/dev/rbd1 2024-09-12T13:59:02.263 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local img=image-primary2 2024-09-12T13:59:02.263 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mntpt=test-primary2 2024-09-12T13:59:02.263 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local demote_md5 promote_md5 2024-09-12T13:59:02.263 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo umount test-primary2 2024-09-12T13:59:02.263 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ COMPARE_PIDS+=($!) 2024-09-12T13:59:02.263 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ compare_demoted_promoted_image 3 2024-09-12T13:59:02.263 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:59:02.263 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local dev=/dev/rbd2 2024-09-12T13:59:02.264 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local img=image-primary3 2024-09-12T13:59:02.264 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mntpt=test-primary3 2024-09-12T13:59:02.264 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local demote_md5 promote_md5 2024-09-12T13:59:02.264 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo umount test-primary3 2024-09-12T13:59:02.264 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ COMPARE_PIDS+=($!) 2024-09-12T13:59:02.264 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:59:02.264 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ compare_demoted_promoted_image 4 2024-09-12T13:59:02.264 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local dev=/dev/rbd3 2024-09-12T13:59:02.264 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local img=image-primary4 2024-09-12T13:59:02.264 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mntpt=test-primary4 2024-09-12T13:59:02.264 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local demote_md5 promote_md5 2024-09-12T13:59:02.264 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo umount test-primary4 2024-09-12T13:59:02.264 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ COMPARE_PIDS+=($!) 2024-09-12T13:59:02.265 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:59:02.265 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ compare_demoted_promoted_image 5 2024-09-12T13:59:02.265 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local dev=/dev/rbd4 2024-09-12T13:59:02.267 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local img=image-primary5 2024-09-12T13:59:02.267 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mntpt=test-primary5 2024-09-12T13:59:02.267 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local demote_md5 promote_md5 2024-09-12T13:59:02.267 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo umount test-primary5 2024-09-12T13:59:02.267 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ COMPARE_PIDS+=($!) 2024-09-12T13:59:02.268 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:59:02.268 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ compare_demoted_promoted_image 6 2024-09-12T13:59:02.268 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local dev=/dev/rbd5 2024-09-12T13:59:02.268 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local img=image-primary6 2024-09-12T13:59:02.268 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mntpt=test-primary6 2024-09-12T13:59:02.268 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local demote_md5 promote_md5 2024-09-12T13:59:02.268 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo umount test-primary6 2024-09-12T13:59:02.268 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ COMPARE_PIDS+=($!) 2024-09-12T13:59:02.268 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:59:02.268 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ compare_demoted_promoted_image 7 2024-09-12T13:59:02.268 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local dev=/dev/rbd6 2024-09-12T13:59:02.268 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local img=image-primary7 2024-09-12T13:59:02.268 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mntpt=test-primary7 2024-09-12T13:59:02.269 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local demote_md5 promote_md5 2024-09-12T13:59:02.269 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo umount test-primary7 2024-09-12T13:59:02.269 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ COMPARE_PIDS+=($!) 2024-09-12T13:59:02.269 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ compare_demoted_promoted_image 8 2024-09-12T13:59:02.269 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:59:02.269 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local dev=/dev/rbd7 2024-09-12T13:59:02.269 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local img=image-primary8 2024-09-12T13:59:02.269 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mntpt=test-primary8 2024-09-12T13:59:02.269 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local demote_md5 promote_md5 2024-09-12T13:59:02.269 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo umount test-primary8 2024-09-12T13:59:02.269 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ COMPARE_PIDS+=($!) 2024-09-12T13:59:02.269 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T13:59:02.269 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ compare_demoted_promoted_image 9 2024-09-12T13:59:02.270 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local dev=/dev/rbd8 2024-09-12T13:59:02.270 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local img=image-primary9 2024-09-12T13:59:02.270 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mntpt=test-primary9 2024-09-12T13:59:02.270 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local demote_md5 promote_md5 2024-09-12T13:59:02.270 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo umount test-primary9 2024-09-12T13:59:02.270 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ COMPARE_PIDS+=($!) 2024-09-12T13:59:02.270 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${COMPARE_PIDS[@]} 2024-09-12T13:59:02.270 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ compare_demoted_promoted_image 10 2024-09-12T13:59:02.270 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 125582 2024-09-12T13:59:02.270 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local dev=/dev/rbd9 2024-09-12T13:59:02.270 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local img=image-primary10 2024-09-12T13:59:02.270 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mntpt=test-primary10 2024-09-12T13:59:02.270 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local demote_md5 promote_md5 2024-09-12T13:59:02.270 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo umount test-primary10 2024-09-12T13:59:02.976 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ awk '{print $1}' 2024-09-12T13:59:02.977 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo md5sum /dev/rbd1 2024-09-12T13:59:03.202 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo md5sum /dev/rbd2 2024-09-12T13:59:03.202 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ awk '{print $1}' 2024-09-12T13:59:03.259 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo md5sum /dev/rbd6 2024-09-12T13:59:03.261 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ awk '{print $1}' 2024-09-12T13:59:03.271 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo md5sum /dev/rbd4 2024-09-12T13:59:03.282 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ awk '{print $1}' 2024-09-12T13:59:03.287 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo md5sum /dev/rbd8 2024-09-12T13:59:03.292 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ awk '{print $1}' 2024-09-12T13:59:03.330 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ awk '{print $1}' 2024-09-12T13:59:03.330 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo md5sum /dev/rbd3 2024-09-12T13:59:03.345 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo md5sum /dev/rbd5 2024-09-12T13:59:03.348 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ awk '{print $1}' 2024-09-12T13:59:03.406 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ awk '{print $1}' 2024-09-12T13:59:03.406 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo md5sum /dev/rbd9 2024-09-12T13:59:03.443 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo md5sum /dev/rbd0 2024-09-12T13:59:03.446 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ awk '{print $1}' 2024-09-12T13:59:03.451 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ awk '{print $1}' 2024-09-12T13:59:03.451 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo md5sum /dev/rbd7 2024-09-12T13:59:51.366 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ demote_md5=52053d6300d10447696f51f977e345de 2024-09-12T13:59:51.367 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo rbd --cluster cluster1 device unmap -t krbd mirror/image-primary8 2024-09-12T13:59:51.761 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ demote_image cluster1 mirror image-primary8 2024-09-12T13:59:51.761 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:59:51.761 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:59:51.761 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:59:51.761 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 mirror image demote mirror/image-primary8 2024-09-12T13:59:56.583 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Image demoted to non-primary 2024-09-12T13:59:56.619 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary8 up+unknown 2024-09-12T13:59:56.619 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:59:56.619 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:59:56.619 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:59:56.619 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:59:56.619 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:59:56.619 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:59:56.619 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:59:56.619 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:59:57.036 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ demote_md5=b0d008a8a9853471a66e05226b789b0d 2024-09-12T13:59:57.036 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo rbd --cluster cluster1 device unmap -t krbd mirror/image-primary7 2024-09-12T13:59:57.327 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ demote_image cluster1 mirror image-primary7 2024-09-12T13:59:57.327 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:59:57.328 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:59:57.328 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:59:57.328 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 mirror image demote mirror/image-primary7 2024-09-12T13:59:57.596 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Image demoted to non-primary 2024-09-12T13:59:57.620 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary7 up+unknown 2024-09-12T13:59:57.620 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:59:57.620 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:59:57.620 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:59:57.620 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:59:57.620 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:59:57.620 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:59:57.620 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:59:57.620 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T13:59:57.631 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary8 up+unknown '' '' 2024-09-12T13:59:57.631 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:59:57.631 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:59:57.632 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T13:59:57.632 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:59:57.632 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:59:57.632 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:59:57.632 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:59:57.634 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:59:57.634 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary8 2024-09-12T13:59:58.017 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary8: 2024-09-12T13:59:58.017 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 3b4d0b11-5c89-4212-85d3-673c05d34c4b 2024-09-12T13:59:58.017 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:59:58.017 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:59:58.017 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:59:58.017 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:59:47' 2024-09-12T13:59:58.017 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:59:58.056 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:59:58.056 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:59:58.056 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 2 2024-09-12T13:59:58.217 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ demote_md5=529c2397fc816698df697aebe7c795bb 2024-09-12T13:59:58.218 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo rbd --cluster cluster1 device unmap -t krbd mirror/image-primary4 2024-09-12T13:59:58.539 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ demote_image cluster1 mirror image-primary4 2024-09-12T13:59:58.541 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:59:58.546 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:59:58.546 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T13:59:58.546 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 mirror image demote mirror/image-primary4 2024-09-12T13:59:58.629 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary7 up+unknown '' '' 2024-09-12T13:59:58.629 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:59:58.629 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:59:58.630 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T13:59:58.630 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T13:59:58.630 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T13:59:58.630 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T13:59:58.630 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T13:59:58.631 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T13:59:58.631 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary7 2024-09-12T13:59:58.665 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ demote_md5=ce70d4e2e62e55a655c7eab38f710d5f 2024-09-12T13:59:58.665 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo rbd --cluster cluster1 device unmap -t krbd mirror/image-primary10 2024-09-12T13:59:58.801 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary7: 2024-09-12T13:59:58.801 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 700ac9cd-0dca-415e-b998-79da95582617 2024-09-12T13:59:58.802 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T13:59:58.802 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T13:59:58.802 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T13:59:58.802 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:59:47' 2024-09-12T13:59:58.802 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T13:59:58.806 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T13:59:58.806 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T13:59:58.806 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 2 2024-09-12T13:59:58.810 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ demote_md5=c9fed370b723bc4064e2a8903a705a35 2024-09-12T13:59:58.810 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo rbd --cluster cluster1 device unmap -t krbd mirror/image-primary6 2024-09-12T13:59:58.901 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ demote_image cluster1 mirror image-primary10 2024-09-12T13:59:58.901 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:59:58.901 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:59:58.901 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T13:59:58.902 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 mirror image demote mirror/image-primary10 2024-09-12T13:59:58.999 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ demote_md5=c6621649b323834dc1c05405228ef4ef 2024-09-12T13:59:59.000 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo rbd --cluster cluster1 device unmap -t krbd mirror/image-primary2 2024-09-12T13:59:59.014 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ demote_image cluster1 mirror image-primary6 2024-09-12T13:59:59.015 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:59:59.015 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:59:59.015 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T13:59:59.015 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 mirror image demote mirror/image-primary6 2024-09-12T13:59:59.104 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ demote_image cluster1 mirror image-primary2 2024-09-12T13:59:59.104 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:59:59.104 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:59:59.104 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T13:59:59.104 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 mirror image demote mirror/image-primary2 2024-09-12T13:59:59.436 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ demote_md5=9be7a78afff9f66c8c56e194e9abfe39 2024-09-12T13:59:59.436 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo rbd --cluster cluster1 device unmap -t krbd mirror/image-primary9 2024-09-12T13:59:59.520 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ demote_md5=02511b187a77833c75a65a214cb45aca 2024-09-12T13:59:59.520 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo rbd --cluster cluster1 device unmap -t krbd mirror/image-primary1 2024-09-12T13:59:59.541 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ demote_md5=e1b3a544e30e3055ba282d8ae18b851f 2024-09-12T13:59:59.541 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo rbd --cluster cluster1 device unmap -t krbd mirror/image-primary3 2024-09-12T13:59:59.592 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ demote_image cluster1 mirror image-primary9 2024-09-12T13:59:59.592 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:59:59.592 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:59:59.592 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T13:59:59.592 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 mirror image demote mirror/image-primary9 2024-09-12T13:59:59.646 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ demote_image cluster1 mirror image-primary1 2024-09-12T13:59:59.646 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:59:59.646 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:59:59.646 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T13:59:59.646 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 mirror image demote mirror/image-primary1 2024-09-12T13:59:59.664 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ demote_image cluster1 mirror image-primary3 2024-09-12T13:59:59.664 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T13:59:59.664 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T13:59:59.664 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T13:59:59.664 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 mirror image demote mirror/image-primary3 2024-09-12T13:59:59.992 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ demote_md5=e1196c387225daba9797472f838a3155 2024-09-12T13:59:59.992 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo rbd --cluster cluster1 device unmap -t krbd mirror/image-primary5 2024-09-12T14:00:00.062 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary8 up+unknown '' '' 2024-09-12T14:00:00.064 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:00:00.064 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:00:00.064 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T14:00:00.064 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:00:00.064 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:00:00.064 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:00:00.064 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:00:00.065 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:00:00.065 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary8 2024-09-12T14:00:00.176 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ demote_image cluster1 mirror image-primary5 2024-09-12T14:00:00.176 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:00:00.177 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:00:00.177 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T14:00:00.177 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 mirror image demote mirror/image-primary5 2024-09-12T14:00:00.195 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary8: 2024-09-12T14:00:00.196 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 3b4d0b11-5c89-4212-85d3-673c05d34c4b 2024-09-12T14:00:00.196 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T14:00:00.196 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T14:00:00.196 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T14:00:00.196 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:59:47' 2024-09-12T14:00:00.196 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T14:00:00.209 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T14:00:00.213 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:00:00.214 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 4 2024-09-12T14:00:00.612 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Image demoted to non-primary 2024-09-12T14:00:00.613 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Image demoted to non-primary 2024-09-12T14:00:00.615 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Image demoted to non-primary 2024-09-12T14:00:00.626 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary10 up+unknown 2024-09-12T14:00:00.627 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:00:00.627 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:00:00.627 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T14:00:00.627 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:00:00.627 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:00:00.627 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:00:00.627 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:00:00.627 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T14:00:00.633 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary6 up+unknown 2024-09-12T14:00:00.633 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:00:00.633 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:00:00.633 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T14:00:00.633 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:00:00.633 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:00:00.634 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:00:00.634 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:00:00.634 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T14:00:00.641 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary2 up+unknown 2024-09-12T14:00:00.642 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:00:00.642 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:00:00.642 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T14:00:00.642 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:00:00.642 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:00:00.642 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:00:00.642 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:00:00.642 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T14:00:00.810 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary7 up+unknown '' '' 2024-09-12T14:00:00.811 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:00:00.811 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:00:00.811 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T14:00:00.811 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:00:00.811 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:00:00.811 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:00:00.811 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:00:00.811 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:00:00.811 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary7 2024-09-12T14:00:00.881 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary7: 2024-09-12T14:00:00.881 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 700ac9cd-0dca-415e-b998-79da95582617 2024-09-12T14:00:00.881 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T14:00:00.881 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T14:00:00.881 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T14:00:00.881 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:59:47' 2024-09-12T14:00:00.881 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T14:00:00.883 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T14:00:00.884 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:00:00.884 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 4 2024-09-12T14:00:01.613 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Image demoted to non-primary 2024-09-12T14:00:01.614 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Image demoted to non-primary 2024-09-12T14:00:01.630 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary10 up+unknown '' '' 2024-09-12T14:00:01.630 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:00:01.630 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:00:01.630 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T14:00:01.630 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:00:01.630 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:00:01.630 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:00:01.630 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:00:01.633 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:00:01.634 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary10 2024-09-12T14:00:01.637 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary6 up+unknown '' '' 2024-09-12T14:00:01.637 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:00:01.637 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:00:01.637 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T14:00:01.638 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:00:01.638 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:00:01.638 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:00:01.638 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:00:01.638 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Image demoted to non-primary 2024-09-12T14:00:01.640 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:00:01.641 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary6 2024-09-12T14:00:01.644 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary2 up+unknown '' '' 2024-09-12T14:00:01.646 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:00:01.646 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:00:01.647 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T14:00:01.648 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:00:01.648 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:00:01.648 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:00:01.648 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:00:01.650 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Image demoted to non-primary 2024-09-12T14:00:01.651 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:00:01.651 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary2 2024-09-12T14:00:01.675 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary9 up+unknown 2024-09-12T14:00:01.675 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:00:01.675 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:00:01.675 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T14:00:01.675 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:00:01.675 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:00:01.675 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:00:01.676 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:00:01.676 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T14:00:01.682 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary5 up+unknown 2024-09-12T14:00:01.682 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:00:01.683 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:00:01.683 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T14:00:01.683 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:00:01.683 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:00:01.683 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:00:01.683 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:00:01.683 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T14:00:01.688 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary4 up+unknown 2024-09-12T14:00:01.688 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:00:01.689 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:00:01.689 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T14:00:01.689 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:00:01.689 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:00:01.689 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:00:01.689 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:00:01.689 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T14:00:01.718 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary1 up+unknown 2024-09-12T14:00:01.719 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:00:01.719 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:00:01.719 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T14:00:01.720 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:00:01.720 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:00:01.720 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:00:01.721 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:00:01.721 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T14:00:01.875 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary6: 2024-09-12T14:00:01.875 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: a63c238c-944e-44a1-be22-edc69436b860 2024-09-12T14:00:01.875 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T14:00:01.875 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T14:00:01.875 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T14:00:01.875 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:59:47' 2024-09-12T14:00:01.876 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T14:00:01.879 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T14:00:01.879 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:00:01.879 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 2 2024-09-12T14:00:01.909 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary10: 2024-09-12T14:00:01.909 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 3238adfd-24b7-48b2-ab32-66aaa1c7b583 2024-09-12T14:00:01.909 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T14:00:01.909 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T14:00:01.909 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T14:00:01.909 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:59:47' 2024-09-12T14:00:01.909 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T14:00:01.921 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T14:00:01.921 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:00:01.921 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 2 2024-09-12T14:00:01.922 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary2: 2024-09-12T14:00:01.922 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: fef52d8d-af21-4d3d-97e9-021eac35b36a 2024-09-12T14:00:01.922 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T14:00:01.922 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T14:00:01.922 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T14:00:01.929 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:59:47' 2024-09-12T14:00:01.929 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T14:00:01.934 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T14:00:01.934 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:00:01.934 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 2 2024-09-12T14:00:02.682 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary9 up+unknown '' '' 2024-09-12T14:00:02.682 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:00:02.682 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:00:02.682 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T14:00:02.682 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:00:02.682 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:00:02.682 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:00:02.682 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:00:02.685 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:00:02.685 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary9 2024-09-12T14:00:02.687 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary5 up+unknown '' '' 2024-09-12T14:00:02.687 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:00:02.687 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:00:02.687 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T14:00:02.687 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:00:02.687 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:00:02.687 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:00:02.687 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:00:02.688 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:00:02.688 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary5 2024-09-12T14:00:02.700 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary4 up+unknown '' '' 2024-09-12T14:00:02.700 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:00:02.700 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:00:02.701 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T14:00:02.701 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:00:02.701 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:00:02.701 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:00:02.701 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:00:02.703 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:00:02.703 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary4 2024-09-12T14:00:02.725 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary1 up+unknown '' '' 2024-09-12T14:00:02.726 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:00:02.727 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:00:02.727 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T14:00:02.727 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:00:02.727 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:00:02.727 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:00:02.727 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:00:02.730 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:00:02.730 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary1 2024-09-12T14:00:02.845 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary9: 2024-09-12T14:00:02.845 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 41a87723-a798-494d-b064-df92256304f2 2024-09-12T14:00:02.846 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T14:00:02.846 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T14:00:02.846 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T14:00:02.846 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:59:47' 2024-09-12T14:00:02.846 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T14:00:02.846 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary5: 2024-09-12T14:00:02.846 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: fb69fceb-7a10-4ab7-ad45-ec4124a4621f 2024-09-12T14:00:02.846 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T14:00:02.846 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T14:00:02.846 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T14:00:02.846 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:59:47' 2024-09-12T14:00:02.846 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T14:00:02.849 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T14:00:02.849 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:00:02.850 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 2 2024-09-12T14:00:02.851 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T14:00:02.851 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:00:02.851 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 2 2024-09-12T14:00:02.854 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary1: 2024-09-12T14:00:02.854 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: a862d41a-8d34-4c63-bea6-4195101bf11a 2024-09-12T14:00:02.854 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T14:00:02.854 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T14:00:02.854 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T14:00:02.854 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:59:47' 2024-09-12T14:00:02.854 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T14:00:02.857 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T14:00:02.857 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:00:02.857 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 2 2024-09-12T14:00:02.861 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary4: 2024-09-12T14:00:02.861 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 8face4e5-1ee3-427e-b5bc-7b05be9d6e87 2024-09-12T14:00:02.861 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T14:00:02.861 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T14:00:02.861 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T14:00:02.861 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:59:47' 2024-09-12T14:00:02.861 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T14:00:02.863 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T14:00:02.863 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:00:02.863 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 2 2024-09-12T14:00:03.883 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary6 up+unknown '' '' 2024-09-12T14:00:03.883 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:00:03.883 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:00:03.883 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T14:00:03.883 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:00:03.883 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:00:03.883 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:00:03.883 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:00:03.884 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:00:03.884 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary6 2024-09-12T14:00:03.925 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary10 up+unknown '' '' 2024-09-12T14:00:03.925 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:00:03.925 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:00:03.925 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T14:00:03.925 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:00:03.925 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:00:03.925 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:00:03.925 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:00:03.926 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:00:03.926 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary10 2024-09-12T14:00:03.937 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary2 up+unknown '' '' 2024-09-12T14:00:03.937 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:00:03.937 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:00:03.937 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T14:00:03.937 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:00:03.937 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:00:03.938 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:00:03.938 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:00:03.939 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:00:03.939 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary2 2024-09-12T14:00:03.962 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary6: 2024-09-12T14:00:03.962 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: a63c238c-944e-44a1-be22-edc69436b860 2024-09-12T14:00:03.962 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T14:00:03.962 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T14:00:03.962 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T14:00:03.962 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:59:47' 2024-09-12T14:00:03.962 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T14:00:03.964 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T14:00:03.965 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:00:03.965 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 4 2024-09-12T14:00:04.001 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary10: 2024-09-12T14:00:04.001 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 3238adfd-24b7-48b2-ab32-66aaa1c7b583 2024-09-12T14:00:04.002 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T14:00:04.002 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T14:00:04.002 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T14:00:04.002 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:59:47' 2024-09-12T14:00:04.002 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T14:00:04.004 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T14:00:04.004 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:00:04.004 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 4 2024-09-12T14:00:04.011 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary2: 2024-09-12T14:00:04.011 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: fef52d8d-af21-4d3d-97e9-021eac35b36a 2024-09-12T14:00:04.012 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T14:00:04.012 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T14:00:04.012 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T14:00:04.012 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:59:47' 2024-09-12T14:00:04.012 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T14:00:04.014 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T14:00:04.014 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:00:04.014 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 4 2024-09-12T14:00:04.217 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary8 up+unknown '' '' 2024-09-12T14:00:04.218 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:00:04.218 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:00:04.218 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T14:00:04.218 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:00:04.218 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:00:04.218 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:00:04.218 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:00:04.218 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:00:04.219 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary8 2024-09-12T14:00:04.293 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary8: 2024-09-12T14:00:04.293 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 3b4d0b11-5c89-4212-85d3-673c05d34c4b 2024-09-12T14:00:04.293 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T14:00:04.294 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T14:00:04.294 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T14:00:04.294 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:59:47' 2024-09-12T14:00:04.294 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T14:00:04.296 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T14:00:04.296 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:00:04.296 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 8 2024-09-12T14:00:04.412 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Image demoted to non-primary 2024-09-12T14:00:04.423 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary3 up+unknown 2024-09-12T14:00:04.423 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:00:04.423 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:00:04.424 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T14:00:04.424 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:00:04.424 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:00:04.424 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:00:04.424 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:00:04.424 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T14:00:04.853 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary5 up+unknown '' '' 2024-09-12T14:00:04.854 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:00:04.854 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:00:04.854 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T14:00:04.854 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:00:04.854 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:00:04.854 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:00:04.854 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:00:04.854 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary9 up+unknown '' '' 2024-09-12T14:00:04.854 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:00:04.854 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:00:04.855 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T14:00:04.855 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:00:04.855 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:00:04.855 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:00:04.855 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:00:04.855 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:00:04.855 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary5 2024-09-12T14:00:04.855 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:00:04.856 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary9 2024-09-12T14:00:04.859 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary1 up+unknown '' '' 2024-09-12T14:00:04.859 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:00:04.859 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:00:04.859 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T14:00:04.859 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:00:04.859 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:00:04.860 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:00:04.860 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:00:04.860 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:00:04.860 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary1 2024-09-12T14:00:04.866 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary4 up+unknown '' '' 2024-09-12T14:00:04.866 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:00:04.866 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:00:04.866 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T14:00:04.866 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:00:04.866 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:00:04.866 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:00:04.866 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:00:04.867 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:00:04.867 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary4 2024-09-12T14:00:04.886 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary7 up+unknown '' '' 2024-09-12T14:00:04.886 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:00:04.886 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:00:04.886 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T14:00:04.886 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:00:04.886 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:00:04.887 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:00:04.887 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:00:04.887 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:00:04.887 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary7 2024-09-12T14:00:04.942 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary5: 2024-09-12T14:00:04.942 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: fb69fceb-7a10-4ab7-ad45-ec4124a4621f 2024-09-12T14:00:04.942 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T14:00:04.942 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T14:00:04.942 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T14:00:04.943 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:59:47' 2024-09-12T14:00:04.943 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T14:00:04.943 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary9: 2024-09-12T14:00:04.943 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 41a87723-a798-494d-b064-df92256304f2 2024-09-12T14:00:04.943 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T14:00:04.943 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T14:00:04.943 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T14:00:04.943 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:59:47' 2024-09-12T14:00:04.943 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T14:00:04.945 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T14:00:04.946 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:00:04.946 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 4 2024-09-12T14:00:04.946 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T14:00:04.946 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:00:04.946 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 4 2024-09-12T14:00:04.947 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary1: 2024-09-12T14:00:04.947 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: a862d41a-8d34-4c63-bea6-4195101bf11a 2024-09-12T14:00:04.947 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T14:00:04.947 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T14:00:04.947 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T14:00:04.948 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:59:47' 2024-09-12T14:00:04.948 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T14:00:04.950 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T14:00:04.950 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:00:04.950 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 4 2024-09-12T14:00:04.966 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary4: 2024-09-12T14:00:04.966 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 8face4e5-1ee3-427e-b5bc-7b05be9d6e87 2024-09-12T14:00:04.966 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T14:00:04.966 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T14:00:04.966 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T14:00:04.966 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:59:47' 2024-09-12T14:00:04.966 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T14:00:04.969 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T14:00:04.969 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:00:04.969 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 4 2024-09-12T14:00:04.978 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary7: 2024-09-12T14:00:04.978 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 700ac9cd-0dca-415e-b998-79da95582617 2024-09-12T14:00:04.978 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T14:00:04.978 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T14:00:04.978 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T14:00:04.979 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:59:47' 2024-09-12T14:00:04.979 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T14:00:04.981 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T14:00:04.981 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:00:04.981 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 8 2024-09-12T14:00:05.425 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary3 up+unknown '' '' 2024-09-12T14:00:05.426 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:00:05.426 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:00:05.426 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T14:00:05.426 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:00:05.426 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:00:05.426 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:00:05.426 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:00:05.426 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:00:05.426 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary3 2024-09-12T14:00:05.500 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary3: 2024-09-12T14:00:05.500 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: a8c53447-94fc-4fe3-b559-d953f6f94c54 2024-09-12T14:00:05.500 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T14:00:05.500 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T14:00:05.501 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T14:00:05.501 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:59:47' 2024-09-12T14:00:05.501 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T14:00:05.503 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T14:00:05.503 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:00:05.503 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 2 2024-09-12T14:00:07.505 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary3 up+unknown '' '' 2024-09-12T14:00:07.505 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:00:07.506 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:00:07.506 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T14:00:07.506 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:00:07.506 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:00:07.506 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:00:07.506 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:00:07.506 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:00:07.506 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary3 2024-09-12T14:00:07.590 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary3: 2024-09-12T14:00:07.590 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: a8c53447-94fc-4fe3-b559-d953f6f94c54 2024-09-12T14:00:07.590 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T14:00:07.590 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T14:00:07.590 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T14:00:07.590 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:59:47' 2024-09-12T14:00:07.590 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T14:00:07.593 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T14:00:07.593 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:00:07.593 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 4 2024-09-12T14:00:07.967 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary6 up+unknown '' '' 2024-09-12T14:00:07.967 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:00:07.967 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:00:07.968 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T14:00:07.968 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:00:07.968 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:00:07.968 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:00:07.968 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:00:07.968 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:00:07.968 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary6 2024-09-12T14:00:08.007 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary10 up+unknown '' '' 2024-09-12T14:00:08.007 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:00:08.007 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:00:08.007 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T14:00:08.007 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:00:08.007 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:00:08.007 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:00:08.007 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:00:08.008 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:00:08.008 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary10 2024-09-12T14:00:08.017 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary2 up+unknown '' '' 2024-09-12T14:00:08.017 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:00:08.017 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:00:08.017 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T14:00:08.017 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:00:08.017 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:00:08.017 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:00:08.017 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:00:08.018 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:00:08.018 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary2 2024-09-12T14:00:08.049 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary6: 2024-09-12T14:00:08.049 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: a63c238c-944e-44a1-be22-edc69436b860 2024-09-12T14:00:08.049 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T14:00:08.049 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T14:00:08.049 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T14:00:08.049 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:59:47' 2024-09-12T14:00:08.049 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T14:00:08.052 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T14:00:08.052 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:00:08.052 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 8 2024-09-12T14:00:08.087 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary10: 2024-09-12T14:00:08.087 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 3238adfd-24b7-48b2-ab32-66aaa1c7b583 2024-09-12T14:00:08.087 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T14:00:08.087 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T14:00:08.087 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T14:00:08.087 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:59:47' 2024-09-12T14:00:08.087 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T14:00:08.090 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T14:00:08.090 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:00:08.090 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 8 2024-09-12T14:00:08.097 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary2: 2024-09-12T14:00:08.097 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: fef52d8d-af21-4d3d-97e9-021eac35b36a 2024-09-12T14:00:08.098 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T14:00:08.098 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T14:00:08.098 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T14:00:08.098 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:59:47' 2024-09-12T14:00:08.098 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T14:00:08.100 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T14:00:08.101 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:00:08.101 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 8 2024-09-12T14:00:08.949 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary5 up+unknown '' '' 2024-09-12T14:00:08.949 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:00:08.949 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:00:08.949 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T14:00:08.949 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:00:08.949 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:00:08.949 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary9 up+unknown '' '' 2024-09-12T14:00:08.949 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:00:08.950 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:00:08.950 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:00:08.950 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:00:08.950 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T14:00:08.950 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:00:08.950 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:00:08.950 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:00:08.950 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:00:08.950 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:00:08.950 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary9 2024-09-12T14:00:08.951 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:00:08.951 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary5 2024-09-12T14:00:08.953 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary1 up+unknown '' '' 2024-09-12T14:00:08.953 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:00:08.953 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:00:08.953 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T14:00:08.953 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:00:08.953 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:00:08.953 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:00:08.953 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:00:08.954 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:00:08.954 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary1 2024-09-12T14:00:08.971 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary4 up+unknown '' '' 2024-09-12T14:00:08.972 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:00:08.972 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:00:08.972 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T14:00:08.972 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:00:08.972 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:00:08.972 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:00:08.972 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:00:08.972 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:00:08.973 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary4 2024-09-12T14:00:09.035 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary9: 2024-09-12T14:00:09.035 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 41a87723-a798-494d-b064-df92256304f2 2024-09-12T14:00:09.036 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T14:00:09.036 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T14:00:09.036 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T14:00:09.036 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:59:47' 2024-09-12T14:00:09.036 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T14:00:09.037 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary5: 2024-09-12T14:00:09.037 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: fb69fceb-7a10-4ab7-ad45-ec4124a4621f 2024-09-12T14:00:09.037 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T14:00:09.037 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T14:00:09.037 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T14:00:09.037 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:59:47' 2024-09-12T14:00:09.037 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T14:00:09.037 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary1: 2024-09-12T14:00:09.037 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: a862d41a-8d34-4c63-bea6-4195101bf11a 2024-09-12T14:00:09.037 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T14:00:09.037 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T14:00:09.037 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T14:00:09.038 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:59:47' 2024-09-12T14:00:09.038 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T14:00:09.038 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T14:00:09.038 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:00:09.038 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 8 2024-09-12T14:00:09.039 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T14:00:09.039 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:00:09.039 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 8 2024-09-12T14:00:09.040 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T14:00:09.040 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:00:09.040 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 8 2024-09-12T14:00:09.054 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary4: 2024-09-12T14:00:09.054 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 8face4e5-1ee3-427e-b5bc-7b05be9d6e87 2024-09-12T14:00:09.054 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T14:00:09.054 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T14:00:09.054 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T14:00:09.054 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:59:47' 2024-09-12T14:00:09.054 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T14:00:09.056 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T14:00:09.057 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:00:09.057 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 8 2024-09-12T14:00:11.596 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary3 up+unknown '' '' 2024-09-12T14:00:11.596 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:00:11.596 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:00:11.596 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T14:00:11.596 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:00:11.596 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:00:11.596 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:00:11.596 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:00:11.597 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:00:11.597 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary3 2024-09-12T14:00:11.679 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary3: 2024-09-12T14:00:11.679 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: a8c53447-94fc-4fe3-b559-d953f6f94c54 2024-09-12T14:00:11.679 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T14:00:11.679 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T14:00:11.680 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T14:00:11.680 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:59:47' 2024-09-12T14:00:11.680 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T14:00:11.682 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T14:00:11.682 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:00:11.682 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 8 2024-09-12T14:00:12.298 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary8 up+unknown '' '' 2024-09-12T14:00:12.299 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:00:12.299 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:00:12.299 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T14:00:12.299 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:00:12.299 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:00:12.299 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:00:12.299 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:00:12.299 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:00:12.299 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary8 2024-09-12T14:00:12.378 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary8: 2024-09-12T14:00:12.378 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 3b4d0b11-5c89-4212-85d3-673c05d34c4b 2024-09-12T14:00:12.378 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T14:00:12.378 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T14:00:12.379 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T14:00:12.379 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:59:47' 2024-09-12T14:00:12.379 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T14:00:12.381 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T14:00:12.381 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:00:12.381 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 8 2024-09-12T14:00:12.984 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary7 up+unknown '' '' 2024-09-12T14:00:12.984 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:00:12.984 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:00:12.984 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T14:00:12.984 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:00:12.985 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:00:12.985 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:00:12.985 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:00:12.985 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:00:12.985 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary7 2024-09-12T14:00:13.065 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary7: 2024-09-12T14:00:13.065 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 700ac9cd-0dca-415e-b998-79da95582617 2024-09-12T14:00:13.065 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T14:00:13.065 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T14:00:13.065 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T14:00:13.065 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:59:47' 2024-09-12T14:00:13.065 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T14:00:13.067 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T14:00:13.068 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:00:13.068 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 8 2024-09-12T14:00:16.055 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary6 up+unknown '' '' 2024-09-12T14:00:16.055 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:00:16.055 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:00:16.055 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T14:00:16.055 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:00:16.055 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:00:16.056 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:00:16.056 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:00:16.056 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:00:16.056 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary6 2024-09-12T14:00:16.093 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary10 up+unknown '' '' 2024-09-12T14:00:16.093 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:00:16.093 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:00:16.093 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T14:00:16.093 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:00:16.093 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:00:16.093 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:00:16.093 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:00:16.093 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:00:16.094 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary10 2024-09-12T14:00:16.102 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary2 up+unknown '' '' 2024-09-12T14:00:16.102 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:00:16.102 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:00:16.102 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T14:00:16.103 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:00:16.103 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:00:16.103 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:00:16.103 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:00:16.103 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:00:16.103 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary2 2024-09-12T14:00:16.136 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary6: 2024-09-12T14:00:16.136 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: a63c238c-944e-44a1-be22-edc69436b860 2024-09-12T14:00:16.136 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T14:00:16.136 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T14:00:16.136 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T14:00:16.136 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:59:47' 2024-09-12T14:00:16.136 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T14:00:16.139 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T14:00:16.139 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:00:16.139 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 8 2024-09-12T14:00:16.173 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary10: 2024-09-12T14:00:16.174 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 3238adfd-24b7-48b2-ab32-66aaa1c7b583 2024-09-12T14:00:16.174 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T14:00:16.174 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T14:00:16.174 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T14:00:16.174 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:59:47' 2024-09-12T14:00:16.174 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T14:00:16.176 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T14:00:16.177 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:00:16.177 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 8 2024-09-12T14:00:16.179 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary2: 2024-09-12T14:00:16.180 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: fef52d8d-af21-4d3d-97e9-021eac35b36a 2024-09-12T14:00:16.180 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T14:00:16.180 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T14:00:16.180 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T14:00:16.180 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:59:47' 2024-09-12T14:00:16.180 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T14:00:16.182 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T14:00:16.182 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:00:16.183 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 8 2024-09-12T14:00:17.042 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary9 up+unknown '' '' 2024-09-12T14:00:17.042 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:00:17.042 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:00:17.042 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T14:00:17.042 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:00:17.042 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:00:17.042 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:00:17.042 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:00:17.042 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary5 up+unknown '' '' 2024-09-12T14:00:17.042 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:00:17.042 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:00:17.042 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T14:00:17.043 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:00:17.043 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:00:17.043 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:00:17.043 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:00:17.043 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary1 up+unknown '' '' 2024-09-12T14:00:17.044 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:00:17.044 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary9 2024-09-12T14:00:17.044 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:00:17.044 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:00:17.044 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T14:00:17.044 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:00:17.044 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:00:17.044 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:00:17.044 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:00:17.044 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:00:17.044 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary5 2024-09-12T14:00:17.044 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:00:17.044 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary1 2024-09-12T14:00:17.059 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary4 up+unknown '' '' 2024-09-12T14:00:17.059 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:00:17.059 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:00:17.059 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T14:00:17.060 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:00:17.060 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:00:17.060 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:00:17.060 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:00:17.060 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:00:17.060 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary4 2024-09-12T14:00:17.130 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary9: 2024-09-12T14:00:17.130 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 41a87723-a798-494d-b064-df92256304f2 2024-09-12T14:00:17.130 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T14:00:17.130 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T14:00:17.130 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T14:00:17.130 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:59:47' 2024-09-12T14:00:17.130 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T14:00:17.131 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary5: 2024-09-12T14:00:17.131 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: fb69fceb-7a10-4ab7-ad45-ec4124a4621f 2024-09-12T14:00:17.131 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T14:00:17.131 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T14:00:17.131 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T14:00:17.131 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:59:47' 2024-09-12T14:00:17.131 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T14:00:17.132 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary1: 2024-09-12T14:00:17.132 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: a862d41a-8d34-4c63-bea6-4195101bf11a 2024-09-12T14:00:17.132 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T14:00:17.132 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T14:00:17.133 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T14:00:17.133 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:59:47' 2024-09-12T14:00:17.133 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T14:00:17.133 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T14:00:17.133 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:00:17.133 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 8 2024-09-12T14:00:17.134 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T14:00:17.134 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:00:17.134 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 8 2024-09-12T14:00:17.135 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T14:00:17.135 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:00:17.135 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 8 2024-09-12T14:00:17.145 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary4: 2024-09-12T14:00:17.145 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 8face4e5-1ee3-427e-b5bc-7b05be9d6e87 2024-09-12T14:00:17.145 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T14:00:17.145 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T14:00:17.145 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T14:00:17.145 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 13:59:47' 2024-09-12T14:00:17.145 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T14:00:17.148 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T14:00:17.148 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:00:17.148 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 8 2024-09-12T14:00:19.684 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary3 up+unknown '' '' 2024-09-12T14:00:19.685 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:00:19.685 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:00:19.685 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T14:00:19.685 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:00:19.685 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:00:19.685 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:00:19.685 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:00:19.686 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:00:19.686 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary3 2024-09-12T14:00:19.768 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary3: 2024-09-12T14:00:19.768 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: a8c53447-94fc-4fe3-b559-d953f6f94c54 2024-09-12T14:00:19.768 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+unknown 2024-09-12T14:00:19.768 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: remote image is not primary 2024-09-12T14:00:19.769 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T14:00:19.769 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 14:00:17' 2024-09-12T14:00:19.769 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T14:00:19.770 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+unknown 2024-09-12T14:00:19.771 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T14:00:19.773 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: remote image is not primary 2024-09-12T14:00:19.774 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T14:00:19.774 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T14:00:19.775 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T14:00:19.777 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T14:00:19.777 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T14:00:19.780 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T14:00:19.780 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T14:00:19.781 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T14:00:19.785 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 14:00:17' 2024-09-12T14:00:19.785 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary3 up+unknown '2024-09-12 14:00:17' 2024-09-12T14:00:19.785 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:00:19.785 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:00:19.785 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T14:00:19.785 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:00:19.785 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 14:00:17' 2024-09-12T14:00:19.786 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:00:19.786 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:00:19.786 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-09-12T14:00:19.894 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041934232mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary1a862d41a-8d34-4c63-bea6-4195101bf11aup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:00:17image-primary103238adfd-24b7-48b2-ab32-66aaa1c7b583up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:00:17image-primary2fef52d8d-af21-4d3d-97e9-021eac35b36aup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:00:17image-primary3a8c53447-94fc-4fe3-b559-d953f6f94c54up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:00:17image-primary48face4e5-1ee3-427e-b5bc-7b05be9d6e87up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:00:17image-primary5fb69fceb-7a10-4ab7-ad45-ec4124a4621fup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:00:17image-primary6a63c238c-944e-44a1-be22-edc69436b860up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:00:17image-primary7700ac9cd-0dca-415e-b998-79da95582617up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:00:17image-primary83b4d0b11-5c89-4212-85d3-673c05d34c4bup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:00:17image-primary941a87723-a798-494d-b064-df92256304f2up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:00:17' 2024-09-12T14:00:19.895 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T14:00:19.895 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary3'\'']/last_update' 2024-09-12T14:00:19.936 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 14:00:17' 2024-09-12T14:00:19.937 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary3'\'']/state' 2024-09-12T14:00:19.940 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+unknown 2024-09-12T14:00:19.941 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T14:00:19.941 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+unknown 2024-09-12T14:00:19.943 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T14:00:19.944 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:00:19.944 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:00:19.944 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster2 mirror image-primary3 up+unknown 2024-09-12T14:00:19.944 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T14:00:19.944 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:00:19.945 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T14:00:19.945 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:00:19.945 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:00:19.945 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:00:19.945 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:00:19.945 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T14:00:20.384 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary8 up+unknown '' '' 2024-09-12T14:00:20.384 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:00:20.384 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:00:20.384 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T14:00:20.384 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:00:20.384 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:00:20.384 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:00:20.384 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:00:20.385 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:00:20.385 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary8 2024-09-12T14:00:20.461 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary8: 2024-09-12T14:00:20.461 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 3b4d0b11-5c89-4212-85d3-673c05d34c4b 2024-09-12T14:00:20.461 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+unknown 2024-09-12T14:00:20.461 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: remote image is not primary 2024-09-12T14:00:20.461 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T14:00:20.461 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 14:00:17' 2024-09-12T14:00:20.461 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T14:00:20.463 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+unknown 2024-09-12T14:00:20.464 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T14:00:20.466 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: remote image is not primary 2024-09-12T14:00:20.466 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T14:00:20.467 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T14:00:20.467 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T14:00:20.469 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T14:00:20.470 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T14:00:20.472 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T14:00:20.472 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T14:00:20.473 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T14:00:20.477 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 14:00:17' 2024-09-12T14:00:20.477 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary8 up+unknown '2024-09-12 14:00:17' 2024-09-12T14:00:20.477 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:00:20.477 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:00:20.477 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T14:00:20.477 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:00:20.477 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 14:00:17' 2024-09-12T14:00:20.478 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:00:20.478 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:00:20.478 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-09-12T14:00:20.587 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041934232mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary1a862d41a-8d34-4c63-bea6-4195101bf11aup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:00:17image-primary103238adfd-24b7-48b2-ab32-66aaa1c7b583up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:00:17image-primary2fef52d8d-af21-4d3d-97e9-021eac35b36aup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:00:17image-primary3a8c53447-94fc-4fe3-b559-d953f6f94c54up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:00:17image-primary48face4e5-1ee3-427e-b5bc-7b05be9d6e87up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:00:17image-primary5fb69fceb-7a10-4ab7-ad45-ec4124a4621fup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:00:17image-primary6a63c238c-944e-44a1-be22-edc69436b860up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:00:17image-primary7700ac9cd-0dca-415e-b998-79da95582617up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:00:17image-primary83b4d0b11-5c89-4212-85d3-673c05d34c4bup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:00:17image-primary941a87723-a798-494d-b064-df92256304f2up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:00:17' 2024-09-12T14:00:20.587 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T14:00:20.588 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary8'\'']/last_update' 2024-09-12T14:00:20.592 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 14:00:17' 2024-09-12T14:00:20.593 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary8'\'']/state' 2024-09-12T14:00:20.596 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+unknown 2024-09-12T14:00:20.597 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T14:00:20.597 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+unknown 2024-09-12T14:00:20.599 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T14:00:20.601 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:00:20.601 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:00:20.601 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster2 mirror image-primary8 up+unknown 2024-09-12T14:00:20.601 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T14:00:20.601 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:00:20.601 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T14:00:20.601 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:00:20.601 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:00:20.601 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:00:20.601 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:00:20.601 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T14:00:20.946 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster2 mirror image-primary3 up+unknown '' '' 2024-09-12T14:00:20.947 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T14:00:20.947 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:00:20.947 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T14:00:20.947 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:00:20.947 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:00:20.947 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:00:20.947 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:00:20.947 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:00:20.948 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster2 mirror image status mirror/image-primary3 2024-09-12T14:00:21.030 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary3: 2024-09-12T14:00:21.030 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: a8c53447-94fc-4fe3-b559-d953f6f94c54 2024-09-12T14:00:21.030 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+unknown 2024-09-12T14:00:21.031 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: remote image is not primary 2024-09-12T14:00:21.031 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T14:00:21.031 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 14:00:17' 2024-09-12T14:00:21.031 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T14:00:21.032 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+unknown 2024-09-12T14:00:21.033 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T14:00:21.035 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: remote image is not primary 2024-09-12T14:00:21.036 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T14:00:21.036 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T14:00:21.036 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T14:00:21.038 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T14:00:21.039 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T14:00:21.041 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T14:00:21.041 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T14:00:21.042 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T14:00:21.046 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 14:00:17' 2024-09-12T14:00:21.046 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster2 mirror image-primary3 up+unknown '2024-09-12 14:00:17' 2024-09-12T14:00:21.046 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T14:00:21.046 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:00:21.046 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T14:00:21.046 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:00:21.046 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 14:00:17' 2024-09-12T14:00:21.047 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:00:21.047 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:00:21.047 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2024-09-12T14:00:21.070 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary7 up+unknown '' '' 2024-09-12T14:00:21.070 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:00:21.070 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:00:21.070 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T14:00:21.070 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:00:21.070 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:00:21.070 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:00:21.071 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:00:21.071 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:00:21.071 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary7 2024-09-12T14:00:21.151 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041924231mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary1a862d41a-8d34-4c63-bea6-4195101bf11aup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:00:17image-primary103238adfd-24b7-48b2-ab32-66aaa1c7b583up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:00:17image-primary2fef52d8d-af21-4d3d-97e9-021eac35b36aup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:00:17image-primary3a8c53447-94fc-4fe3-b559-d953f6f94c54up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:00:17image-primary48face4e5-1ee3-427e-b5bc-7b05be9d6e87up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:00:17image-primary5fb69fceb-7a10-4ab7-ad45-ec4124a4621fup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:00:17image-primary6a63c238c-944e-44a1-be22-edc69436b860up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:00:17image-primary7700ac9cd-0dca-415e-b998-79da95582617up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:00:17image-primary83b4d0b11-5c89-4212-85d3-673c05d34c4bup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:00:17image-primary941a87723-a798-494d-b064-df92256304f2up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:00:17' 2024-09-12T14:00:21.151 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T14:00:21.152 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary3'\'']/last_update' 2024-09-12T14:00:21.155 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 14:00:17' 2024-09-12T14:00:21.156 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary3'\'']/state' 2024-09-12T14:00:21.157 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary7: 2024-09-12T14:00:21.157 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 700ac9cd-0dca-415e-b998-79da95582617 2024-09-12T14:00:21.157 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+unknown 2024-09-12T14:00:21.157 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: remote image is not primary 2024-09-12T14:00:21.157 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T14:00:21.157 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 14:00:17' 2024-09-12T14:00:21.157 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T14:00:21.159 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+unknown 2024-09-12T14:00:21.160 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T14:00:21.160 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+unknown 2024-09-12T14:00:21.161 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T14:00:21.161 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+unknown 2024-09-12T14:00:21.162 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: remote image is not primary 2024-09-12T14:00:21.163 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T14:00:21.164 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T14:00:21.164 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T14:00:21.165 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T14:00:21.165 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:00:21.165 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:00:21.165 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ promote_image cluster2 mirror image-primary3 2024-09-12T14:00:21.165 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T14:00:21.165 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:00:21.165 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T14:00:21.165 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local force= 2024-09-12T14:00:21.166 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 mirror image promote mirror/image-primary3 2024-09-12T14:00:21.166 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T14:00:21.166 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T14:00:21.169 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T14:00:21.169 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T14:00:21.170 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T14:00:21.174 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 14:00:17' 2024-09-12T14:00:21.174 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary7 up+unknown '2024-09-12 14:00:17' 2024-09-12T14:00:21.174 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:00:21.174 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:00:21.174 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T14:00:21.174 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:00:21.174 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 14:00:17' 2024-09-12T14:00:21.174 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:00:21.175 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:00:21.175 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-09-12T14:00:21.283 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041934232mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary1a862d41a-8d34-4c63-bea6-4195101bf11aup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:00:17image-primary103238adfd-24b7-48b2-ab32-66aaa1c7b583up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:00:17image-primary2fef52d8d-af21-4d3d-97e9-021eac35b36aup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:00:17image-primary3a8c53447-94fc-4fe3-b559-d953f6f94c54up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:00:17image-primary48face4e5-1ee3-427e-b5bc-7b05be9d6e87up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:00:17image-primary5fb69fceb-7a10-4ab7-ad45-ec4124a4621fup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:00:17image-primary6a63c238c-944e-44a1-be22-edc69436b860up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:00:17image-primary7700ac9cd-0dca-415e-b998-79da95582617up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:00:17image-primary83b4d0b11-5c89-4212-85d3-673c05d34c4bup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:00:17image-primary941a87723-a798-494d-b064-df92256304f2up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:00:17' 2024-09-12T14:00:21.284 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T14:00:21.284 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary7'\'']/last_update' 2024-09-12T14:00:21.288 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 14:00:17' 2024-09-12T14:00:21.289 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary7'\'']/state' 2024-09-12T14:00:21.292 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+unknown 2024-09-12T14:00:21.293 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T14:00:21.293 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+unknown 2024-09-12T14:00:21.295 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T14:00:21.296 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:00:21.296 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:00:21.296 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster2 mirror image-primary7 up+unknown 2024-09-12T14:00:21.296 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T14:00:21.296 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:00:21.296 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T14:00:21.296 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:00:21.297 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:00:21.297 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:00:21.297 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:00:21.297 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T14:00:21.603 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster2 mirror image-primary8 up+unknown '' '' 2024-09-12T14:00:21.603 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T14:00:21.603 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:00:21.603 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T14:00:21.603 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:00:21.603 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:00:21.603 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:00:21.603 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:00:21.603 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:00:21.603 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster2 mirror image status mirror/image-primary8 2024-09-12T14:00:21.676 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary8: 2024-09-12T14:00:21.676 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 3b4d0b11-5c89-4212-85d3-673c05d34c4b 2024-09-12T14:00:21.676 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+unknown 2024-09-12T14:00:21.677 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: remote image is not primary 2024-09-12T14:00:21.677 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T14:00:21.677 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 14:00:17' 2024-09-12T14:00:21.677 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T14:00:21.679 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+unknown 2024-09-12T14:00:21.679 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T14:00:21.682 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: remote image is not primary 2024-09-12T14:00:21.682 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Image promoted to primary 2024-09-12T14:00:21.683 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T14:00:21.683 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T14:00:21.683 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T14:00:21.686 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T14:00:21.686 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T14:00:21.689 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T14:00:21.690 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T14:00:21.690 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T14:00:21.695 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 14:00:17' 2024-09-12T14:00:21.695 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster2 mirror image-primary8 up+unknown '2024-09-12 14:00:17' 2024-09-12T14:00:21.695 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T14:00:21.695 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:00:21.695 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T14:00:21.695 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:00:21.695 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 14:00:17' 2024-09-12T14:00:21.695 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:00:21.695 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \n\b\d ]] 2024-09-12T14:00:21.696 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \k\r\b\d ]] 2024-09-12T14:00:21.696 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:00:21.696 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2024-09-12T14:00:21.696 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo rbd --cluster cluster2 device map -t krbd mirror/image-primary3 2024-09-12T14:00:21.804 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041924231mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary1a862d41a-8d34-4c63-bea6-4195101bf11aup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:00:17image-primary103238adfd-24b7-48b2-ab32-66aaa1c7b583up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:00:17image-primary2fef52d8d-af21-4d3d-97e9-021eac35b36aup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:00:17image-primary3a8c53447-94fc-4fe3-b559-d953f6f94c54up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:00:17image-primary48face4e5-1ee3-427e-b5bc-7b05be9d6e87up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:00:17image-primary5fb69fceb-7a10-4ab7-ad45-ec4124a4621fup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:00:17image-primary6a63c238c-944e-44a1-be22-edc69436b860up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:00:17image-primary7700ac9cd-0dca-415e-b998-79da95582617up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:00:17image-primary83b4d0b11-5c89-4212-85d3-673c05d34c4bup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:00:17image-primary941a87723-a798-494d-b064-df92256304f2up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:00:17' 2024-09-12T14:00:21.805 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T14:00:21.805 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary8'\'']/last_update' 2024-09-12T14:00:21.810 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 14:00:17' 2024-09-12T14:00:21.811 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary8'\'']/state' 2024-09-12T14:00:21.816 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+unknown 2024-09-12T14:00:21.816 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T14:00:21.816 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+unknown 2024-09-12T14:00:21.818 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T14:00:21.819 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:00:21.819 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:00:21.819 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ promote_image cluster2 mirror image-primary8 2024-09-12T14:00:21.819 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T14:00:21.820 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:00:21.820 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T14:00:21.820 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local force= 2024-09-12T14:00:21.820 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 mirror image promote mirror/image-primary8 2024-09-12T14:00:21.864 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ dev=/dev/rbd0 2024-09-12T14:00:21.865 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo md5sum /dev/rbd0 2024-09-12T14:00:21.865 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ awk '{print $1}' 2024-09-12T14:00:22.298 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster2 mirror image-primary7 up+unknown '' '' 2024-09-12T14:00:22.298 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T14:00:22.299 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:00:22.299 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T14:00:22.299 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:00:22.299 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:00:22.299 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:00:22.299 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:00:22.299 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:00:22.299 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster2 mirror image status mirror/image-primary7 2024-09-12T14:00:22.372 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary7: 2024-09-12T14:00:22.372 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 700ac9cd-0dca-415e-b998-79da95582617 2024-09-12T14:00:22.372 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+unknown 2024-09-12T14:00:22.372 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: remote image is not primary 2024-09-12T14:00:22.372 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T14:00:22.372 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 14:00:17' 2024-09-12T14:00:22.372 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T14:00:22.374 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+unknown 2024-09-12T14:00:22.375 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T14:00:22.377 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: remote image is not primary 2024-09-12T14:00:22.378 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T14:00:22.378 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T14:00:22.379 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T14:00:22.381 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T14:00:22.381 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T14:00:22.383 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T14:00:22.384 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T14:00:22.384 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T14:00:22.388 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 14:00:17' 2024-09-12T14:00:22.388 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster2 mirror image-primary7 up+unknown '2024-09-12 14:00:17' 2024-09-12T14:00:22.388 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T14:00:22.389 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:00:22.389 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T14:00:22.389 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:00:22.389 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 14:00:17' 2024-09-12T14:00:22.389 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:00:22.389 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:00:22.389 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2024-09-12T14:00:22.505 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041924231mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary1a862d41a-8d34-4c63-bea6-4195101bf11aup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:00:17image-primary103238adfd-24b7-48b2-ab32-66aaa1c7b583up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:00:17image-primary2fef52d8d-af21-4d3d-97e9-021eac35b36aup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:00:17image-primary3a8c53447-94fc-4fe3-b559-d953f6f94c54up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:00:17image-primary48face4e5-1ee3-427e-b5bc-7b05be9d6e87up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:00:17image-primary5fb69fceb-7a10-4ab7-ad45-ec4124a4621fup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:00:17image-primary6a63c238c-944e-44a1-be22-edc69436b860up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:00:17image-primary7700ac9cd-0dca-415e-b998-79da95582617up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:00:17image-primary83b4d0b11-5c89-4212-85d3-673c05d34c4bup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:00:17image-primary941a87723-a798-494d-b064-df92256304f2up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:00:17' 2024-09-12T14:00:22.505 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T14:00:22.506 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary7'\'']/last_update' 2024-09-12T14:00:22.510 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 14:00:17' 2024-09-12T14:00:22.510 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary7'\'']/state' 2024-09-12T14:00:22.514 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+unknown 2024-09-12T14:00:22.515 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T14:00:22.515 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+unknown 2024-09-12T14:00:22.517 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T14:00:22.518 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:00:22.518 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:00:22.518 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ promote_image cluster2 mirror image-primary7 2024-09-12T14:00:22.518 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T14:00:22.518 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:00:22.518 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T14:00:22.519 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local force= 2024-09-12T14:00:22.519 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 mirror image promote mirror/image-primary7 2024-09-12T14:00:22.687 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Image promoted to primary 2024-09-12T14:00:22.691 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Image promoted to primary 2024-09-12T14:00:22.702 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \n\b\d ]] 2024-09-12T14:00:22.702 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \k\r\b\d ]] 2024-09-12T14:00:22.703 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo rbd --cluster cluster2 device map -t krbd mirror/image-primary7 2024-09-12T14:00:22.706 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \n\b\d ]] 2024-09-12T14:00:22.706 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \k\r\b\d ]] 2024-09-12T14:00:22.706 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo rbd --cluster cluster2 device map -t krbd mirror/image-primary8 2024-09-12T14:00:22.836 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ dev=/dev/rbd1 2024-09-12T14:00:22.837 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo md5sum /dev/rbd1 2024-09-12T14:00:22.838 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ awk '{print $1}' 2024-09-12T14:00:22.865 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ dev=/dev/rbd2 2024-09-12T14:00:22.874 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo md5sum /dev/rbd2 2024-09-12T14:00:22.874 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ awk '{print $1}' 2024-09-12T14:00:24.142 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary6 up+unknown '' '' 2024-09-12T14:00:24.142 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:00:24.142 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:00:24.142 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T14:00:24.142 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:00:24.142 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:00:24.142 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:00:24.143 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:00:24.143 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:00:24.143 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary6 2024-09-12T14:00:24.179 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary10 up+unknown '' '' 2024-09-12T14:00:24.179 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:00:24.180 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:00:24.180 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T14:00:24.180 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:00:24.180 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:00:24.180 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:00:24.180 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:00:24.180 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:00:24.181 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary10 2024-09-12T14:00:24.188 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary2 up+unknown '' '' 2024-09-12T14:00:24.188 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:00:24.189 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:00:24.189 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T14:00:24.189 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:00:24.189 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:00:24.189 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:00:24.189 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:00:24.190 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:00:24.190 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary2 2024-09-12T14:00:24.258 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary6: 2024-09-12T14:00:24.258 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: a63c238c-944e-44a1-be22-edc69436b860 2024-09-12T14:00:24.258 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+unknown 2024-09-12T14:00:24.258 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: remote image is not primary 2024-09-12T14:00:24.258 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T14:00:24.258 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 14:00:17' 2024-09-12T14:00:24.258 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T14:00:24.261 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+unknown 2024-09-12T14:00:24.262 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T14:00:24.265 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: remote image is not primary 2024-09-12T14:00:24.266 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T14:00:24.267 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T14:00:24.267 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T14:00:24.270 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T14:00:24.270 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T14:00:24.273 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T14:00:24.273 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T14:00:24.274 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T14:00:24.279 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 14:00:17' 2024-09-12T14:00:24.279 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary6 up+unknown '2024-09-12 14:00:17' 2024-09-12T14:00:24.279 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:00:24.280 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:00:24.280 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T14:00:24.280 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:00:24.280 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 14:00:17' 2024-09-12T14:00:24.280 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:00:24.280 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:00:24.280 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-09-12T14:00:24.286 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary2: 2024-09-12T14:00:24.286 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: fef52d8d-af21-4d3d-97e9-021eac35b36a 2024-09-12T14:00:24.286 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+unknown 2024-09-12T14:00:24.286 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: remote image is not primary 2024-09-12T14:00:24.286 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T14:00:24.286 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 14:00:17' 2024-09-12T14:00:24.286 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T14:00:24.291 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+unknown 2024-09-12T14:00:24.292 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary10: 2024-09-12T14:00:24.292 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 3238adfd-24b7-48b2-ab32-66aaa1c7b583 2024-09-12T14:00:24.292 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+unknown 2024-09-12T14:00:24.292 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: remote image is not primary 2024-09-12T14:00:24.293 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T14:00:24.293 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 14:00:17' 2024-09-12T14:00:24.293 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T14:00:24.293 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T14:00:24.295 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+unknown 2024-09-12T14:00:24.295 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T14:00:24.295 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: remote image is not primary 2024-09-12T14:00:24.296 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T14:00:24.297 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T14:00:24.298 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T14:00:24.301 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: remote image is not primary 2024-09-12T14:00:24.301 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T14:00:24.301 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T14:00:24.301 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T14:00:24.303 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T14:00:24.303 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T14:00:24.305 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T14:00:24.305 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T14:00:24.306 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T14:00:24.307 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T14:00:24.308 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T14:00:24.308 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T14:00:24.309 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T14:00:24.310 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T14:00:24.315 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 14:00:17' 2024-09-12T14:00:24.315 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary2 up+unknown '2024-09-12 14:00:17' 2024-09-12T14:00:24.315 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:00:24.315 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:00:24.315 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T14:00:24.315 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:00:24.315 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 14:00:17' 2024-09-12T14:00:24.315 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:00:24.316 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:00:24.316 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-09-12T14:00:24.317 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 14:00:17' 2024-09-12T14:00:24.317 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary10 up+unknown '2024-09-12 14:00:17' 2024-09-12T14:00:24.317 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:00:24.317 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:00:24.317 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T14:00:24.317 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:00:24.317 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 14:00:17' 2024-09-12T14:00:24.317 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:00:24.320 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:00:24.320 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-09-12T14:00:24.412 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041934232mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary1a862d41a-8d34-4c63-bea6-4195101bf11aup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:00:17image-primary103238adfd-24b7-48b2-ab32-66aaa1c7b583up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:00:17image-primary2fef52d8d-af21-4d3d-97e9-021eac35b36aup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:00:17image-primary3a8c53447-94fc-4fe3-b559-d953f6f94c54up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:00:17image-primary48face4e5-1ee3-427e-b5bc-7b05be9d6e87up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:00:17image-primary5fb69fceb-7a10-4ab7-ad45-ec4124a4621fup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:00:17image-primary6a63c238c-944e-44a1-be22-edc69436b860up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:00:17image-primary7700ac9cd-0dca-415e-b998-79da95582617up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:00:17image-primary83b4d0b11-5c89-4212-85d3-673c05d34c4bup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:00:17image-primary941a87723-a798-494d-b064-df92256304f2up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:00:17' 2024-09-12T14:00:24.413 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T14:00:24.413 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary6'\'']/last_update' 2024-09-12T14:00:24.420 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 14:00:17' 2024-09-12T14:00:24.421 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary6'\'']/state' 2024-09-12T14:00:24.428 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+unknown 2024-09-12T14:00:24.430 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T14:00:24.430 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+unknown 2024-09-12T14:00:24.434 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T14:00:24.436 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:00:24.436 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:00:24.437 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster2 mirror image-primary6 up+unknown 2024-09-12T14:00:24.437 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T14:00:24.437 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:00:24.437 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T14:00:24.437 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:00:24.437 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:00:24.437 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:00:24.437 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:00:24.437 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T14:00:24.485 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041934232mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary1a862d41a-8d34-4c63-bea6-4195101bf11aup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:00:17image-primary103238adfd-24b7-48b2-ab32-66aaa1c7b583up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:00:17image-primary2fef52d8d-af21-4d3d-97e9-021eac35b36aup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:00:17image-primary3a8c53447-94fc-4fe3-b559-d953f6f94c54up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:00:17image-primary48face4e5-1ee3-427e-b5bc-7b05be9d6e87up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:00:17image-primary5fb69fceb-7a10-4ab7-ad45-ec4124a4621fup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:00:17image-primary6a63c238c-944e-44a1-be22-edc69436b860up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:00:17image-primary7700ac9cd-0dca-415e-b998-79da95582617up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:00:17image-primary83b4d0b11-5c89-4212-85d3-673c05d34c4bup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:00:17image-primary941a87723-a798-494d-b064-df92256304f2up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:00:17' 2024-09-12T14:00:24.485 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T14:00:24.486 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary10'\'']/last_update' 2024-09-12T14:00:24.490 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 14:00:17' 2024-09-12T14:00:24.491 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary10'\'']/state' 2024-09-12T14:00:24.494 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041934232mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary1a862d41a-8d34-4c63-bea6-4195101bf11aup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:00:17image-primary103238adfd-24b7-48b2-ab32-66aaa1c7b583up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:00:17image-primary2fef52d8d-af21-4d3d-97e9-021eac35b36aup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:00:17image-primary3a8c53447-94fc-4fe3-b559-d953f6f94c54up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:00:17image-primary48face4e5-1ee3-427e-b5bc-7b05be9d6e87up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:00:17image-primary5fb69fceb-7a10-4ab7-ad45-ec4124a4621fup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:00:17image-primary6a63c238c-944e-44a1-be22-edc69436b860up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:00:17image-primary7700ac9cd-0dca-415e-b998-79da95582617up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:00:17image-primary83b4d0b11-5c89-4212-85d3-673c05d34c4bup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:00:17image-primary941a87723-a798-494d-b064-df92256304f2up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:00:17' 2024-09-12T14:00:24.494 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T14:00:24.495 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary2'\'']/last_update' 2024-09-12T14:00:24.496 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+unknown 2024-09-12T14:00:24.496 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T14:00:24.497 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+unknown 2024-09-12T14:00:24.499 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 14:00:17' 2024-09-12T14:00:24.500 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary2'\'']/state' 2024-09-12T14:00:24.502 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T14:00:24.503 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:00:24.503 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:00:24.503 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster2 mirror image-primary10 up+unknown 2024-09-12T14:00:24.503 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T14:00:24.504 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:00:24.504 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T14:00:24.504 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:00:24.504 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:00:24.504 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:00:24.504 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:00:24.504 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T14:00:24.505 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+unknown 2024-09-12T14:00:24.505 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T14:00:24.506 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+unknown 2024-09-12T14:00:24.508 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T14:00:24.509 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:00:24.509 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:00:24.509 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster2 mirror image-primary2 up+unknown 2024-09-12T14:00:24.509 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T14:00:24.510 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:00:24.510 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T14:00:24.510 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:00:24.510 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:00:24.510 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:00:24.510 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:00:24.510 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T14:00:25.136 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary9 up+unknown '' '' 2024-09-12T14:00:25.136 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:00:25.136 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:00:25.136 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T14:00:25.136 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:00:25.136 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:00:25.136 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:00:25.137 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:00:25.138 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary5 up+unknown '' '' 2024-09-12T14:00:25.138 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:00:25.138 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:00:25.138 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T14:00:25.138 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:00:25.139 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:00:25.139 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:00:25.139 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:00:25.139 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:00:25.139 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary9 2024-09-12T14:00:25.139 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:00:25.139 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary5 2024-09-12T14:00:25.139 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary1 up+unknown '' '' 2024-09-12T14:00:25.139 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:00:25.139 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:00:25.139 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T14:00:25.139 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:00:25.139 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:00:25.139 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:00:25.140 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:00:25.140 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:00:25.140 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary1 2024-09-12T14:00:25.151 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary4 up+unknown '' '' 2024-09-12T14:00:25.151 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:00:25.151 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:00:25.151 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T14:00:25.151 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:00:25.152 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:00:25.152 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:00:25.152 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:00:25.152 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:00:25.153 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary4 2024-09-12T14:00:25.244 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary1: 2024-09-12T14:00:25.245 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: a862d41a-8d34-4c63-bea6-4195101bf11a 2024-09-12T14:00:25.245 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+unknown 2024-09-12T14:00:25.245 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: remote image is not primary 2024-09-12T14:00:25.245 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T14:00:25.245 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 14:00:17' 2024-09-12T14:00:25.245 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T14:00:25.247 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+unknown 2024-09-12T14:00:25.248 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T14:00:25.248 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary9: 2024-09-12T14:00:25.248 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 41a87723-a798-494d-b064-df92256304f2 2024-09-12T14:00:25.248 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+unknown 2024-09-12T14:00:25.248 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: remote image is not primary 2024-09-12T14:00:25.248 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T14:00:25.249 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 14:00:17' 2024-09-12T14:00:25.249 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T14:00:25.249 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary5: 2024-09-12T14:00:25.249 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: fb69fceb-7a10-4ab7-ad45-ec4124a4621f 2024-09-12T14:00:25.250 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+unknown 2024-09-12T14:00:25.250 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: remote image is not primary 2024-09-12T14:00:25.250 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T14:00:25.250 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 14:00:17' 2024-09-12T14:00:25.250 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T14:00:25.251 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+unknown 2024-09-12T14:00:25.252 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T14:00:25.253 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: remote image is not primary 2024-09-12T14:00:25.254 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+unknown 2024-09-12T14:00:25.254 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T14:00:25.254 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T14:00:25.256 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: remote image is not primary 2024-09-12T14:00:25.256 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T14:00:25.256 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T14:00:25.256 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T14:00:25.258 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T14:00:25.258 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T14:00:25.258 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: remote image is not primary 2024-09-12T14:00:25.259 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T14:00:25.259 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T14:00:25.260 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T14:00:25.260 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T14:00:25.260 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T14:00:25.261 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T14:00:25.262 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary4: 2024-09-12T14:00:25.262 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 8face4e5-1ee3-427e-b5bc-7b05be9d6e87 2024-09-12T14:00:25.262 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+unknown 2024-09-12T14:00:25.262 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: remote image is not primary 2024-09-12T14:00:25.262 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T14:00:25.262 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 14:00:17' 2024-09-12T14:00:25.262 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T14:00:25.262 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T14:00:25.263 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T14:00:25.264 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T14:00:25.264 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T14:00:25.265 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T14:00:25.266 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T14:00:25.266 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+unknown 2024-09-12T14:00:25.266 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T14:00:25.266 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T14:00:25.268 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T14:00:25.268 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T14:00:25.268 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T14:00:25.268 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T14:00:25.270 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T14:00:25.271 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: remote image is not primary 2024-09-12T14:00:25.271 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T14:00:25.273 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 14:00:17' 2024-09-12T14:00:25.273 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary1 up+unknown '2024-09-12 14:00:17' 2024-09-12T14:00:25.273 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:00:25.273 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:00:25.273 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T14:00:25.273 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:00:25.273 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 14:00:17' 2024-09-12T14:00:25.273 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:00:25.273 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T14:00:25.274 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T14:00:25.274 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:00:25.274 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-09-12T14:00:25.275 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 14:00:17' 2024-09-12T14:00:25.275 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary9 up+unknown '2024-09-12 14:00:17' 2024-09-12T14:00:25.275 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:00:25.275 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:00:25.275 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T14:00:25.275 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:00:25.275 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 14:00:17' 2024-09-12T14:00:25.276 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:00:25.276 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 14:00:17' 2024-09-12T14:00:25.276 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary5 up+unknown '2024-09-12 14:00:17' 2024-09-12T14:00:25.276 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:00:25.276 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:00:25.276 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T14:00:25.276 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:00:25.276 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 14:00:17' 2024-09-12T14:00:25.276 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:00:25.277 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:00:25.277 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-09-12T14:00:25.277 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:00:25.277 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-09-12T14:00:25.277 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T14:00:25.278 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T14:00:25.281 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T14:00:25.283 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T14:00:25.283 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T14:00:25.290 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 14:00:17' 2024-09-12T14:00:25.290 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary4 up+unknown '2024-09-12 14:00:17' 2024-09-12T14:00:25.290 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:00:25.290 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:00:25.290 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T14:00:25.291 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:00:25.291 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 14:00:17' 2024-09-12T14:00:25.291 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:00:25.291 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:00:25.291 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-09-12T14:00:25.438 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041934232mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary1a862d41a-8d34-4c63-bea6-4195101bf11aup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:00:17image-primary103238adfd-24b7-48b2-ab32-66aaa1c7b583up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:00:17image-primary2fef52d8d-af21-4d3d-97e9-021eac35b36aup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:00:17image-primary3a8c53447-94fc-4fe3-b559-d953f6f94c54up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:00:17image-primary48face4e5-1ee3-427e-b5bc-7b05be9d6e87up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:00:17image-primary5fb69fceb-7a10-4ab7-ad45-ec4124a4621fup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:00:17image-primary6a63c238c-944e-44a1-be22-edc69436b860up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:00:17image-primary7700ac9cd-0dca-415e-b998-79da95582617up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:00:17image-primary83b4d0b11-5c89-4212-85d3-673c05d34c4bup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:00:17image-primary941a87723-a798-494d-b064-df92256304f2up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:00:17' 2024-09-12T14:00:25.438 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T14:00:25.438 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary9'\'']/last_update' 2024-09-12T14:00:25.440 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster2 mirror image-primary6 up+unknown '' '' 2024-09-12T14:00:25.440 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T14:00:25.440 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:00:25.440 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T14:00:25.440 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:00:25.440 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:00:25.440 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:00:25.441 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:00:25.441 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:00:25.441 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster2 mirror image status mirror/image-primary6 2024-09-12T14:00:25.443 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 14:00:17' 2024-09-12T14:00:25.444 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary9'\'']/state' 2024-09-12T14:00:25.449 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+unknown 2024-09-12T14:00:25.450 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T14:00:25.451 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+unknown 2024-09-12T14:00:25.454 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T14:00:25.455 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:00:25.455 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:00:25.455 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster2 mirror image-primary9 up+unknown 2024-09-12T14:00:25.455 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T14:00:25.455 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:00:25.455 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T14:00:25.455 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:00:25.455 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:00:25.455 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:00:25.456 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:00:25.456 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T14:00:25.462 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041934232mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary1a862d41a-8d34-4c63-bea6-4195101bf11aup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:00:17image-primary103238adfd-24b7-48b2-ab32-66aaa1c7b583up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:00:17image-primary2fef52d8d-af21-4d3d-97e9-021eac35b36aup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:00:17image-primary3a8c53447-94fc-4fe3-b559-d953f6f94c54up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:00:17image-primary48face4e5-1ee3-427e-b5bc-7b05be9d6e87up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:00:17image-primary5fb69fceb-7a10-4ab7-ad45-ec4124a4621fup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:00:17image-primary6a63c238c-944e-44a1-be22-edc69436b860up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:00:17image-primary7700ac9cd-0dca-415e-b998-79da95582617up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:00:17image-primary83b4d0b11-5c89-4212-85d3-673c05d34c4bup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:00:17image-primary941a87723-a798-494d-b064-df92256304f2up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:00:17' 2024-09-12T14:00:25.462 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T14:00:25.462 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary5'\'']/last_update' 2024-09-12T14:00:25.467 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 14:00:17' 2024-09-12T14:00:25.468 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary5'\'']/state' 2024-09-12T14:00:25.473 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+unknown 2024-09-12T14:00:25.474 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T14:00:25.474 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+unknown 2024-09-12T14:00:25.477 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T14:00:25.478 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:00:25.479 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:00:25.479 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster2 mirror image-primary5 up+unknown 2024-09-12T14:00:25.479 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T14:00:25.479 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:00:25.479 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T14:00:25.479 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:00:25.479 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:00:25.479 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:00:25.479 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:00:25.479 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T14:00:25.486 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041934232mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary1a862d41a-8d34-4c63-bea6-4195101bf11aup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:00:17image-primary103238adfd-24b7-48b2-ab32-66aaa1c7b583up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:00:17image-primary2fef52d8d-af21-4d3d-97e9-021eac35b36aup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:00:17image-primary3a8c53447-94fc-4fe3-b559-d953f6f94c54up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:00:17image-primary48face4e5-1ee3-427e-b5bc-7b05be9d6e87up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:00:17image-primary5fb69fceb-7a10-4ab7-ad45-ec4124a4621fup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:00:17image-primary6a63c238c-944e-44a1-be22-edc69436b860up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:00:17image-primary7700ac9cd-0dca-415e-b998-79da95582617up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:00:17image-primary83b4d0b11-5c89-4212-85d3-673c05d34c4bup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:00:17image-primary941a87723-a798-494d-b064-df92256304f2up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:00:17' 2024-09-12T14:00:25.486 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T14:00:25.487 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary1'\'']/last_update' 2024-09-12T14:00:25.491 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 14:00:17' 2024-09-12T14:00:25.492 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary1'\'']/state' 2024-09-12T14:00:25.495 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041934232mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary1a862d41a-8d34-4c63-bea6-4195101bf11aup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:00:17image-primary103238adfd-24b7-48b2-ab32-66aaa1c7b583up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:00:17image-primary2fef52d8d-af21-4d3d-97e9-021eac35b36aup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:00:17image-primary3a8c53447-94fc-4fe3-b559-d953f6f94c54up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:00:17image-primary48face4e5-1ee3-427e-b5bc-7b05be9d6e87up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:00:17image-primary5fb69fceb-7a10-4ab7-ad45-ec4124a4621fup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:00:17image-primary6a63c238c-944e-44a1-be22-edc69436b860up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:00:17image-primary7700ac9cd-0dca-415e-b998-79da95582617up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:00:17image-primary83b4d0b11-5c89-4212-85d3-673c05d34c4bup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:00:17image-primary941a87723-a798-494d-b064-df92256304f2up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:00:17' 2024-09-12T14:00:25.495 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T14:00:25.496 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary4'\'']/last_update' 2024-09-12T14:00:25.496 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+unknown 2024-09-12T14:00:25.497 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T14:00:25.498 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+unknown 2024-09-12T14:00:25.504 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 14:00:17' 2024-09-12T14:00:25.504 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T14:00:25.506 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:00:25.506 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:00:25.506 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary4'\'']/state' 2024-09-12T14:00:25.506 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster2 mirror image-primary1 up+unknown 2024-09-12T14:00:25.506 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T14:00:25.506 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:00:25.506 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T14:00:25.506 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:00:25.506 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:00:25.506 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:00:25.507 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:00:25.507 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T14:00:25.507 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster2 mirror image-primary10 up+unknown '' '' 2024-09-12T14:00:25.507 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T14:00:25.507 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:00:25.507 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T14:00:25.507 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:00:25.507 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:00:25.507 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:00:25.507 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:00:25.508 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:00:25.508 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster2 mirror image status mirror/image-primary10 2024-09-12T14:00:25.508 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+unknown 2024-09-12T14:00:25.509 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T14:00:25.510 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+unknown 2024-09-12T14:00:25.512 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster2 mirror image-primary2 up+unknown '' '' 2024-09-12T14:00:25.512 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T14:00:25.512 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:00:25.512 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T14:00:25.512 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:00:25.512 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:00:25.512 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:00:25.512 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:00:25.513 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:00:25.513 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster2 mirror image status mirror/image-primary2 2024-09-12T14:00:25.514 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T14:00:25.515 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:00:25.515 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:00:25.515 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster2 mirror image-primary4 up+unknown 2024-09-12T14:00:25.515 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T14:00:25.515 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:00:25.515 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T14:00:25.515 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:00:25.516 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:00:25.516 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:00:25.516 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:00:25.516 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T14:00:25.555 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary6: 2024-09-12T14:00:25.555 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: a63c238c-944e-44a1-be22-edc69436b860 2024-09-12T14:00:25.556 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+unknown 2024-09-12T14:00:25.556 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: remote image is not primary 2024-09-12T14:00:25.556 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T14:00:25.556 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 14:00:17' 2024-09-12T14:00:25.556 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T14:00:25.560 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+unknown 2024-09-12T14:00:25.560 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T14:00:25.565 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: remote image is not primary 2024-09-12T14:00:25.566 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T14:00:25.567 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T14:00:25.567 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T14:00:25.571 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T14:00:25.572 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T14:00:25.577 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T14:00:25.578 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T14:00:25.579 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T14:00:25.585 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 14:00:17' 2024-09-12T14:00:25.586 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster2 mirror image-primary6 up+unknown '2024-09-12 14:00:17' 2024-09-12T14:00:25.586 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T14:00:25.586 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:00:25.586 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T14:00:25.586 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:00:25.586 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 14:00:17' 2024-09-12T14:00:25.586 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:00:25.587 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:00:25.587 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2024-09-12T14:00:25.605 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary10: 2024-09-12T14:00:25.605 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 3238adfd-24b7-48b2-ab32-66aaa1c7b583 2024-09-12T14:00:25.606 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+unknown 2024-09-12T14:00:25.606 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: remote image is not primary 2024-09-12T14:00:25.606 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T14:00:25.606 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 14:00:17' 2024-09-12T14:00:25.606 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T14:00:25.608 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+unknown 2024-09-12T14:00:25.609 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T14:00:25.613 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: remote image is not primary 2024-09-12T14:00:25.614 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T14:00:25.615 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T14:00:25.615 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T14:00:25.618 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T14:00:25.618 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T14:00:25.623 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T14:00:25.623 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T14:00:25.625 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T14:00:25.629 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary2: 2024-09-12T14:00:25.630 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: fef52d8d-af21-4d3d-97e9-021eac35b36a 2024-09-12T14:00:25.630 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+unknown 2024-09-12T14:00:25.630 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: remote image is not primary 2024-09-12T14:00:25.630 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T14:00:25.630 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 14:00:17' 2024-09-12T14:00:25.630 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T14:00:25.631 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 14:00:17' 2024-09-12T14:00:25.631 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster2 mirror image-primary10 up+unknown '2024-09-12 14:00:17' 2024-09-12T14:00:25.631 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T14:00:25.631 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:00:25.631 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T14:00:25.631 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:00:25.631 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 14:00:17' 2024-09-12T14:00:25.632 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:00:25.632 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:00:25.632 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2024-09-12T14:00:25.632 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+unknown 2024-09-12T14:00:25.633 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T14:00:25.638 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: remote image is not primary 2024-09-12T14:00:25.639 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T14:00:25.640 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T14:00:25.640 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T14:00:25.643 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T14:00:25.644 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T14:00:25.648 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T14:00:25.649 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T14:00:25.650 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T14:00:25.656 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 14:00:17' 2024-09-12T14:00:25.656 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster2 mirror image-primary2 up+unknown '2024-09-12 14:00:17' 2024-09-12T14:00:25.656 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T14:00:25.656 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:00:25.657 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T14:00:25.657 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:00:25.657 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 14:00:17' 2024-09-12T14:00:25.657 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:00:25.657 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:00:25.657 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2024-09-12T14:00:25.781 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041924231mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary1a862d41a-8d34-4c63-bea6-4195101bf11aup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:00:17image-primary103238adfd-24b7-48b2-ab32-66aaa1c7b583up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:00:17image-primary2fef52d8d-af21-4d3d-97e9-021eac35b36aup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:00:17image-primary3a8c53447-94fc-4fe3-b559-d953f6f94c54up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:00:17image-primary48face4e5-1ee3-427e-b5bc-7b05be9d6e87up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:00:17image-primary5fb69fceb-7a10-4ab7-ad45-ec4124a4621fup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:00:17image-primary6a63c238c-944e-44a1-be22-edc69436b860up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:00:17image-primary7700ac9cd-0dca-415e-b998-79da95582617up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:00:17image-primary83b4d0b11-5c89-4212-85d3-673c05d34c4bup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:00:17image-primary941a87723-a798-494d-b064-df92256304f2up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:00:17' 2024-09-12T14:00:25.782 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T14:00:25.782 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary6'\'']/last_update' 2024-09-12T14:00:25.794 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 14:00:17' 2024-09-12T14:00:25.796 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary6'\'']/state' 2024-09-12T14:00:25.807 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+unknown 2024-09-12T14:00:25.809 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T14:00:25.809 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+unknown 2024-09-12T14:00:25.817 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T14:00:25.818 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:00:25.818 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:00:25.819 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ promote_image cluster2 mirror image-primary6 2024-09-12T14:00:25.820 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T14:00:25.820 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:00:25.820 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T14:00:25.820 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local force= 2024-09-12T14:00:25.820 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 mirror image promote mirror/image-primary6 2024-09-12T14:00:25.845 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041924231mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary1a862d41a-8d34-4c63-bea6-4195101bf11aup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:00:17image-primary103238adfd-24b7-48b2-ab32-66aaa1c7b583up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:00:17image-primary2fef52d8d-af21-4d3d-97e9-021eac35b36aup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:00:17image-primary3a8c53447-94fc-4fe3-b559-d953f6f94c54up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:00:17image-primary48face4e5-1ee3-427e-b5bc-7b05be9d6e87up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:00:17image-primary5fb69fceb-7a10-4ab7-ad45-ec4124a4621fup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:00:17image-primary6a63c238c-944e-44a1-be22-edc69436b860up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:00:17image-primary7700ac9cd-0dca-415e-b998-79da95582617up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:00:17image-primary83b4d0b11-5c89-4212-85d3-673c05d34c4bup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:00:17image-primary941a87723-a798-494d-b064-df92256304f2up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:00:17' 2024-09-12T14:00:25.845 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T14:00:25.846 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary10'\'']/last_update' 2024-09-12T14:00:25.857 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 14:00:17' 2024-09-12T14:00:25.859 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary10'\'']/state' 2024-09-12T14:00:25.868 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+unknown 2024-09-12T14:00:25.868 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T14:00:25.869 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+unknown 2024-09-12T14:00:25.874 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T14:00:25.875 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:00:25.875 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:00:25.875 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ promote_image cluster2 mirror image-primary10 2024-09-12T14:00:25.875 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T14:00:25.875 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:00:25.875 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T14:00:25.875 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local force= 2024-09-12T14:00:25.875 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 mirror image promote mirror/image-primary10 2024-09-12T14:00:25.894 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041924231mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary1a862d41a-8d34-4c63-bea6-4195101bf11aup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:00:17image-primary103238adfd-24b7-48b2-ab32-66aaa1c7b583up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:00:17image-primary2fef52d8d-af21-4d3d-97e9-021eac35b36aup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:00:17image-primary3a8c53447-94fc-4fe3-b559-d953f6f94c54up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:00:17image-primary48face4e5-1ee3-427e-b5bc-7b05be9d6e87up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:00:17image-primary5fb69fceb-7a10-4ab7-ad45-ec4124a4621fup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:00:17image-primary6a63c238c-944e-44a1-be22-edc69436b860up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:00:17image-primary7700ac9cd-0dca-415e-b998-79da95582617up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:00:17image-primary83b4d0b11-5c89-4212-85d3-673c05d34c4bup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:00:17image-primary941a87723-a798-494d-b064-df92256304f2up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:00:17' 2024-09-12T14:00:25.894 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T14:00:25.895 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary2'\'']/last_update' 2024-09-12T14:00:25.902 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 14:00:17' 2024-09-12T14:00:25.904 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary2'\'']/state' 2024-09-12T14:00:25.910 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+unknown 2024-09-12T14:00:25.911 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T14:00:25.912 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+unknown 2024-09-12T14:00:25.915 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T14:00:25.917 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:00:25.917 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:00:25.917 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ promote_image cluster2 mirror image-primary2 2024-09-12T14:00:25.917 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T14:00:25.917 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:00:25.917 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T14:00:25.917 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local force= 2024-09-12T14:00:25.917 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 mirror image promote mirror/image-primary2 2024-09-12T14:00:26.458 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster2 mirror image-primary9 up+unknown '' '' 2024-09-12T14:00:26.458 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T14:00:26.459 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:00:26.459 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T14:00:26.459 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:00:26.459 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:00:26.459 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:00:26.459 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:00:26.459 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:00:26.459 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster2 mirror image status mirror/image-primary9 2024-09-12T14:00:26.482 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster2 mirror image-primary5 up+unknown '' '' 2024-09-12T14:00:26.482 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T14:00:26.482 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:00:26.482 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T14:00:26.482 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:00:26.482 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:00:26.483 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:00:26.483 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:00:26.483 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:00:26.483 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster2 mirror image status mirror/image-primary5 2024-09-12T14:00:26.506 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster2 mirror image-primary1 up+unknown '' '' 2024-09-12T14:00:26.506 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T14:00:26.506 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:00:26.506 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T14:00:26.506 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:00:26.506 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:00:26.506 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:00:26.506 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:00:26.507 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:00:26.507 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster2 mirror image status mirror/image-primary1 2024-09-12T14:00:26.519 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster2 mirror image-primary4 up+unknown '' '' 2024-09-12T14:00:26.519 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T14:00:26.519 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:00:26.519 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T14:00:26.520 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:00:26.520 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:00:26.520 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:00:26.520 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:00:26.520 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:00:26.520 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster2 mirror image status mirror/image-primary4 2024-09-12T14:00:26.562 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary9: 2024-09-12T14:00:26.562 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 41a87723-a798-494d-b064-df92256304f2 2024-09-12T14:00:26.562 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+unknown 2024-09-12T14:00:26.562 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: remote image is not primary 2024-09-12T14:00:26.562 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T14:00:26.562 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 14:00:17' 2024-09-12T14:00:26.562 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T14:00:26.566 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+unknown 2024-09-12T14:00:26.566 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T14:00:26.572 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: remote image is not primary 2024-09-12T14:00:26.572 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T14:00:26.573 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T14:00:26.575 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T14:00:26.578 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T14:00:26.579 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T14:00:26.585 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T14:00:26.585 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T14:00:26.587 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T14:00:26.594 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 14:00:17' 2024-09-12T14:00:26.594 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster2 mirror image-primary9 up+unknown '2024-09-12 14:00:17' 2024-09-12T14:00:26.594 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T14:00:26.595 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:00:26.595 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T14:00:26.595 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:00:26.595 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 14:00:17' 2024-09-12T14:00:26.596 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:00:26.597 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:00:26.597 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2024-09-12T14:00:26.607 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary5: 2024-09-12T14:00:26.608 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: fb69fceb-7a10-4ab7-ad45-ec4124a4621f 2024-09-12T14:00:26.608 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+unknown 2024-09-12T14:00:26.608 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: remote image is not primary 2024-09-12T14:00:26.608 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T14:00:26.608 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 14:00:17' 2024-09-12T14:00:26.608 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T14:00:26.612 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+unknown 2024-09-12T14:00:26.612 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T14:00:26.617 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: remote image is not primary 2024-09-12T14:00:26.618 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T14:00:26.620 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T14:00:26.620 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T14:00:26.622 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary1: 2024-09-12T14:00:26.622 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: a862d41a-8d34-4c63-bea6-4195101bf11a 2024-09-12T14:00:26.622 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+unknown 2024-09-12T14:00:26.622 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: remote image is not primary 2024-09-12T14:00:26.622 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T14:00:26.622 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 14:00:17' 2024-09-12T14:00:26.622 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T14:00:26.624 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T14:00:26.625 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T14:00:26.626 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+unknown 2024-09-12T14:00:26.627 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T14:00:26.629 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T14:00:26.630 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T14:00:26.633 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T14:00:26.633 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: remote image is not primary 2024-09-12T14:00:26.634 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T14:00:26.635 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T14:00:26.636 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T14:00:26.640 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T14:00:26.640 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 14:00:17' 2024-09-12T14:00:26.640 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster2 mirror image-primary5 up+unknown '2024-09-12 14:00:17' 2024-09-12T14:00:26.640 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T14:00:26.640 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:00:26.640 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T14:00:26.640 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:00:26.640 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 14:00:17' 2024-09-12T14:00:26.641 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:00:26.641 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T14:00:26.641 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:00:26.641 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2024-09-12T14:00:26.645 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T14:00:26.645 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary4: 2024-09-12T14:00:26.645 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 8face4e5-1ee3-427e-b5bc-7b05be9d6e87 2024-09-12T14:00:26.645 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+unknown 2024-09-12T14:00:26.645 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: remote image is not primary 2024-09-12T14:00:26.645 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T14:00:26.646 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 14:00:17' 2024-09-12T14:00:26.646 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T14:00:26.646 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T14:00:26.646 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T14:00:26.649 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+unknown 2024-09-12T14:00:26.650 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T14:00:26.652 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 14:00:17' 2024-09-12T14:00:26.652 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster2 mirror image-primary1 up+unknown '2024-09-12 14:00:17' 2024-09-12T14:00:26.652 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T14:00:26.652 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:00:26.652 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T14:00:26.652 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:00:26.652 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 14:00:17' 2024-09-12T14:00:26.652 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:00:26.653 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:00:26.653 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2024-09-12T14:00:26.653 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: remote image is not primary 2024-09-12T14:00:26.654 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T14:00:26.655 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T14:00:26.655 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T14:00:26.658 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T14:00:26.658 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T14:00:26.663 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T14:00:26.663 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T14:00:26.664 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T14:00:26.670 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 14:00:17' 2024-09-12T14:00:26.671 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster2 mirror image-primary4 up+unknown '2024-09-12 14:00:17' 2024-09-12T14:00:26.671 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T14:00:26.671 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:00:26.671 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T14:00:26.671 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:00:26.671 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 14:00:17' 2024-09-12T14:00:26.671 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:00:26.671 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:00:26.672 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2024-09-12T14:00:26.739 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Image promoted to primary 2024-09-12T14:00:26.751 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Image promoted to primary 2024-09-12T14:00:26.755 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Image promoted to primary 2024-09-12T14:00:26.787 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \n\b\d ]] 2024-09-12T14:00:26.788 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \k\r\b\d ]] 2024-09-12T14:00:26.792 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo rbd --cluster cluster2 device map -t krbd mirror/image-primary10 2024-09-12T14:00:26.803 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \n\b\d ]] 2024-09-12T14:00:26.805 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \k\r\b\d ]] 2024-09-12T14:00:26.806 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo rbd --cluster cluster2 device map -t krbd mirror/image-primary2 2024-09-12T14:00:26.812 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \n\b\d ]] 2024-09-12T14:00:26.812 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \k\r\b\d ]] 2024-09-12T14:00:26.815 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo rbd --cluster cluster2 device map -t krbd mirror/image-primary6 2024-09-12T14:00:26.907 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041924231mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary1a862d41a-8d34-4c63-bea6-4195101bf11aup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:00:17image-primary103238adfd-24b7-48b2-ab32-66aaa1c7b583up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:00:17image-primary2fef52d8d-af21-4d3d-97e9-021eac35b36aup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:00:17image-primary3a8c53447-94fc-4fe3-b559-d953f6f94c54up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:00:17image-primary48face4e5-1ee3-427e-b5bc-7b05be9d6e87up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:00:17image-primary5fb69fceb-7a10-4ab7-ad45-ec4124a4621fup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:00:17image-primary6a63c238c-944e-44a1-be22-edc69436b860up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:00:17image-primary7700ac9cd-0dca-415e-b998-79da95582617up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:00:17image-primary83b4d0b11-5c89-4212-85d3-673c05d34c4bup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:00:17image-primary941a87723-a798-494d-b064-df92256304f2up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:00:17' 2024-09-12T14:00:26.908 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T14:00:26.909 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary9'\'']/last_update' 2024-09-12T14:00:26.938 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 14:00:17' 2024-09-12T14:00:26.940 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary9'\'']/state' 2024-09-12T14:00:26.969 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+unknown 2024-09-12T14:00:26.973 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T14:00:26.974 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+unknown 2024-09-12T14:00:26.981 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T14:00:26.984 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:00:26.984 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:00:26.984 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ promote_image cluster2 mirror image-primary9 2024-09-12T14:00:26.984 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T14:00:26.984 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:00:26.985 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T14:00:26.985 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local force= 2024-09-12T14:00:26.985 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 mirror image promote mirror/image-primary9 2024-09-12T14:00:27.024 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041924231mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary1a862d41a-8d34-4c63-bea6-4195101bf11aup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:00:17image-primary103238adfd-24b7-48b2-ab32-66aaa1c7b583up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:00:17image-primary2fef52d8d-af21-4d3d-97e9-021eac35b36aup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:00:17image-primary3a8c53447-94fc-4fe3-b559-d953f6f94c54up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:00:17image-primary48face4e5-1ee3-427e-b5bc-7b05be9d6e87up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:00:17image-primary5fb69fceb-7a10-4ab7-ad45-ec4124a4621fup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:00:17image-primary6a63c238c-944e-44a1-be22-edc69436b860up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:00:17image-primary7700ac9cd-0dca-415e-b998-79da95582617up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:00:17image-primary83b4d0b11-5c89-4212-85d3-673c05d34c4bup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:00:17image-primary941a87723-a798-494d-b064-df92256304f2up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:00:17' 2024-09-12T14:00:27.025 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T14:00:27.026 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary1'\'']/last_update' 2024-09-12T14:00:27.032 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041924231mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary1a862d41a-8d34-4c63-bea6-4195101bf11aup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:00:17image-primary103238adfd-24b7-48b2-ab32-66aaa1c7b583up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:00:17image-primary2fef52d8d-af21-4d3d-97e9-021eac35b36aup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:00:17image-primary3a8c53447-94fc-4fe3-b559-d953f6f94c54up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:00:17image-primary48face4e5-1ee3-427e-b5bc-7b05be9d6e87up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:00:17image-primary5fb69fceb-7a10-4ab7-ad45-ec4124a4621fup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:00:17image-primary6a63c238c-944e-44a1-be22-edc69436b860up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:00:17image-primary7700ac9cd-0dca-415e-b998-79da95582617up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:00:17image-primary83b4d0b11-5c89-4212-85d3-673c05d34c4bup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:00:17image-primary941a87723-a798-494d-b064-df92256304f2up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:00:17' 2024-09-12T14:00:27.032 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T14:00:27.033 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary5'\'']/last_update' 2024-09-12T14:00:27.038 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 14:00:17' 2024-09-12T14:00:27.040 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041924231mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary1a862d41a-8d34-4c63-bea6-4195101bf11aup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:00:17image-primary103238adfd-24b7-48b2-ab32-66aaa1c7b583up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:00:17image-primary2fef52d8d-af21-4d3d-97e9-021eac35b36aup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:00:17image-primary3a8c53447-94fc-4fe3-b559-d953f6f94c54up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:00:17image-primary48face4e5-1ee3-427e-b5bc-7b05be9d6e87up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:00:17image-primary5fb69fceb-7a10-4ab7-ad45-ec4124a4621fup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:00:17image-primary6a63c238c-944e-44a1-be22-edc69436b860up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:00:17image-primary7700ac9cd-0dca-415e-b998-79da95582617up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:00:17image-primary83b4d0b11-5c89-4212-85d3-673c05d34c4bup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:00:17image-primary941a87723-a798-494d-b064-df92256304f2up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:00:17' 2024-09-12T14:00:27.040 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T14:00:27.040 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary1'\'']/state' 2024-09-12T14:00:27.041 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary4'\'']/last_update' 2024-09-12T14:00:27.042 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 14:00:17' 2024-09-12T14:00:27.043 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary5'\'']/state' 2024-09-12T14:00:27.048 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 14:00:17' 2024-09-12T14:00:27.050 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary4'\'']/state' 2024-09-12T14:00:27.054 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+unknown 2024-09-12T14:00:27.055 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T14:00:27.056 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+unknown 2024-09-12T14:00:27.059 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+unknown 2024-09-12T14:00:27.060 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T14:00:27.062 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T14:00:27.062 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:00:27.062 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:00:27.062 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ promote_image cluster2 mirror image-primary5 2024-09-12T14:00:27.062 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T14:00:27.062 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+unknown 2024-09-12T14:00:27.062 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:00:27.062 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T14:00:27.062 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local force= 2024-09-12T14:00:27.062 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 mirror image promote mirror/image-primary5 2024-09-12T14:00:27.065 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T14:00:27.069 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:00:27.069 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:00:27.069 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ promote_image cluster2 mirror image-primary1 2024-09-12T14:00:27.069 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T14:00:27.069 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:00:27.069 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T14:00:27.070 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local force= 2024-09-12T14:00:27.070 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 mirror image promote mirror/image-primary1 2024-09-12T14:00:27.073 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+unknown 2024-09-12T14:00:27.074 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T14:00:27.076 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+unknown 2024-09-12T14:00:27.081 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T14:00:27.082 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:00:27.082 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:00:27.083 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ promote_image cluster2 mirror image-primary4 2024-09-12T14:00:27.083 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T14:00:27.083 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:00:27.083 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T14:00:27.083 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local force= 2024-09-12T14:00:27.083 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 mirror image promote mirror/image-primary4 2024-09-12T14:00:27.211 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ dev=/dev/rbd3 2024-09-12T14:00:27.221 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ awk '{print $1}' 2024-09-12T14:00:27.222 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo md5sum /dev/rbd3 2024-09-12T14:00:27.227 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ dev=/dev/rbd4 2024-09-12T14:00:27.230 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo md5sum /dev/rbd4 2024-09-12T14:00:27.231 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ awk '{print $1}' 2024-09-12T14:00:27.257 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ dev=/dev/rbd5 2024-09-12T14:00:27.259 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo md5sum /dev/rbd5 2024-09-12T14:00:27.259 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ awk '{print $1}' 2024-09-12T14:00:27.749 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Image promoted to primary 2024-09-12T14:00:27.758 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Image promoted to primary 2024-09-12T14:00:27.765 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Image promoted to primary 2024-09-12T14:00:27.766 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Image promoted to primary 2024-09-12T14:00:27.802 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \n\b\d ]] 2024-09-12T14:00:27.802 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \k\r\b\d ]] 2024-09-12T14:00:27.804 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo rbd --cluster cluster2 device map -t krbd mirror/image-primary4 2024-09-12T14:00:27.806 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \n\b\d ]] 2024-09-12T14:00:27.807 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \k\r\b\d ]] 2024-09-12T14:00:27.810 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo rbd --cluster cluster2 device map -t krbd mirror/image-primary9 2024-09-12T14:00:27.822 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \n\b\d ]] 2024-09-12T14:00:27.822 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \k\r\b\d ]] 2024-09-12T14:00:27.824 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo rbd --cluster cluster2 device map -t krbd mirror/image-primary5 2024-09-12T14:00:27.843 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \n\b\d ]] 2024-09-12T14:00:27.843 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \k\r\b\d ]] 2024-09-12T14:00:27.846 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo rbd --cluster cluster2 device map -t krbd mirror/image-primary1 2024-09-12T14:00:28.228 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ dev=/dev/rbd6 2024-09-12T14:00:28.228 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ dev=/dev/rbd7 2024-09-12T14:00:28.231 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo md5sum /dev/rbd6 2024-09-12T14:00:28.232 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ awk '{print $1}' 2024-09-12T14:00:28.232 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ awk '{print $1}' 2024-09-12T14:00:28.233 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo md5sum /dev/rbd7 2024-09-12T14:00:28.290 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ dev=/dev/rbd8 2024-09-12T14:00:28.292 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo md5sum /dev/rbd8 2024-09-12T14:00:28.293 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ awk '{print $1}' 2024-09-12T14:00:28.316 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ dev=/dev/rbd9 2024-09-12T14:00:28.326 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ awk '{print $1}' 2024-09-12T14:00:28.326 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo md5sum /dev/rbd9 2024-09-12T14:01:12.615 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ promote_md5=e1b3a544e30e3055ba282d8ae18b851f 2024-09-12T14:01:12.616 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo rbd --cluster cluster2 device unmap -t krbd /dev/rbd0 2024-09-12T14:01:12.969 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ e1b3a544e30e3055ba282d8ae18b851f != \e\1\b\3\a\5\4\4\e\3\0\e\3\0\5\5\b\a\2\8\2\d\8\a\e\1\8\b\8\5\1\f ]] 2024-09-12T14:01:16.467 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ promote_md5=52053d6300d10447696f51f977e345de 2024-09-12T14:01:16.467 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo rbd --cluster cluster2 device unmap -t krbd /dev/rbd2 2024-09-12T14:01:16.666 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ promote_md5=b0d008a8a9853471a66e05226b789b0d 2024-09-12T14:01:16.666 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo rbd --cluster cluster2 device unmap -t krbd /dev/rbd1 2024-09-12T14:01:16.726 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ 52053d6300d10447696f51f977e345de != \5\2\0\5\3\d\6\3\0\0\d\1\0\4\4\7\6\9\6\f\5\1\f\9\7\7\e\3\4\5\d\e ]] 2024-09-12T14:01:16.861 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ b0d008a8a9853471a66e05226b789b0d != \b\0\d\0\0\8\a\8\a\9\8\5\3\4\7\1\a\6\6\e\0\5\2\2\6\b\7\8\9\b\0\d ]] 2024-09-12T14:01:20.338 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ promote_md5=c6621649b323834dc1c05405228ef4ef 2024-09-12T14:01:20.338 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo rbd --cluster cluster2 device unmap -t krbd /dev/rbd4 2024-09-12T14:01:20.743 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ c6621649b323834dc1c05405228ef4ef != \c\6\6\2\1\6\4\9\b\3\2\3\8\3\4\d\c\1\c\0\5\4\0\5\2\2\8\e\f\4\e\f ]] 2024-09-12T14:01:20.968 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ promote_md5=e1196c387225daba9797472f838a3155 2024-09-12T14:01:20.968 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo rbd --cluster cluster2 device unmap -t krbd /dev/rbd6 2024-09-12T14:01:21.026 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ promote_md5=02511b187a77833c75a65a214cb45aca 2024-09-12T14:01:21.026 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo rbd --cluster cluster2 device unmap -t krbd /dev/rbd9 2024-09-12T14:01:21.176 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ e1196c387225daba9797472f838a3155 != \e\1\1\9\6\c\3\8\7\2\2\5\d\a\b\a\9\7\9\7\4\7\2\f\8\3\8\a\3\1\5\5 ]] 2024-09-12T14:01:21.182 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ promote_md5=c9fed370b723bc4064e2a8903a705a35 2024-09-12T14:01:21.182 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo rbd --cluster cluster2 device unmap -t krbd /dev/rbd5 2024-09-12T14:01:21.222 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ 02511b187a77833c75a65a214cb45aca != \0\2\5\1\1\b\1\8\7\a\7\7\8\3\3\c\7\5\a\6\5\a\2\1\4\c\b\4\5\a\c\a ]] 2024-09-12T14:01:21.222 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${COMPARE_PIDS[@]} 2024-09-12T14:01:21.222 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 125583 2024-09-12T14:01:21.222 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${COMPARE_PIDS[@]} 2024-09-12T14:01:21.223 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 125585 2024-09-12T14:01:21.223 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${COMPARE_PIDS[@]} 2024-09-12T14:01:21.223 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 125587 2024-09-12T14:01:21.233 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ promote_md5=9be7a78afff9f66c8c56e194e9abfe39 2024-09-12T14:01:21.233 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo rbd --cluster cluster2 device unmap -t krbd /dev/rbd8 2024-09-12T14:01:21.298 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ c9fed370b723bc4064e2a8903a705a35 != \c\9\f\e\d\3\7\0\b\7\2\3\b\c\4\0\6\4\e\2\a\8\9\0\3\a\7\0\5\a\3\5 ]] 2024-09-12T14:01:21.428 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ 9be7a78afff9f66c8c56e194e9abfe39 != \9\b\e\7\a\7\8\a\f\f\f\9\f\6\6\c\8\c\5\6\e\1\9\4\e\9\a\b\f\e\3\9 ]] 2024-09-12T14:01:21.742 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ promote_md5=ce70d4e2e62e55a655c7eab38f710d5f 2024-09-12T14:01:21.742 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo rbd --cluster cluster2 device unmap -t krbd /dev/rbd3 2024-09-12T14:01:21.775 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ promote_md5=529c2397fc816698df697aebe7c795bb 2024-09-12T14:01:21.775 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo rbd --cluster cluster2 device unmap -t krbd /dev/rbd7 2024-09-12T14:01:21.833 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ ce70d4e2e62e55a655c7eab38f710d5f != \c\e\7\0\d\4\e\2\e\6\2\e\5\5\a\6\5\5\c\7\e\a\b\3\8\f\7\1\0\d\5\f ]] 2024-09-12T14:01:21.866 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ 529c2397fc816698df697aebe7c795bb != \5\2\9\c\2\3\9\7\f\c\8\1\6\6\9\8\d\f\6\9\7\a\e\b\e\7\c\7\9\5\b\b ]] 2024-09-12T14:01:21.868 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${COMPARE_PIDS[@]} 2024-09-12T14:01:21.868 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 125589 2024-09-12T14:01:21.868 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${COMPARE_PIDS[@]} 2024-09-12T14:01:21.868 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 125591 2024-09-12T14:01:21.868 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${COMPARE_PIDS[@]} 2024-09-12T14:01:21.868 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 125593 2024-09-12T14:01:21.868 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${COMPARE_PIDS[@]} 2024-09-12T14:01:21.868 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 125595 2024-09-12T14:01:21.868 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${COMPARE_PIDS[@]} 2024-09-12T14:01:21.869 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 125597 2024-09-12T14:01:21.869 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${COMPARE_PIDS[@]} 2024-09-12T14:01:21.869 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 125599 2024-09-12T14:01:21.869 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ (( RET != 0 )) 2024-09-12T14:01:21.869 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T14:01:21.869 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ IMG=image-primary1 2024-09-12T14:01:21.869 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_replaying_status_in_pool_dir cluster1 mirror image-primary1 2024-09-12T14:01:21.869 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:01:21.869 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:01:21.869 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T14:01:21.869 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' snapshot = journal ']' 2024-09-12T14:01:21.869 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary1 up+replaying 2024-09-12T14:01:21.870 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:01:21.870 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:01:21.870 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T14:01:21.870 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T14:01:21.870 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:01:21.870 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:01:21.870 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:01:21.870 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T14:01:22.870 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary1 up+replaying '' '' 2024-09-12T14:01:22.870 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:01:22.870 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:01:22.870 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T14:01:22.870 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T14:01:22.870 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:01:22.870 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:01:22.870 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:01:22.871 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:01:22.871 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary1 2024-09-12T14:01:22.954 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary1: 2024-09-12T14:01:22.954 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: a862d41a-8d34-4c63-bea6-4195101bf11a 2024-09-12T14:01:22.954 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+replaying 2024-09-12T14:01:22.954 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: replaying, {"bytes_per_second":3248742.4,"bytes_per_snapshot":64974848.0,"last_snapshot_bytes":64974848,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726149627,"remote_snapshot_timestamp":1726149627,"replay_state":"idle"} 2024-09-12T14:01:22.955 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T14:01:22.955 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 14:01:17' 2024-09-12T14:01:22.955 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+replaying' 2024-09-12T14:01:22.957 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+replaying 2024-09-12T14:01:22.957 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T14:01:22.959 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: replaying, {"bytes_per_second":3248742.4,"bytes_per_snapshot":64974848.0,"last_snapshot_bytes":64974848,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726149627,"remote_snapshot_timestamp":1726149627,"replay_state":"idle"} 2024-09-12T14:01:22.960 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T14:01:22.960 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+replaying 2024-09-12T14:01:22.961 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T14:01:22.963 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+replaying 2024-09-12T14:01:22.963 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T14:01:22.966 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T14:01:22.966 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T14:01:22.967 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T14:01:22.971 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 14:01:17' 2024-09-12T14:01:22.971 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary1 up+replaying '2024-09-12 14:01:17' 2024-09-12T14:01:22.971 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:01:22.971 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:01:22.971 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T14:01:22.971 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T14:01:22.971 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 14:01:17' 2024-09-12T14:01:22.972 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:01:22.972 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:01:22.972 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-09-12T14:01:23.084 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041934232mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary1a862d41a-8d34-4c63-bea6-4195101bf11aup+replayingreplaying, {"bytes_per_second":3248742.4,"bytes_per_snapshot":64974848.0,"last_snapshot_bytes":64974848,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726149627,"remote_snapshot_timestamp":1726149627,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 14:01:17image-primary103238adfd-24b7-48b2-ab32-66aaa1c7b583up+replayingreplaying, {"bytes_per_second":3141017.6,"bytes_per_snapshot":62820352.0,"last_snapshot_bytes":62820352,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726149626,"remote_snapshot_timestamp":1726149626,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 14:01:17image-primary2fef52d8d-af21-4d3d-97e9-021eac35b36aup+replayingreplaying, {"bytes_per_second":3247923.2,"bytes_per_snapshot":64958464.0,"last_snapshot_bytes":64958464,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726149626,"remote_snapshot_timestamp":1726149626,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 14:01:17image-primary3a8c53447-94fc-4fe3-b559-d953f6f94c54up+replayingreplaying, {"bytes_per_second":3218022.4,"bytes_per_snapshot":64360448.0,"last_snapshot_bytes":64360448,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726149621,"remote_snapshot_timestamp":1726149621,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 14:01:17image-primary48face4e5-1ee3-427e-b5bc-7b05be9d6e87up+replayingreplaying, {"bytes_per_second":3222937.6,"bytes_per_snapshot":64458752.0,"last_snapshot_bytes":64458752,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726149627,"remote_snapshot_timestamp":1726149627,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 14:01:17image-primary5fb69fceb-7a10-4ab7-ad45-ec4124a4621fup+replayingreplaying, {"bytes_per_second":3215360.0,"bytes_per_snapshot":64307200.0,"last_snapshot_bytes":64307200,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726149627,"remote_snapshot_timestamp":1726149627,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 14:01:17image-primary6a63c238c-944e-44a1-be22-edc69436b860up+replayingreplaying, {"bytes_per_second":3329433.6,"bytes_per_snapshot":66588672.0,"last_snapshot_bytes":66588672,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726149626,"remote_snapshot_timestamp":1726149626,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 14:01:17image-primary7700ac9cd-0dca-415e-b998-79da95582617up+replayingreplaying, {"bytes_per_second":3240345.6,"bytes_per_snapshot":64806912.0,"last_snapshot_bytes":64806912,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726149622,"remote_snapshot_timestamp":1726149622,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 14:01:17image-primary83b4d0b11-5c89-4212-85d3-673c05d34c4bup+replayingreplaying, {"bytes_per_second":3368960.0,"bytes_per_snapshot":67379200.0,"last_snapshot_bytes":67379200,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726149622,"remote_snapshot_timestamp":1726149622,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 14:01:17image-primary941a87723-a798-494d-b064-df92256304f2up+replayingreplaying, {"bytes_per_second":3391897.6,"bytes_per_snapshot":67837952.0,"last_snapshot_bytes":67837952,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726149627,"remote_snapshot_timestamp":1726149627,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 14:01:17' 2024-09-12T14:01:23.084 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T14:01:23.084 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary1'\'']/last_update' 2024-09-12T14:01:23.371 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 14:01:17' 2024-09-12T14:01:23.372 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary1'\'']/state' 2024-09-12T14:01:23.376 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+replaying 2024-09-12T14:01:23.377 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+replaying 2024-09-12T14:01:23.377 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+replaying 2024-09-12T14:01:23.379 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+replaying 2024-09-12T14:01:23.380 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:01:23.380 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:01:23.380 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ remove_image cluster2 mirror image-primary1 2024-09-12T14:01:23.380 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T14:01:23.380 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:01:23.380 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T14:01:23.380 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 snap purge mirror/image-primary1 2024-09-12T14:01:23.442 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 rm mirror/image-primary1 2024-09-12T14:01:23.588 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T14:01:23.589+0000 7f67e4800640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x56055b9311e0 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-09-12T14:01:23.589 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T14:01:23.589+0000 7f67eca00640 -1 rbd::mirror::ImageReplayer: 0x5605561ac280 [3/a862d41a-8d34-4c63-bea6-4195101bf11a] operator(): start failed: (85) Interrupted system call should be restarted 2024-09-12T14:01:24.895 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-09-12T14:01:24.901 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_image_removal cluster1 mirror image-primary1 2024-09-12T14:01:24.901 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:01:24.901 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:01:24.901 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T14:01:24.902 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:01:24.902 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 ls mirror 2024-09-12T14:01:24.902 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep -wq image-primary1 2024-09-12T14:01:24.955 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:01:24.955 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rm -rf test-primary1 2024-09-12T14:01:25.001 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T14:01:25.002 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ IMG=image-primary2 2024-09-12T14:01:25.002 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_replaying_status_in_pool_dir cluster1 mirror image-primary2 2024-09-12T14:01:25.002 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:01:25.002 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:01:25.002 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T14:01:25.002 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' snapshot = journal ']' 2024-09-12T14:01:25.002 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary2 up+replaying 2024-09-12T14:01:25.002 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:01:25.002 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:01:25.002 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T14:01:25.002 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T14:01:25.002 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:01:25.002 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:01:25.002 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:01:25.003 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T14:01:26.003 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary2 up+replaying '' '' 2024-09-12T14:01:26.004 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:01:26.004 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:01:26.004 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T14:01:26.004 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T14:01:26.004 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:01:26.004 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:01:26.004 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:01:26.004 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:01:26.004 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary2 2024-09-12T14:01:26.085 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary2: 2024-09-12T14:01:26.086 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: fef52d8d-af21-4d3d-97e9-021eac35b36a 2024-09-12T14:01:26.086 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+replaying 2024-09-12T14:01:26.086 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: replaying, {"bytes_per_second":3247923.2,"bytes_per_snapshot":64958464.0,"last_snapshot_bytes":64958464,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726149626,"remote_snapshot_timestamp":1726149626,"replay_state":"idle"} 2024-09-12T14:01:26.086 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T14:01:26.086 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 14:01:17' 2024-09-12T14:01:26.086 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+replaying' 2024-09-12T14:01:26.088 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+replaying 2024-09-12T14:01:26.088 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T14:01:26.091 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: replaying, {"bytes_per_second":3247923.2,"bytes_per_snapshot":64958464.0,"last_snapshot_bytes":64958464,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726149626,"remote_snapshot_timestamp":1726149626,"replay_state":"idle"} 2024-09-12T14:01:26.091 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T14:01:26.092 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+replaying 2024-09-12T14:01:26.092 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T14:01:26.094 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+replaying 2024-09-12T14:01:26.095 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T14:01:26.097 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T14:01:26.098 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T14:01:26.098 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T14:01:26.102 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 14:01:17' 2024-09-12T14:01:26.102 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary2 up+replaying '2024-09-12 14:01:17' 2024-09-12T14:01:26.103 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:01:26.103 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:01:26.103 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T14:01:26.103 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T14:01:26.103 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 14:01:17' 2024-09-12T14:01:26.103 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:01:26.103 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:01:26.103 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-09-12T14:01:26.210 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING941934232mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary103238adfd-24b7-48b2-ab32-66aaa1c7b583up+replayingreplaying, {"bytes_per_second":3141017.6,"bytes_per_snapshot":62820352.0,"last_snapshot_bytes":62820352,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726149626,"remote_snapshot_timestamp":1726149626,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 14:01:17image-primary2fef52d8d-af21-4d3d-97e9-021eac35b36aup+replayingreplaying, {"bytes_per_second":3247923.2,"bytes_per_snapshot":64958464.0,"last_snapshot_bytes":64958464,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726149626,"remote_snapshot_timestamp":1726149626,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 14:01:17image-primary3a8c53447-94fc-4fe3-b559-d953f6f94c54up+replayingreplaying, {"bytes_per_second":3218022.4,"bytes_per_snapshot":64360448.0,"last_snapshot_bytes":64360448,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726149621,"remote_snapshot_timestamp":1726149621,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 14:01:17image-primary48face4e5-1ee3-427e-b5bc-7b05be9d6e87up+replayingreplaying, {"bytes_per_second":3222937.6,"bytes_per_snapshot":64458752.0,"last_snapshot_bytes":64458752,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726149627,"remote_snapshot_timestamp":1726149627,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 14:01:17image-primary5fb69fceb-7a10-4ab7-ad45-ec4124a4621fup+replayingreplaying, {"bytes_per_second":3215360.0,"bytes_per_snapshot":64307200.0,"last_snapshot_bytes":64307200,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726149627,"remote_snapshot_timestamp":1726149627,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 14:01:17image-primary6a63c238c-944e-44a1-be22-edc69436b860up+replayingreplaying, {"bytes_per_second":3329433.6,"bytes_per_snapshot":66588672.0,"last_snapshot_bytes":66588672,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726149626,"remote_snapshot_timestamp":1726149626,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 14:01:17image-primary7700ac9cd-0dca-415e-b998-79da95582617up+replayingreplaying, {"bytes_per_second":3240345.6,"bytes_per_snapshot":64806912.0,"last_snapshot_bytes":64806912,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726149622,"remote_snapshot_timestamp":1726149622,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 14:01:17image-primary83b4d0b11-5c89-4212-85d3-673c05d34c4bup+replayingreplaying, {"bytes_per_second":3368960.0,"bytes_per_snapshot":67379200.0,"last_snapshot_bytes":67379200,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726149622,"remote_snapshot_timestamp":1726149622,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 14:01:17image-primary941a87723-a798-494d-b064-df92256304f2up+replayingreplaying, {"bytes_per_second":3391897.6,"bytes_per_snapshot":67837952.0,"last_snapshot_bytes":67837952,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726149627,"remote_snapshot_timestamp":1726149627,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 14:01:17' 2024-09-12T14:01:26.211 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T14:01:26.211 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary2'\'']/last_update' 2024-09-12T14:01:26.215 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 14:01:17' 2024-09-12T14:01:26.215 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary2'\'']/state' 2024-09-12T14:01:26.219 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+replaying 2024-09-12T14:01:26.220 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+replaying 2024-09-12T14:01:26.220 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+replaying 2024-09-12T14:01:26.222 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+replaying 2024-09-12T14:01:26.223 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:01:26.223 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:01:26.223 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ remove_image cluster2 mirror image-primary2 2024-09-12T14:01:26.223 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T14:01:26.223 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:01:26.223 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T14:01:26.223 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 snap purge mirror/image-primary2 2024-09-12T14:01:26.283 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 rm mirror/image-primary2 2024-09-12T14:01:26.454 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T14:01:26.455+0000 7f67e5200640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x56055bf60000 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-09-12T14:01:26.454 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T14:01:26.455+0000 7f67ed400640 -1 rbd::mirror::ImageReplayer: 0x56055c198c80 [3/fef52d8d-af21-4d3d-97e9-021eac35b36a] operator(): start failed: (85) Interrupted system call should be restarted 2024-09-12T14:01:27.915 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-09-12T14:01:27.920 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_image_removal cluster1 mirror image-primary2 2024-09-12T14:01:27.921 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:01:27.921 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:01:27.921 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T14:01:27.921 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:01:27.921 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 ls mirror 2024-09-12T14:01:27.921 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep -wq image-primary2 2024-09-12T14:01:27.972 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:01:27.972 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rm -rf test-primary2 2024-09-12T14:01:27.975 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T14:01:27.975 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ IMG=image-primary3 2024-09-12T14:01:27.976 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_replaying_status_in_pool_dir cluster1 mirror image-primary3 2024-09-12T14:01:27.976 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:01:27.976 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:01:27.976 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T14:01:27.976 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' snapshot = journal ']' 2024-09-12T14:01:27.976 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary3 up+replaying 2024-09-12T14:01:27.976 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:01:27.976 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:01:27.976 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T14:01:27.976 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T14:01:27.976 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:01:27.976 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:01:27.977 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:01:27.977 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T14:01:28.976 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary3 up+replaying '' '' 2024-09-12T14:01:28.977 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:01:28.977 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:01:28.977 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T14:01:28.977 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T14:01:28.977 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:01:28.977 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:01:28.977 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:01:28.977 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:01:28.977 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary3 2024-09-12T14:01:29.057 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary3: 2024-09-12T14:01:29.058 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: a8c53447-94fc-4fe3-b559-d953f6f94c54 2024-09-12T14:01:29.058 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+replaying 2024-09-12T14:01:29.058 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: replaying, {"bytes_per_second":3218022.4,"bytes_per_snapshot":64360448.0,"last_snapshot_bytes":64360448,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726149621,"remote_snapshot_timestamp":1726149621,"replay_state":"idle"} 2024-09-12T14:01:29.058 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T14:01:29.058 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 14:01:17' 2024-09-12T14:01:29.058 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+replaying' 2024-09-12T14:01:29.060 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+replaying 2024-09-12T14:01:29.060 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T14:01:29.064 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: replaying, {"bytes_per_second":3218022.4,"bytes_per_snapshot":64360448.0,"last_snapshot_bytes":64360448,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726149621,"remote_snapshot_timestamp":1726149621,"replay_state":"idle"} 2024-09-12T14:01:29.065 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T14:01:29.065 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+replaying 2024-09-12T14:01:29.065 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T14:01:29.066 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+replaying 2024-09-12T14:01:29.066 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T14:01:29.069 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T14:01:29.069 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T14:01:29.070 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T14:01:29.074 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 14:01:17' 2024-09-12T14:01:29.074 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary3 up+replaying '2024-09-12 14:01:17' 2024-09-12T14:01:29.074 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:01:29.074 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:01:29.074 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T14:01:29.074 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T14:01:29.074 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 14:01:17' 2024-09-12T14:01:29.074 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:01:29.075 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:01:29.075 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-09-12T14:01:29.179 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING841934232mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary103238adfd-24b7-48b2-ab32-66aaa1c7b583up+replayingreplaying, {"bytes_per_second":3141017.6,"bytes_per_snapshot":62820352.0,"last_snapshot_bytes":62820352,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726149626,"remote_snapshot_timestamp":1726149626,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 14:01:17image-primary3a8c53447-94fc-4fe3-b559-d953f6f94c54up+replayingreplaying, {"bytes_per_second":3218022.4,"bytes_per_snapshot":64360448.0,"last_snapshot_bytes":64360448,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726149621,"remote_snapshot_timestamp":1726149621,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 14:01:17image-primary48face4e5-1ee3-427e-b5bc-7b05be9d6e87up+replayingreplaying, {"bytes_per_second":3222937.6,"bytes_per_snapshot":64458752.0,"last_snapshot_bytes":64458752,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726149627,"remote_snapshot_timestamp":1726149627,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 14:01:17image-primary5fb69fceb-7a10-4ab7-ad45-ec4124a4621fup+replayingreplaying, {"bytes_per_second":3215360.0,"bytes_per_snapshot":64307200.0,"last_snapshot_bytes":64307200,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726149627,"remote_snapshot_timestamp":1726149627,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 14:01:17image-primary6a63c238c-944e-44a1-be22-edc69436b860up+replayingreplaying, {"bytes_per_second":3329433.6,"bytes_per_snapshot":66588672.0,"last_snapshot_bytes":66588672,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726149626,"remote_snapshot_timestamp":1726149626,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 14:01:17image-primary7700ac9cd-0dca-415e-b998-79da95582617up+replayingreplaying, {"bytes_per_second":3240345.6,"bytes_per_snapshot":64806912.0,"last_snapshot_bytes":64806912,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726149622,"remote_snapshot_timestamp":1726149622,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 14:01:17image-primary83b4d0b11-5c89-4212-85d3-673c05d34c4bup+replayingreplaying, {"bytes_per_second":3368960.0,"bytes_per_snapshot":67379200.0,"last_snapshot_bytes":67379200,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726149622,"remote_snapshot_timestamp":1726149622,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 14:01:17image-primary941a87723-a798-494d-b064-df92256304f2up+replayingreplaying, {"bytes_per_second":3391897.6,"bytes_per_snapshot":67837952.0,"last_snapshot_bytes":67837952,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726149627,"remote_snapshot_timestamp":1726149627,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 14:01:17' 2024-09-12T14:01:29.179 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T14:01:29.180 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary3'\'']/last_update' 2024-09-12T14:01:29.184 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 14:01:17' 2024-09-12T14:01:29.184 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary3'\'']/state' 2024-09-12T14:01:29.188 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+replaying 2024-09-12T14:01:29.189 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+replaying 2024-09-12T14:01:29.189 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+replaying 2024-09-12T14:01:29.191 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+replaying 2024-09-12T14:01:29.192 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:01:29.192 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:01:29.192 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ remove_image cluster2 mirror image-primary3 2024-09-12T14:01:29.192 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T14:01:29.192 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:01:29.192 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T14:01:29.192 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 snap purge mirror/image-primary3 2024-09-12T14:01:29.253 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 rm mirror/image-primary3 2024-09-12T14:01:29.402 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T14:01:29.401+0000 7f67e4800640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x5605566e5520 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-09-12T14:01:29.402 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T14:01:29.401+0000 7f67ed400640 -1 rbd::mirror::ImageReplayer: 0x56055c199680 [3/a8c53447-94fc-4fe3-b559-d953f6f94c54] operator(): start failed: (85) Interrupted system call should be restarted 2024-09-12T14:01:30.523 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-09-12T14:01:30.529 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_image_removal cluster1 mirror image-primary3 2024-09-12T14:01:30.530 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:01:30.530 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:01:30.530 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T14:01:30.530 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:01:30.531 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 ls mirror 2024-09-12T14:01:30.531 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep -wq image-primary3 2024-09-12T14:01:30.588 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:01:30.588 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rm -rf test-primary3 2024-09-12T14:01:30.591 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T14:01:30.591 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ IMG=image-primary4 2024-09-12T14:01:30.591 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_replaying_status_in_pool_dir cluster1 mirror image-primary4 2024-09-12T14:01:30.591 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:01:30.591 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:01:30.591 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T14:01:30.591 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' snapshot = journal ']' 2024-09-12T14:01:30.591 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary4 up+replaying 2024-09-12T14:01:30.591 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:01:30.591 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:01:30.591 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T14:01:30.592 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T14:01:30.592 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:01:30.592 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:01:30.592 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:01:30.592 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T14:01:31.593 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary4 up+replaying '' '' 2024-09-12T14:01:31.593 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:01:31.593 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:01:31.593 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T14:01:31.593 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T14:01:31.593 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:01:31.593 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:01:31.593 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:01:31.594 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:01:31.594 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary4 2024-09-12T14:01:31.708 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary4: 2024-09-12T14:01:31.708 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 8face4e5-1ee3-427e-b5bc-7b05be9d6e87 2024-09-12T14:01:31.708 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+replaying 2024-09-12T14:01:31.708 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: replaying, {"bytes_per_second":3222937.6,"bytes_per_snapshot":64458752.0,"last_snapshot_bytes":64458752,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726149627,"remote_snapshot_timestamp":1726149627,"replay_state":"idle"} 2024-09-12T14:01:31.708 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T14:01:31.709 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 14:01:17' 2024-09-12T14:01:31.709 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+replaying' 2024-09-12T14:01:31.710 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+replaying 2024-09-12T14:01:31.711 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T14:01:31.713 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: replaying, {"bytes_per_second":3222937.6,"bytes_per_snapshot":64458752.0,"last_snapshot_bytes":64458752,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726149627,"remote_snapshot_timestamp":1726149627,"replay_state":"idle"} 2024-09-12T14:01:31.714 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T14:01:31.714 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+replaying 2024-09-12T14:01:31.715 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T14:01:31.716 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+replaying 2024-09-12T14:01:31.717 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T14:01:31.719 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T14:01:31.720 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T14:01:31.720 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T14:01:31.724 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 14:01:17' 2024-09-12T14:01:31.724 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary4 up+replaying '2024-09-12 14:01:17' 2024-09-12T14:01:31.724 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:01:31.724 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:01:31.724 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T14:01:31.725 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T14:01:31.725 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 14:01:17' 2024-09-12T14:01:31.725 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:01:31.725 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:01:31.725 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-09-12T14:01:31.862 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING741934232mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary103238adfd-24b7-48b2-ab32-66aaa1c7b583up+replayingreplaying, {"bytes_per_second":3141017.6,"bytes_per_snapshot":62820352.0,"last_snapshot_bytes":62820352,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726149626,"remote_snapshot_timestamp":1726149626,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 14:01:17image-primary48face4e5-1ee3-427e-b5bc-7b05be9d6e87up+replayingreplaying, {"bytes_per_second":3222937.6,"bytes_per_snapshot":64458752.0,"last_snapshot_bytes":64458752,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726149627,"remote_snapshot_timestamp":1726149627,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 14:01:17image-primary5fb69fceb-7a10-4ab7-ad45-ec4124a4621fup+replayingreplaying, {"bytes_per_second":3215360.0,"bytes_per_snapshot":64307200.0,"last_snapshot_bytes":64307200,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726149627,"remote_snapshot_timestamp":1726149627,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 14:01:17image-primary6a63c238c-944e-44a1-be22-edc69436b860up+replayingreplaying, {"bytes_per_second":3329433.6,"bytes_per_snapshot":66588672.0,"last_snapshot_bytes":66588672,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726149626,"remote_snapshot_timestamp":1726149626,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 14:01:17image-primary7700ac9cd-0dca-415e-b998-79da95582617up+replayingreplaying, {"bytes_per_second":3240345.6,"bytes_per_snapshot":64806912.0,"last_snapshot_bytes":64806912,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726149622,"remote_snapshot_timestamp":1726149622,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 14:01:17image-primary83b4d0b11-5c89-4212-85d3-673c05d34c4bup+replayingreplaying, {"bytes_per_second":3368960.0,"bytes_per_snapshot":67379200.0,"last_snapshot_bytes":67379200,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726149622,"remote_snapshot_timestamp":1726149622,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 14:01:17image-primary941a87723-a798-494d-b064-df92256304f2up+replayingreplaying, {"bytes_per_second":3391897.6,"bytes_per_snapshot":67837952.0,"last_snapshot_bytes":67837952,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726149627,"remote_snapshot_timestamp":1726149627,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 14:01:17' 2024-09-12T14:01:31.862 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T14:01:31.862 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary4'\'']/last_update' 2024-09-12T14:01:31.866 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 14:01:17' 2024-09-12T14:01:31.867 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary4'\'']/state' 2024-09-12T14:01:31.871 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+replaying 2024-09-12T14:01:31.871 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+replaying 2024-09-12T14:01:31.872 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+replaying 2024-09-12T14:01:31.874 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+replaying 2024-09-12T14:01:31.874 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:01:31.874 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:01:31.875 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ remove_image cluster2 mirror image-primary4 2024-09-12T14:01:31.875 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T14:01:31.875 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:01:31.875 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T14:01:31.875 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 snap purge mirror/image-primary4 2024-09-12T14:01:31.938 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 rm mirror/image-primary4 2024-09-12T14:01:32.169 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T14:01:32.169+0000 7f67e4800640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x56055c052680 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-09-12T14:01:32.169 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T14:01:32.169+0000 7f67ed400640 -1 rbd::mirror::ImageReplayer: 0x56055c198780 [3/8face4e5-1ee3-427e-b5bc-7b05be9d6e87] operator(): start failed: (85) Interrupted system call should be restarted 2024-09-12T14:01:33.535 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-09-12T14:01:33.541 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_image_removal cluster1 mirror image-primary4 2024-09-12T14:01:33.541 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:01:33.541 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:01:33.541 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T14:01:33.541 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:01:33.542 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 ls mirror 2024-09-12T14:01:33.542 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep -wq image-primary4 2024-09-12T14:01:33.591 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:01:33.591 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rm -rf test-primary4 2024-09-12T14:01:33.594 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T14:01:33.594 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ IMG=image-primary5 2024-09-12T14:01:33.594 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_replaying_status_in_pool_dir cluster1 mirror image-primary5 2024-09-12T14:01:33.594 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:01:33.594 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:01:33.594 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T14:01:33.594 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' snapshot = journal ']' 2024-09-12T14:01:33.594 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary5 up+replaying 2024-09-12T14:01:33.594 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:01:33.594 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:01:33.594 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T14:01:33.595 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T14:01:33.595 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:01:33.595 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:01:33.595 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:01:33.595 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T14:01:34.596 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary5 up+replaying '' '' 2024-09-12T14:01:34.596 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:01:34.597 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:01:34.597 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T14:01:34.597 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T14:01:34.597 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:01:34.597 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:01:34.597 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:01:34.597 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:01:34.597 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary5 2024-09-12T14:01:34.675 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary5: 2024-09-12T14:01:34.676 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: fb69fceb-7a10-4ab7-ad45-ec4124a4621f 2024-09-12T14:01:34.676 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+replaying 2024-09-12T14:01:34.676 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: replaying, {"bytes_per_second":3215360.0,"bytes_per_snapshot":64307200.0,"last_snapshot_bytes":64307200,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726149627,"remote_snapshot_timestamp":1726149627,"replay_state":"idle"} 2024-09-12T14:01:34.676 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T14:01:34.676 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 14:01:17' 2024-09-12T14:01:34.676 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+replaying' 2024-09-12T14:01:34.678 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+replaying 2024-09-12T14:01:34.679 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T14:01:34.681 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: replaying, {"bytes_per_second":3215360.0,"bytes_per_snapshot":64307200.0,"last_snapshot_bytes":64307200,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726149627,"remote_snapshot_timestamp":1726149627,"replay_state":"idle"} 2024-09-12T14:01:34.681 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T14:01:34.682 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+replaying 2024-09-12T14:01:34.682 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T14:01:34.684 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+replaying 2024-09-12T14:01:34.684 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T14:01:34.687 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T14:01:34.687 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T14:01:34.688 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T14:01:34.692 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 14:01:17' 2024-09-12T14:01:34.692 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary5 up+replaying '2024-09-12 14:01:17' 2024-09-12T14:01:34.692 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:01:34.692 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:01:34.692 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T14:01:34.692 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T14:01:34.693 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 14:01:17' 2024-09-12T14:01:34.693 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:01:34.693 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:01:34.693 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-09-12T14:01:34.791 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING641934232mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary103238adfd-24b7-48b2-ab32-66aaa1c7b583up+replayingreplaying, {"bytes_per_second":3141017.6,"bytes_per_snapshot":62820352.0,"last_snapshot_bytes":62820352,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726149626,"remote_snapshot_timestamp":1726149626,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 14:01:17image-primary5fb69fceb-7a10-4ab7-ad45-ec4124a4621fup+replayingreplaying, {"bytes_per_second":3215360.0,"bytes_per_snapshot":64307200.0,"last_snapshot_bytes":64307200,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726149627,"remote_snapshot_timestamp":1726149627,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 14:01:17image-primary6a63c238c-944e-44a1-be22-edc69436b860up+replayingreplaying, {"bytes_per_second":3329433.6,"bytes_per_snapshot":66588672.0,"last_snapshot_bytes":66588672,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726149626,"remote_snapshot_timestamp":1726149626,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 14:01:17image-primary7700ac9cd-0dca-415e-b998-79da95582617up+replayingreplaying, {"bytes_per_second":3240345.6,"bytes_per_snapshot":64806912.0,"last_snapshot_bytes":64806912,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726149622,"remote_snapshot_timestamp":1726149622,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 14:01:17image-primary83b4d0b11-5c89-4212-85d3-673c05d34c4bup+replayingreplaying, {"bytes_per_second":3368960.0,"bytes_per_snapshot":67379200.0,"last_snapshot_bytes":67379200,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726149622,"remote_snapshot_timestamp":1726149622,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 14:01:17image-primary941a87723-a798-494d-b064-df92256304f2up+replayingreplaying, {"bytes_per_second":3391897.6,"bytes_per_snapshot":67837952.0,"last_snapshot_bytes":67837952,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726149627,"remote_snapshot_timestamp":1726149627,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 14:01:17' 2024-09-12T14:01:34.791 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T14:01:34.791 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary5'\'']/last_update' 2024-09-12T14:01:34.795 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 14:01:17' 2024-09-12T14:01:34.796 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary5'\'']/state' 2024-09-12T14:01:34.799 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+replaying 2024-09-12T14:01:34.800 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+replaying 2024-09-12T14:01:34.800 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+replaying 2024-09-12T14:01:34.802 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+replaying 2024-09-12T14:01:34.803 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:01:34.803 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:01:34.803 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ remove_image cluster2 mirror image-primary5 2024-09-12T14:01:34.803 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T14:01:34.803 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:01:34.803 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T14:01:34.803 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 snap purge mirror/image-primary5 2024-09-12T14:01:34.865 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 rm mirror/image-primary5 2024-09-12T14:01:35.016 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T14:01:35.016+0000 7f67e5200640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x5605566e4680 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-09-12T14:01:35.016 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T14:01:35.016+0000 7f67eca00640 -1 rbd::mirror::ImageReplayer: 0x5605561ac000 [3/fb69fceb-7a10-4ab7-ad45-ec4124a4621f] operator(): start failed: (85) Interrupted system call should be restarted 2024-09-12T14:01:36.564 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-09-12T14:01:36.569 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_image_removal cluster1 mirror image-primary5 2024-09-12T14:01:36.570 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:01:36.570 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:01:36.570 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T14:01:36.570 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:01:36.570 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 ls mirror 2024-09-12T14:01:36.570 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep -wq image-primary5 2024-09-12T14:01:36.621 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:01:36.621 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rm -rf test-primary5 2024-09-12T14:01:36.624 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T14:01:36.624 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ IMG=image-primary6 2024-09-12T14:01:36.624 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_replaying_status_in_pool_dir cluster1 mirror image-primary6 2024-09-12T14:01:36.624 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:01:36.624 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:01:36.624 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T14:01:36.625 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' snapshot = journal ']' 2024-09-12T14:01:36.625 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary6 up+replaying 2024-09-12T14:01:36.625 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:01:36.625 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:01:36.625 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T14:01:36.625 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T14:01:36.625 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:01:36.625 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:01:36.625 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:01:36.625 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T14:01:37.626 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary6 up+replaying '' '' 2024-09-12T14:01:37.626 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:01:37.626 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:01:37.626 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T14:01:37.626 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T14:01:37.626 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:01:37.626 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:01:37.627 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:01:37.627 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:01:37.627 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary6 2024-09-12T14:01:37.707 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary6: 2024-09-12T14:01:37.708 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: a63c238c-944e-44a1-be22-edc69436b860 2024-09-12T14:01:37.708 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+replaying 2024-09-12T14:01:37.708 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: replaying, {"bytes_per_second":3329433.6,"bytes_per_snapshot":66588672.0,"last_snapshot_bytes":66588672,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726149626,"remote_snapshot_timestamp":1726149626,"replay_state":"idle"} 2024-09-12T14:01:37.708 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T14:01:37.708 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 14:01:17' 2024-09-12T14:01:37.708 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+replaying' 2024-09-12T14:01:37.710 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+replaying 2024-09-12T14:01:37.710 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T14:01:37.713 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: replaying, {"bytes_per_second":3329433.6,"bytes_per_snapshot":66588672.0,"last_snapshot_bytes":66588672,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726149626,"remote_snapshot_timestamp":1726149626,"replay_state":"idle"} 2024-09-12T14:01:37.713 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T14:01:37.714 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+replaying 2024-09-12T14:01:37.714 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T14:01:37.716 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+replaying 2024-09-12T14:01:37.716 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T14:01:37.719 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T14:01:37.720 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T14:01:37.720 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T14:01:37.724 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 14:01:17' 2024-09-12T14:01:37.724 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary6 up+replaying '2024-09-12 14:01:17' 2024-09-12T14:01:37.724 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:01:37.724 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:01:37.724 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T14:01:37.724 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T14:01:37.725 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 14:01:17' 2024-09-12T14:01:37.725 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:01:37.725 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:01:37.725 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-09-12T14:01:37.816 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING541934232mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary103238adfd-24b7-48b2-ab32-66aaa1c7b583up+replayingreplaying, {"bytes_per_second":3141017.6,"bytes_per_snapshot":62820352.0,"last_snapshot_bytes":62820352,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726149626,"remote_snapshot_timestamp":1726149626,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 14:01:17image-primary6a63c238c-944e-44a1-be22-edc69436b860up+replayingreplaying, {"bytes_per_second":3329433.6,"bytes_per_snapshot":66588672.0,"last_snapshot_bytes":66588672,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726149626,"remote_snapshot_timestamp":1726149626,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 14:01:17image-primary7700ac9cd-0dca-415e-b998-79da95582617up+replayingreplaying, {"bytes_per_second":3240345.6,"bytes_per_snapshot":64806912.0,"last_snapshot_bytes":64806912,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726149622,"remote_snapshot_timestamp":1726149622,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 14:01:17image-primary83b4d0b11-5c89-4212-85d3-673c05d34c4bup+replayingreplaying, {"bytes_per_second":3368960.0,"bytes_per_snapshot":67379200.0,"last_snapshot_bytes":67379200,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726149622,"remote_snapshot_timestamp":1726149622,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 14:01:17image-primary941a87723-a798-494d-b064-df92256304f2up+replayingreplaying, {"bytes_per_second":3391897.6,"bytes_per_snapshot":67837952.0,"last_snapshot_bytes":67837952,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726149627,"remote_snapshot_timestamp":1726149627,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 14:01:17' 2024-09-12T14:01:37.816 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T14:01:37.817 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary6'\'']/last_update' 2024-09-12T14:01:37.821 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 14:01:17' 2024-09-12T14:01:37.822 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary6'\'']/state' 2024-09-12T14:01:37.825 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+replaying 2024-09-12T14:01:37.826 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+replaying 2024-09-12T14:01:37.826 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+replaying 2024-09-12T14:01:37.828 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+replaying 2024-09-12T14:01:37.829 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:01:37.829 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:01:37.829 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ remove_image cluster2 mirror image-primary6 2024-09-12T14:01:37.830 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T14:01:37.830 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:01:37.830 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T14:01:37.830 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 snap purge mirror/image-primary6 2024-09-12T14:01:37.891 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 rm mirror/image-primary6 2024-09-12T14:01:38.042 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T14:01:38.042+0000 7f67e5200640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x56055b865040 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-09-12T14:01:38.042 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T14:01:38.042+0000 7f67eca00640 -1 rbd::mirror::ImageReplayer: 0x56055c198500 [3/a63c238c-944e-44a1-be22-edc69436b860] operator(): start failed: (85) Interrupted system call should be restarted 2024-09-12T14:01:39.562 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-09-12T14:01:39.568 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_image_removal cluster1 mirror image-primary6 2024-09-12T14:01:39.569 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:01:39.569 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:01:39.569 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T14:01:39.569 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:01:39.569 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 ls mirror 2024-09-12T14:01:39.569 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep -wq image-primary6 2024-09-12T14:01:39.627 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:01:39.627 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rm -rf test-primary6 2024-09-12T14:01:39.629 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T14:01:39.630 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ IMG=image-primary7 2024-09-12T14:01:39.630 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_replaying_status_in_pool_dir cluster1 mirror image-primary7 2024-09-12T14:01:39.630 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:01:39.630 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:01:39.630 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T14:01:39.630 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' snapshot = journal ']' 2024-09-12T14:01:39.630 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary7 up+replaying 2024-09-12T14:01:39.630 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:01:39.630 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:01:39.630 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T14:01:39.630 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T14:01:39.630 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:01:39.630 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:01:39.631 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:01:39.631 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T14:01:40.632 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary7 up+replaying '' '' 2024-09-12T14:01:40.632 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:01:40.632 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:01:40.632 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T14:01:40.632 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T14:01:40.632 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:01:40.632 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:01:40.632 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:01:40.633 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:01:40.633 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary7 2024-09-12T14:01:40.734 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary7: 2024-09-12T14:01:40.734 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 700ac9cd-0dca-415e-b998-79da95582617 2024-09-12T14:01:40.734 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+replaying 2024-09-12T14:01:40.734 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: replaying, {"bytes_per_second":3240345.6,"bytes_per_snapshot":64806912.0,"last_snapshot_bytes":64806912,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726149622,"remote_snapshot_timestamp":1726149622,"replay_state":"idle"} 2024-09-12T14:01:40.734 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T14:01:40.734 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 14:01:17' 2024-09-12T14:01:40.735 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+replaying' 2024-09-12T14:01:40.737 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+replaying 2024-09-12T14:01:40.737 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T14:01:40.739 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: replaying, {"bytes_per_second":3240345.6,"bytes_per_snapshot":64806912.0,"last_snapshot_bytes":64806912,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726149622,"remote_snapshot_timestamp":1726149622,"replay_state":"idle"} 2024-09-12T14:01:40.740 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T14:01:40.741 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+replaying 2024-09-12T14:01:40.741 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T14:01:40.743 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+replaying 2024-09-12T14:01:40.743 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T14:01:40.745 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T14:01:40.746 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T14:01:40.746 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T14:01:40.750 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 14:01:17' 2024-09-12T14:01:40.750 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary7 up+replaying '2024-09-12 14:01:17' 2024-09-12T14:01:40.751 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:01:40.751 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:01:40.751 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T14:01:40.751 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T14:01:40.751 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 14:01:17' 2024-09-12T14:01:40.751 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:01:40.751 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:01:40.751 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-09-12T14:01:40.882 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING441934232mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary103238adfd-24b7-48b2-ab32-66aaa1c7b583up+replayingreplaying, {"bytes_per_second":3141017.6,"bytes_per_snapshot":62820352.0,"last_snapshot_bytes":62820352,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726149626,"remote_snapshot_timestamp":1726149626,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 14:01:17image-primary7700ac9cd-0dca-415e-b998-79da95582617up+replayingreplaying, {"bytes_per_second":3240345.6,"bytes_per_snapshot":64806912.0,"last_snapshot_bytes":64806912,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726149622,"remote_snapshot_timestamp":1726149622,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 14:01:17image-primary83b4d0b11-5c89-4212-85d3-673c05d34c4bup+replayingreplaying, {"bytes_per_second":3368960.0,"bytes_per_snapshot":67379200.0,"last_snapshot_bytes":67379200,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726149622,"remote_snapshot_timestamp":1726149622,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 14:01:17image-primary941a87723-a798-494d-b064-df92256304f2up+replayingreplaying, {"bytes_per_second":3391897.6,"bytes_per_snapshot":67837952.0,"last_snapshot_bytes":67837952,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726149627,"remote_snapshot_timestamp":1726149627,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 14:01:17' 2024-09-12T14:01:40.882 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T14:01:40.882 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary7'\'']/last_update' 2024-09-12T14:01:40.886 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 14:01:17' 2024-09-12T14:01:40.887 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary7'\'']/state' 2024-09-12T14:01:40.890 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+replaying 2024-09-12T14:01:40.891 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+replaying 2024-09-12T14:01:40.891 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+replaying 2024-09-12T14:01:40.893 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+replaying 2024-09-12T14:01:40.894 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:01:40.894 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:01:40.894 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ remove_image cluster2 mirror image-primary7 2024-09-12T14:01:40.894 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T14:01:40.894 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:01:40.894 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T14:01:40.895 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 snap purge mirror/image-primary7 2024-09-12T14:01:40.960 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 rm mirror/image-primary7 2024-09-12T14:01:41.174 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T14:01:41.174+0000 7f67e4800640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x560555778b60 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-09-12T14:01:41.175 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T14:01:41.174+0000 7f67ed400640 -1 rbd::mirror::ImageReplayer: 0x56055c199180 [3/700ac9cd-0dca-415e-b998-79da95582617] operator(): start failed: (85) Interrupted system call should be restarted 2024-09-12T14:01:42.569 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-09-12T14:01:42.575 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_image_removal cluster1 mirror image-primary7 2024-09-12T14:01:42.575 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:01:42.575 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:01:42.575 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T14:01:42.575 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:01:42.575 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 ls mirror 2024-09-12T14:01:42.576 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep -wq image-primary7 2024-09-12T14:01:42.625 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:01:42.625 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rm -rf test-primary7 2024-09-12T14:01:42.628 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T14:01:42.628 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ IMG=image-primary8 2024-09-12T14:01:42.628 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_replaying_status_in_pool_dir cluster1 mirror image-primary8 2024-09-12T14:01:42.628 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:01:42.628 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:01:42.628 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T14:01:42.629 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' snapshot = journal ']' 2024-09-12T14:01:42.629 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary8 up+replaying 2024-09-12T14:01:42.629 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:01:42.629 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:01:42.629 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T14:01:42.629 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T14:01:42.629 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:01:42.629 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:01:42.629 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:01:42.629 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T14:01:43.630 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary8 up+replaying '' '' 2024-09-12T14:01:43.631 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:01:43.631 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:01:43.631 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T14:01:43.631 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T14:01:43.631 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:01:43.631 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:01:43.631 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:01:43.631 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:01:43.631 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary8 2024-09-12T14:01:43.712 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary8: 2024-09-12T14:01:43.712 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 3b4d0b11-5c89-4212-85d3-673c05d34c4b 2024-09-12T14:01:43.712 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+replaying 2024-09-12T14:01:43.712 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: replaying, {"bytes_per_second":3368960.0,"bytes_per_snapshot":67379200.0,"last_snapshot_bytes":67379200,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726149622,"remote_snapshot_timestamp":1726149622,"replay_state":"idle"} 2024-09-12T14:01:43.712 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T14:01:43.712 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 14:01:17' 2024-09-12T14:01:43.712 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+replaying' 2024-09-12T14:01:43.714 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+replaying 2024-09-12T14:01:43.715 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T14:01:43.717 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: replaying, {"bytes_per_second":3368960.0,"bytes_per_snapshot":67379200.0,"last_snapshot_bytes":67379200,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726149622,"remote_snapshot_timestamp":1726149622,"replay_state":"idle"} 2024-09-12T14:01:43.717 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T14:01:43.718 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+replaying 2024-09-12T14:01:43.718 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T14:01:43.720 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+replaying 2024-09-12T14:01:43.721 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T14:01:43.723 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T14:01:43.723 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T14:01:43.724 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T14:01:43.728 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 14:01:17' 2024-09-12T14:01:43.728 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary8 up+replaying '2024-09-12 14:01:17' 2024-09-12T14:01:43.728 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:01:43.729 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:01:43.729 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T14:01:43.729 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T14:01:43.729 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 14:01:17' 2024-09-12T14:01:43.729 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:01:43.729 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:01:43.729 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-09-12T14:01:43.821 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING341934232mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary103238adfd-24b7-48b2-ab32-66aaa1c7b583up+replayingreplaying, {"bytes_per_second":3141017.6,"bytes_per_snapshot":62820352.0,"last_snapshot_bytes":62820352,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726149626,"remote_snapshot_timestamp":1726149626,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 14:01:17image-primary83b4d0b11-5c89-4212-85d3-673c05d34c4bup+replayingreplaying, {"bytes_per_second":3368960.0,"bytes_per_snapshot":67379200.0,"last_snapshot_bytes":67379200,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726149622,"remote_snapshot_timestamp":1726149622,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 14:01:17image-primary941a87723-a798-494d-b064-df92256304f2up+replayingreplaying, {"bytes_per_second":3391897.6,"bytes_per_snapshot":67837952.0,"last_snapshot_bytes":67837952,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726149627,"remote_snapshot_timestamp":1726149627,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 14:01:17' 2024-09-12T14:01:43.821 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T14:01:43.821 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary8'\'']/last_update' 2024-09-12T14:01:43.821 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 14:01:17' 2024-09-12T14:01:43.822 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary8'\'']/state' 2024-09-12T14:01:43.826 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+replaying 2024-09-12T14:01:43.826 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+replaying 2024-09-12T14:01:43.827 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+replaying 2024-09-12T14:01:43.829 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+replaying 2024-09-12T14:01:43.829 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:01:43.829 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:01:43.829 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ remove_image cluster2 mirror image-primary8 2024-09-12T14:01:43.829 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T14:01:43.830 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:01:43.830 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T14:01:43.830 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 snap purge mirror/image-primary8 2024-09-12T14:01:43.892 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 rm mirror/image-primary8 2024-09-12T14:01:44.041 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T14:01:44.041+0000 7f67e4800640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x56055b8cab60 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-09-12T14:01:44.042 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T14:01:44.041+0000 7f67eca00640 -1 rbd::mirror::ImageReplayer: 0x5605561acf00 [3/3b4d0b11-5c89-4212-85d3-673c05d34c4b] operator(): start failed: (85) Interrupted system call should be restarted 2024-09-12T14:01:45.537 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-09-12T14:01:45.543 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_image_removal cluster1 mirror image-primary8 2024-09-12T14:01:45.543 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:01:45.543 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:01:45.543 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T14:01:45.543 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:01:45.544 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 ls mirror 2024-09-12T14:01:45.544 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep -wq image-primary8 2024-09-12T14:01:45.593 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:01:45.593 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rm -rf test-primary8 2024-09-12T14:01:45.595 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T14:01:45.595 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ IMG=image-primary9 2024-09-12T14:01:45.595 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_replaying_status_in_pool_dir cluster1 mirror image-primary9 2024-09-12T14:01:45.595 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:01:45.596 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:01:45.596 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T14:01:45.596 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' snapshot = journal ']' 2024-09-12T14:01:45.596 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary9 up+replaying 2024-09-12T14:01:45.596 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:01:45.596 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:01:45.596 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T14:01:45.596 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T14:01:45.596 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:01:45.596 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:01:45.596 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:01:45.596 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T14:01:46.598 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary9 up+replaying '' '' 2024-09-12T14:01:46.598 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:01:46.598 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:01:46.598 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T14:01:46.598 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T14:01:46.598 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:01:46.598 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:01:46.598 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:01:46.599 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:01:46.599 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary9 2024-09-12T14:01:46.681 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary9: 2024-09-12T14:01:46.681 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 41a87723-a798-494d-b064-df92256304f2 2024-09-12T14:01:46.681 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+replaying 2024-09-12T14:01:46.681 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: replaying, {"bytes_per_second":3391897.6,"bytes_per_snapshot":67837952.0,"last_snapshot_bytes":67837952,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726149627,"remote_snapshot_timestamp":1726149627,"replay_state":"idle"} 2024-09-12T14:01:46.682 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T14:01:46.682 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 14:01:17' 2024-09-12T14:01:46.682 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+replaying' 2024-09-12T14:01:46.684 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+replaying 2024-09-12T14:01:46.684 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T14:01:46.687 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: replaying, {"bytes_per_second":3391897.6,"bytes_per_snapshot":67837952.0,"last_snapshot_bytes":67837952,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726149627,"remote_snapshot_timestamp":1726149627,"replay_state":"idle"} 2024-09-12T14:01:46.687 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T14:01:46.688 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+replaying 2024-09-12T14:01:46.688 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T14:01:46.690 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+replaying 2024-09-12T14:01:46.690 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T14:01:46.693 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T14:01:46.693 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T14:01:46.694 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T14:01:46.699 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 14:01:17' 2024-09-12T14:01:46.699 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary9 up+replaying '2024-09-12 14:01:17' 2024-09-12T14:01:46.699 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:01:46.699 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:01:46.699 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T14:01:46.699 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T14:01:46.699 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 14:01:17' 2024-09-12T14:01:46.699 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:01:46.699 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:01:46.699 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-09-12T14:01:46.786 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING241934232mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary103238adfd-24b7-48b2-ab32-66aaa1c7b583up+replayingreplaying, {"bytes_per_second":3141017.6,"bytes_per_snapshot":62820352.0,"last_snapshot_bytes":62820352,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726149626,"remote_snapshot_timestamp":1726149626,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 14:01:17image-primary941a87723-a798-494d-b064-df92256304f2up+replayingreplaying, {"bytes_per_second":3391897.6,"bytes_per_snapshot":67837952.0,"last_snapshot_bytes":67837952,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726149627,"remote_snapshot_timestamp":1726149627,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 14:01:17' 2024-09-12T14:01:46.786 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T14:01:46.787 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary9'\'']/last_update' 2024-09-12T14:01:46.791 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 14:01:17' 2024-09-12T14:01:46.791 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary9'\'']/state' 2024-09-12T14:01:46.795 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+replaying 2024-09-12T14:01:46.795 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+replaying 2024-09-12T14:01:46.796 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+replaying 2024-09-12T14:01:46.798 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+replaying 2024-09-12T14:01:46.799 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:01:46.799 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:01:46.799 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ remove_image cluster2 mirror image-primary9 2024-09-12T14:01:46.799 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T14:01:46.799 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:01:46.799 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T14:01:46.799 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 snap purge mirror/image-primary9 2024-09-12T14:01:46.859 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 rm mirror/image-primary9 2024-09-12T14:01:47.005 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T14:01:47.004+0000 7f67e5200640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x56055c0524e0 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-09-12T14:01:47.005 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T14:01:47.004+0000 7f67eca00640 -1 rbd::mirror::ImageReplayer: 0x5605561ad680 [3/41a87723-a798-494d-b064-df92256304f2] operator(): start failed: (85) Interrupted system call should be restarted 2024-09-12T14:01:47.905 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T14:01:47.905+0000 7f67e4800640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x56055be681a0 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-09-12T14:01:47.905 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T14:01:47.905+0000 7f67ed400640 -1 rbd::mirror::ImageReplayer: 0x5605561ad680 [3/41a87723-a798-494d-b064-df92256304f2] operator(): start failed: (85) Interrupted system call should be restarted 2024-09-12T14:01:48.864 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-09-12T14:01:48.868 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_image_removal cluster1 mirror image-primary9 2024-09-12T14:01:48.869 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:01:48.869 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:01:48.869 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T14:01:48.869 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:01:48.869 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 ls mirror 2024-09-12T14:01:48.869 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep -wq image-primary9 2024-09-12T14:01:48.924 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:01:48.924 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rm -rf test-primary9 2024-09-12T14:01:48.926 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T14:01:48.926 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ IMG=image-primary10 2024-09-12T14:01:48.926 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_replaying_status_in_pool_dir cluster1 mirror image-primary10 2024-09-12T14:01:48.927 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:01:48.927 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:01:48.927 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T14:01:48.927 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' snapshot = journal ']' 2024-09-12T14:01:48.927 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary10 up+replaying 2024-09-12T14:01:48.927 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:01:48.927 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:01:48.927 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T14:01:48.927 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T14:01:48.927 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:01:48.927 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:01:48.927 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:01:48.927 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T14:01:49.929 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary10 up+replaying '' '' 2024-09-12T14:01:49.929 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:01:49.929 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:01:49.929 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T14:01:49.929 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T14:01:49.929 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:01:49.929 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:01:49.929 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:01:49.930 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:01:49.930 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary10 2024-09-12T14:01:50.031 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary10: 2024-09-12T14:01:50.032 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 3238adfd-24b7-48b2-ab32-66aaa1c7b583 2024-09-12T14:01:50.032 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+replaying 2024-09-12T14:01:50.032 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: replaying, {"bytes_per_second":0.0,"bytes_per_snapshot":62820352.0,"last_snapshot_bytes":62820352,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726149626,"remote_snapshot_timestamp":1726149626,"replay_state":"idle"} 2024-09-12T14:01:50.032 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T14:01:50.032 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 14:01:47' 2024-09-12T14:01:50.032 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+replaying' 2024-09-12T14:01:50.034 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+replaying 2024-09-12T14:01:50.034 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T14:01:50.037 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: replaying, {"bytes_per_second":0.0,"bytes_per_snapshot":62820352.0,"last_snapshot_bytes":62820352,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726149626,"remote_snapshot_timestamp":1726149626,"replay_state":"idle"} 2024-09-12T14:01:50.037 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T14:01:50.037 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+replaying 2024-09-12T14:01:50.038 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T14:01:50.040 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+replaying 2024-09-12T14:01:50.040 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T14:01:50.043 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T14:01:50.043 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T14:01:50.044 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T14:01:50.048 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 14:01:47' 2024-09-12T14:01:50.048 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary10 up+replaying '2024-09-12 14:01:47' 2024-09-12T14:01:50.048 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:01:50.048 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:01:50.048 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T14:01:50.048 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T14:01:50.048 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 14:01:47' 2024-09-12T14:01:50.048 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:01:50.049 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:01:50.049 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-09-12T14:01:50.167 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING141934232mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary103238adfd-24b7-48b2-ab32-66aaa1c7b583up+replayingreplaying, {"bytes_per_second":0.0,"bytes_per_snapshot":62820352.0,"last_snapshot_bytes":62820352,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726149626,"remote_snapshot_timestamp":1726149626,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 14:01:47' 2024-09-12T14:01:50.167 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T14:01:50.167 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary10'\'']/last_update' 2024-09-12T14:01:50.173 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 14:01:47' 2024-09-12T14:01:50.173 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary10'\'']/state' 2024-09-12T14:01:50.176 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+replaying 2024-09-12T14:01:50.176 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+replaying 2024-09-12T14:01:50.177 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+replaying 2024-09-12T14:01:50.179 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+replaying 2024-09-12T14:01:50.179 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:01:50.180 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:01:50.180 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ remove_image cluster2 mirror image-primary10 2024-09-12T14:01:50.180 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T14:01:50.180 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:01:50.180 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T14:01:50.180 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 snap purge mirror/image-primary10 2024-09-12T14:01:50.242 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 rm mirror/image-primary10 2024-09-12T14:01:50.489 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T14:01:50.489+0000 7f67e4800640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x56055b8cb040 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-09-12T14:01:50.489 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T14:01:50.489+0000 7f67eca00640 -1 rbd::mirror::ImageReplayer: 0x5605573cbb80 [3/3238adfd-24b7-48b2-ab32-66aaa1c7b583] operator(): start failed: (85) Interrupted system call should be restarted 2024-09-12T14:01:51.600 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-09-12T14:01:51.606 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_image_removal cluster1 mirror image-primary10 2024-09-12T14:01:51.606 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:01:51.606 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:01:51.606 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T14:01:51.607 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:01:51.607 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 ls mirror 2024-09-12T14:01:51.607 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep -wq image-primary10 2024-09-12T14:01:51.658 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:01:51.658 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rm -rf test-primary10 2024-09-12T14:01:51.661 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..10} 2024-09-12T14:01:51.661 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEVS=() 2024-09-12T14:01:51.662 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ SNAP_PIDS=() 2024-09-12T14:01:51.662 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ COMPARE_PIDS=() 2024-09-12T14:01:51.662 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ WORKLOAD_PIDS=() 2024-09-12T14:01:51.662 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ RET=0 2024-09-12T14:01:51.662 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T14:01:51.662 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ IMG=image-primary1 2024-09-12T14:01:51.662 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ MNTPT=test-primary1 2024-09-12T14:01:51.662 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ create_image_and_enable_mirror cluster1 mirror image-primary1 snapshot 10G 2024-09-12T14:01:51.662 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:01:51.662 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T14:01:51.662 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:01:51.662 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T14:01:51.662 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T14:01:51.663 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T14:01:51.663 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mode=snapshot 2024-09-12T14:01:51.663 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n snapshot ']' 2024-09-12T14:01:51.663 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T14:01:51.663 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ create_image cluster1 mirror image-primary1 10G 2024-09-12T14:01:51.663 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:01:51.663 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T14:01:51.663 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:01:51.663 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T14:01:51.663 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T14:01:51.663 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T14:01:51.663 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local size=128 2024-09-12T14:01:51.663 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n 10G ']' 2024-09-12T14:01:51.663 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ size=10G 2024-09-12T14:01:51.664 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T14:01:51.664 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 create --size 10G --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image-primary1 2024-09-12T14:01:51.736 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ is_pool_mirror_mode_image mirror 2024-09-12T14:01:51.736 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:01:51.736 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' image = image ']' 2024-09-12T14:01:51.736 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:01:51.737 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ enable_mirror cluster1 mirror image-primary1 snapshot 2024-09-12T14:01:51.737 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:01:51.737 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:01:51.737 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T14:01:51.737 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mode=snapshot 2024-09-12T14:01:51.737 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 mirror image enable mirror/image-primary1 snapshot 2024-09-12T14:01:52.534 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Mirroring enabled 2024-09-12T14:01:52.549 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 info mirror/image-primary1 2024-09-12T14:01:52.606 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:rbd image 'image-primary1': 2024-09-12T14:01:52.606 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: size 10 GiB in 2560 objects 2024-09-12T14:01:52.606 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: order 22 (4 MiB objects) 2024-09-12T14:01:52.607 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: snapshot_count: 1 2024-09-12T14:01:52.607 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: id: 381d3c0b09da 2024-09-12T14:01:52.607 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: block_name_prefix: rbd_data.381d3c0b09da 2024-09-12T14:01:52.607 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: format: 2 2024-09-12T14:01:52.607 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: features: layering, exclusive-lock, object-map, fast-diff 2024-09-12T14:01:52.607 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: op_features: 2024-09-12T14:01:52.607 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: flags: 2024-09-12T14:01:52.607 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: create_timestamp: Thu Sep 12 14:01:51 2024 2024-09-12T14:01:52.607 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: access_timestamp: Thu Sep 12 14:01:51 2024 2024-09-12T14:01:52.607 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: modify_timestamp: Thu Sep 12 14:01:51 2024 2024-09-12T14:01:52.607 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring state: enabled 2024-09-12T14:01:52.607 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring mode: snapshot 2024-09-12T14:01:52.607 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring global id: 9bf3eb7e-eb1b-464c-b684-743ee2313f71 2024-09-12T14:01:52.607 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring primary: true 2024-09-12T14:01:52.612 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \n\b\d ]] 2024-09-12T14:01:52.612 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \k\r\b\d ]] 2024-09-12T14:01:52.613 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo rbd --cluster cluster1 device map -t krbd mirror/image-primary1 2024-09-12T14:01:52.760 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEV=/dev/rbd0 2024-09-12T14:01:52.760 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEVS+=($DEV) 2024-09-12T14:01:52.760 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo mkfs.ext4 /dev/rbd0 2024-09-12T14:01:52.806 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:mke2fs 1.46.5 (30-Dec-2021) 2024-09-12T14:01:52.817 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Discarding device blocks: 0/2621440 done 2024-09-12T14:01:52.818 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Creating filesystem with 2621440 4k blocks and 655360 inodes 2024-09-12T14:01:52.818 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Filesystem UUID: e615edf0-4207-4c4a-8c11-1feae4d89c6e 2024-09-12T14:01:52.818 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Superblock backups stored on blocks: 2024-09-12T14:01:52.819 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 2024-09-12T14:01:52.819 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 2024-09-12T14:01:52.819 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Allocating group tables: 0/80 done 2024-09-12T14:01:52.819 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Writing inode tables: 0/80 done 2024-09-12T14:01:52.824 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Creating journal (16384 blocks): done 2024-09-12T14:01:52.885 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Writing superblocks and filesystem accounting information: 0/80 done 2024-09-12T14:01:52.885 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 2024-09-12T14:01:52.889 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mkdir test-primary1 2024-09-12T14:01:52.898 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo mount /dev/rbd0 test-primary1 2024-09-12T14:01:52.955 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ whoami 2024-09-12T14:01:52.967 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo chown ubuntu test-primary1 2024-09-12T14:01:52.991 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ SNAP_PIDS+=($!) 2024-09-12T14:01:52.991 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ take_mirror_snapshots cluster1 mirror image-primary1 2024-09-12T14:01:52.991 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:01:52.991 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:01:52.992 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T14:01:52.992 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:01:52.992 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T14:01:52.992 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:01:52.992 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:01:52.992 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T14:01:52.992 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T14:01:52.992 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ WORKLOAD_PIDS+=($!) 2024-09-12T14:01:52.992 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T14:01:52.992 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ slow_untar_workload test-primary1 2024-09-12T14:01:52.992 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ IMG=image-primary2 2024-09-12T14:01:52.992 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ MNTPT=test-primary2 2024-09-12T14:01:52.994 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ create_image_and_enable_mirror cluster1 mirror image-primary2 snapshot 10G 2024-09-12T14:01:52.994 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mountpt=test-primary1 2024-09-12T14:01:52.994 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ cp linux-5.4.tar.gz test-primary1 2024-09-12T14:01:52.994 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:01:52.994 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T14:01:52.994 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:01:52.994 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T14:01:52.994 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T14:01:52.994 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T14:01:52.994 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mode=snapshot 2024-09-12T14:01:52.994 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n snapshot ']' 2024-09-12T14:01:52.995 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T14:01:52.995 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ create_image cluster1 mirror image-primary2 10G 2024-09-12T14:01:52.995 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:01:52.995 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T14:01:52.995 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:01:52.995 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T14:01:52.995 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T14:01:52.995 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T14:01:52.995 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local size=128 2024-09-12T14:01:52.995 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n 10G ']' 2024-09-12T14:01:52.995 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ size=10G 2024-09-12T14:01:52.995 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T14:01:52.995 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 create --size 10G --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image-primary2 2024-09-12T14:01:53.067 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ is_pool_mirror_mode_image mirror 2024-09-12T14:01:53.067 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:01:53.067 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' image = image ']' 2024-09-12T14:01:53.067 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:01:53.068 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ enable_mirror cluster1 mirror image-primary2 snapshot 2024-09-12T14:01:53.068 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:01:53.068 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:01:53.068 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T14:01:53.068 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mode=snapshot 2024-09-12T14:01:53.068 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 mirror image enable mirror/image-primary2 snapshot 2024-09-12T14:01:53.278 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local ret=0 2024-09-12T14:01:53.278 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ timeout 5m bash -c 'zcat test-primary1/linux-5.4.tar.gz | pv -L 256K | tar xf - -C test-primary1' 2024-09-12T14:01:53.440 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Mirroring enabled 2024-09-12T14:01:53.447 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2644 2024-09-12T14:01:53.458 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:01:53.459 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 info mirror/image-primary2 2024-09-12T14:01:53.513 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:rbd image 'image-primary2': 2024-09-12T14:01:53.513 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: size 10 GiB in 2560 objects 2024-09-12T14:01:53.513 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: order 22 (4 MiB objects) 2024-09-12T14:01:53.513 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: snapshot_count: 1 2024-09-12T14:01:53.513 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: id: 382df4344840 2024-09-12T14:01:53.513 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: block_name_prefix: rbd_data.382df4344840 2024-09-12T14:01:53.513 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: format: 2 2024-09-12T14:01:53.514 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: features: layering, exclusive-lock, object-map, fast-diff 2024-09-12T14:01:53.514 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: op_features: 2024-09-12T14:01:53.514 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: flags: 2024-09-12T14:01:53.514 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: create_timestamp: Thu Sep 12 14:01:53 2024 2024-09-12T14:01:53.514 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: access_timestamp: Thu Sep 12 14:01:53 2024 2024-09-12T14:01:53.514 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: modify_timestamp: Thu Sep 12 14:01:53 2024 2024-09-12T14:01:53.514 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring state: enabled 2024-09-12T14:01:53.514 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring mode: snapshot 2024-09-12T14:01:53.514 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring global id: ccaafd72-518c-4588-86d7-96c322bcf1bd 2024-09-12T14:01:53.514 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring primary: true 2024-09-12T14:01:53.518 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \n\b\d ]] 2024-09-12T14:01:53.518 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \k\r\b\d ]] 2024-09-12T14:01:53.519 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo rbd --cluster cluster1 device map -t krbd mirror/image-primary2 2024-09-12T14:01:53.636 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEV=/dev/rbd1 2024-09-12T14:01:53.637 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEVS+=($DEV) 2024-09-12T14:01:53.637 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo mkfs.ext4 /dev/rbd1 2024-09-12T14:01:53.657 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:mke2fs 1.46.5 (30-Dec-2021) 2024-09-12T14:01:53.670 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Discarding device blocks: 0/2621440 done 2024-09-12T14:01:53.670 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Creating filesystem with 2621440 4k blocks and 655360 inodes 2024-09-12T14:01:53.670 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Filesystem UUID: fa83f004-b99f-4d76-8ea0-75fc0ee8a71a 2024-09-12T14:01:53.670 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Superblock backups stored on blocks: 2024-09-12T14:01:53.670 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 2024-09-12T14:01:53.670 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 2024-09-12T14:01:53.670 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Allocating group tables: 0/80 done 2024-09-12T14:01:53.670 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Writing inode tables: 0/80 done 2024-09-12T14:01:53.675 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Creating journal (16384 blocks): done 2024-09-12T14:01:53.733 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Writing superblocks and filesystem accounting information: 0/80 done 2024-09-12T14:01:53.733 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 2024-09-12T14:01:53.736 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mkdir test-primary2 2024-09-12T14:01:53.738 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo mount /dev/rbd1 test-primary2 2024-09-12T14:01:53.792 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ whoami 2024-09-12T14:01:53.794 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo chown ubuntu test-primary2 2024-09-12T14:01:53.819 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ SNAP_PIDS+=($!) 2024-09-12T14:01:53.819 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ take_mirror_snapshots cluster1 mirror image-primary2 2024-09-12T14:01:53.819 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:01:53.819 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:01:53.819 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T14:01:53.819 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:01:53.819 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T14:01:53.819 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:01:53.820 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:01:53.820 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T14:01:53.820 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T14:01:53.820 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ WORKLOAD_PIDS+=($!) 2024-09-12T14:01:53.820 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T14:01:53.820 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ slow_untar_workload test-primary2 2024-09-12T14:01:53.820 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ IMG=image-primary3 2024-09-12T14:01:53.820 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ MNTPT=test-primary3 2024-09-12T14:01:53.820 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ create_image_and_enable_mirror cluster1 mirror image-primary3 snapshot 10G 2024-09-12T14:01:53.820 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mountpt=test-primary2 2024-09-12T14:01:53.820 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:01:53.820 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ cp linux-5.4.tar.gz test-primary2 2024-09-12T14:01:53.820 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T14:01:53.820 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:01:53.821 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T14:01:53.821 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T14:01:53.821 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T14:01:53.821 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mode=snapshot 2024-09-12T14:01:53.821 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n snapshot ']' 2024-09-12T14:01:53.821 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T14:01:53.821 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ create_image cluster1 mirror image-primary3 10G 2024-09-12T14:01:53.821 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:01:53.821 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T14:01:53.821 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:01:53.821 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T14:01:53.821 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T14:01:53.822 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T14:01:53.822 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local size=128 2024-09-12T14:01:53.822 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n 10G ']' 2024-09-12T14:01:53.822 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ size=10G 2024-09-12T14:01:53.822 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T14:01:53.822 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 create --size 10G --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image-primary3 2024-09-12T14:01:53.892 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ is_pool_mirror_mode_image mirror 2024-09-12T14:01:53.892 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:01:53.892 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' image = image ']' 2024-09-12T14:01:53.892 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:01:53.892 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ enable_mirror cluster1 mirror image-primary3 snapshot 2024-09-12T14:01:53.892 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:01:53.892 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:01:53.892 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T14:01:53.892 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mode=snapshot 2024-09-12T14:01:53.892 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 mirror image enable mirror/image-primary3 snapshot 2024-09-12T14:01:54.105 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local ret=0 2024-09-12T14:01:54.106 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ timeout 5m bash -c 'zcat test-primary2/linux-5.4.tar.gz | pv -L 256K | tar xf - -C test-primary2' 2024-09-12T14:01:54.437 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Mirroring enabled 2024-09-12T14:01:54.442 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2646 2024-09-12T14:01:54.452 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 info mirror/image-primary3 2024-09-12T14:01:54.452 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:01:54.508 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:rbd image 'image-primary3': 2024-09-12T14:01:54.508 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: size 10 GiB in 2560 objects 2024-09-12T14:01:54.508 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: order 22 (4 MiB objects) 2024-09-12T14:01:54.509 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: snapshot_count: 1 2024-09-12T14:01:54.509 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: id: 383bb7a52697 2024-09-12T14:01:54.509 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: block_name_prefix: rbd_data.383bb7a52697 2024-09-12T14:01:54.509 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: format: 2 2024-09-12T14:01:54.509 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: features: layering, exclusive-lock, object-map, fast-diff 2024-09-12T14:01:54.509 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: op_features: 2024-09-12T14:01:54.509 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: flags: 2024-09-12T14:01:54.509 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: create_timestamp: Thu Sep 12 14:01:53 2024 2024-09-12T14:01:54.509 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: access_timestamp: Thu Sep 12 14:01:53 2024 2024-09-12T14:01:54.509 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: modify_timestamp: Thu Sep 12 14:01:53 2024 2024-09-12T14:01:54.509 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring state: enabled 2024-09-12T14:01:54.509 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring mode: snapshot 2024-09-12T14:01:54.509 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring global id: d356f103-adde-4c7a-855e-96419b23abef 2024-09-12T14:01:54.509 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring primary: true 2024-09-12T14:01:54.514 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \n\b\d ]] 2024-09-12T14:01:54.514 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \k\r\b\d ]] 2024-09-12T14:01:54.515 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo rbd --cluster cluster1 device map -t krbd mirror/image-primary3 2024-09-12T14:01:54.642 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEV=/dev/rbd2 2024-09-12T14:01:54.642 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEVS+=($DEV) 2024-09-12T14:01:54.643 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo mkfs.ext4 /dev/rbd2 2024-09-12T14:01:54.672 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:mke2fs 1.46.5 (30-Dec-2021) 2024-09-12T14:01:54.689 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Discarding device blocks: 0/2621440 done 2024-09-12T14:01:54.690 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Creating filesystem with 2621440 4k blocks and 655360 inodes 2024-09-12T14:01:54.690 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Filesystem UUID: ee2c246a-7c4f-4c7c-be1e-fe853efc61da 2024-09-12T14:01:54.690 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Superblock backups stored on blocks: 2024-09-12T14:01:54.690 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 2024-09-12T14:01:54.690 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 2024-09-12T14:01:54.690 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Allocating group tables: 0/80 done 2024-09-12T14:01:54.690 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Writing inode tables: 0/80 done 2024-09-12T14:01:54.695 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Creating journal (16384 blocks): done 2024-09-12T14:01:54.758 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Writing superblocks and filesystem accounting information: 0/80 done 2024-09-12T14:01:54.759 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 2024-09-12T14:01:54.761 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mkdir test-primary3 2024-09-12T14:01:54.763 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo mount /dev/rbd2 test-primary3 2024-09-12T14:01:54.820 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ whoami 2024-09-12T14:01:54.822 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo chown ubuntu test-primary3 2024-09-12T14:01:54.846 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ SNAP_PIDS+=($!) 2024-09-12T14:01:54.846 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ take_mirror_snapshots cluster1 mirror image-primary3 2024-09-12T14:01:54.846 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:01:54.846 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:01:54.846 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T14:01:54.846 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:01:54.847 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T14:01:54.847 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:01:54.847 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:01:54.847 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ WORKLOAD_PIDS+=($!) 2024-09-12T14:01:54.847 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T14:01:54.847 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T14:01:54.847 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ slow_untar_workload test-primary3 2024-09-12T14:01:54.847 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T14:01:54.847 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ IMG=image-primary4 2024-09-12T14:01:54.847 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ MNTPT=test-primary4 2024-09-12T14:01:54.847 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ create_image_and_enable_mirror cluster1 mirror image-primary4 snapshot 10G 2024-09-12T14:01:54.849 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mountpt=test-primary3 2024-09-12T14:01:54.849 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ cp linux-5.4.tar.gz test-primary3 2024-09-12T14:01:54.849 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:01:54.849 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T14:01:54.849 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:01:54.849 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T14:01:54.849 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T14:01:54.849 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T14:01:54.849 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mode=snapshot 2024-09-12T14:01:54.849 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n snapshot ']' 2024-09-12T14:01:54.849 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T14:01:54.849 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ create_image cluster1 mirror image-primary4 10G 2024-09-12T14:01:54.850 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:01:54.850 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T14:01:54.850 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:01:54.850 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T14:01:54.850 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T14:01:54.850 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T14:01:54.850 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local size=128 2024-09-12T14:01:54.850 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n 10G ']' 2024-09-12T14:01:54.850 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ size=10G 2024-09-12T14:01:54.850 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T14:01:54.850 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 create --size 10G --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image-primary4 2024-09-12T14:01:54.920 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ is_pool_mirror_mode_image mirror 2024-09-12T14:01:54.920 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:01:54.921 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' image = image ']' 2024-09-12T14:01:54.921 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:01:54.921 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ enable_mirror cluster1 mirror image-primary4 snapshot 2024-09-12T14:01:54.921 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:01:54.921 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:01:54.921 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T14:01:54.921 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mode=snapshot 2024-09-12T14:01:54.921 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 mirror image enable mirror/image-primary4 snapshot 2024-09-12T14:01:55.133 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local ret=0 2024-09-12T14:01:55.134 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ timeout 5m bash -c 'zcat test-primary3/linux-5.4.tar.gz | pv -L 256K | tar xf - -C test-primary3' 2024-09-12T14:01:55.457 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Mirroring enabled 2024-09-12T14:01:55.458 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2648 2024-09-12T14:01:55.470 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:01:55.473 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 info mirror/image-primary4 2024-09-12T14:01:55.533 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:rbd image 'image-primary4': 2024-09-12T14:01:55.534 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: size 10 GiB in 2560 objects 2024-09-12T14:01:55.534 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: order 22 (4 MiB objects) 2024-09-12T14:01:55.534 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: snapshot_count: 1 2024-09-12T14:01:55.534 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: id: 384ad2daf5f3 2024-09-12T14:01:55.534 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: block_name_prefix: rbd_data.384ad2daf5f3 2024-09-12T14:01:55.534 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: format: 2 2024-09-12T14:01:55.534 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: features: layering, exclusive-lock, object-map, fast-diff 2024-09-12T14:01:55.534 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: op_features: 2024-09-12T14:01:55.534 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: flags: 2024-09-12T14:01:55.534 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: create_timestamp: Thu Sep 12 14:01:54 2024 2024-09-12T14:01:55.535 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: access_timestamp: Thu Sep 12 14:01:54 2024 2024-09-12T14:01:55.535 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: modify_timestamp: Thu Sep 12 14:01:54 2024 2024-09-12T14:01:55.535 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring state: enabled 2024-09-12T14:01:55.535 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring mode: snapshot 2024-09-12T14:01:55.535 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring global id: a6cc92cf-b177-4aea-95c5-b61892fd9062 2024-09-12T14:01:55.535 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring primary: true 2024-09-12T14:01:55.539 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \n\b\d ]] 2024-09-12T14:01:55.539 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \k\r\b\d ]] 2024-09-12T14:01:55.541 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo rbd --cluster cluster1 device map -t krbd mirror/image-primary4 2024-09-12T14:01:55.666 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEV=/dev/rbd3 2024-09-12T14:01:55.666 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEVS+=($DEV) 2024-09-12T14:01:55.666 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo mkfs.ext4 /dev/rbd3 2024-09-12T14:01:55.686 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:mke2fs 1.46.5 (30-Dec-2021) 2024-09-12T14:01:55.701 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Discarding device blocks: 0/2621440 done 2024-09-12T14:01:55.701 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Creating filesystem with 2621440 4k blocks and 655360 inodes 2024-09-12T14:01:55.701 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Filesystem UUID: 620d7469-7599-46ba-897a-45e9b8f4871c 2024-09-12T14:01:55.701 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Superblock backups stored on blocks: 2024-09-12T14:01:55.701 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 2024-09-12T14:01:55.701 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 2024-09-12T14:01:55.701 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Allocating group tables: 0/80 done 2024-09-12T14:01:55.701 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Writing inode tables: 0/80 done 2024-09-12T14:01:55.706 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Creating journal (16384 blocks): done 2024-09-12T14:01:55.764 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Writing superblocks and filesystem accounting information: 0/80 done 2024-09-12T14:01:55.764 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 2024-09-12T14:01:55.767 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mkdir test-primary4 2024-09-12T14:01:55.769 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo mount /dev/rbd3 test-primary4 2024-09-12T14:01:55.825 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ whoami 2024-09-12T14:01:55.827 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo chown ubuntu test-primary4 2024-09-12T14:01:55.851 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ SNAP_PIDS+=($!) 2024-09-12T14:01:55.851 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ take_mirror_snapshots cluster1 mirror image-primary4 2024-09-12T14:01:55.851 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:01:55.851 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:01:55.851 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T14:01:55.852 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:01:55.852 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T14:01:55.852 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:01:55.852 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ WORKLOAD_PIDS+=($!) 2024-09-12T14:01:55.852 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:01:55.852 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T14:01:55.852 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T14:01:55.852 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T14:01:55.852 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ IMG=image-primary5 2024-09-12T14:01:55.852 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ MNTPT=test-primary5 2024-09-12T14:01:55.852 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ create_image_and_enable_mirror cluster1 mirror image-primary5 snapshot 10G 2024-09-12T14:01:55.852 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ slow_untar_workload test-primary4 2024-09-12T14:01:55.853 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:01:55.854 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T14:01:55.854 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:01:55.854 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T14:01:55.854 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T14:01:55.854 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T14:01:55.854 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mode=snapshot 2024-09-12T14:01:55.854 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n snapshot ']' 2024-09-12T14:01:55.854 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T14:01:55.854 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ create_image cluster1 mirror image-primary5 10G 2024-09-12T14:01:55.854 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:01:55.854 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mountpt=test-primary4 2024-09-12T14:01:55.855 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T14:01:55.855 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:01:55.855 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T14:01:55.855 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T14:01:55.855 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T14:01:55.855 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local size=128 2024-09-12T14:01:55.855 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ cp linux-5.4.tar.gz test-primary4 2024-09-12T14:01:55.855 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n 10G ']' 2024-09-12T14:01:55.855 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ size=10G 2024-09-12T14:01:55.855 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T14:01:55.855 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 create --size 10G --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image-primary5 2024-09-12T14:01:55.924 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ is_pool_mirror_mode_image mirror 2024-09-12T14:01:55.924 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:01:55.924 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' image = image ']' 2024-09-12T14:01:55.924 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:01:55.924 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ enable_mirror cluster1 mirror image-primary5 snapshot 2024-09-12T14:01:55.924 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:01:55.924 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:01:55.924 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T14:01:55.924 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mode=snapshot 2024-09-12T14:01:55.924 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 mirror image enable mirror/image-primary5 snapshot 2024-09-12T14:01:56.138 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local ret=0 2024-09-12T14:01:56.138 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ timeout 5m bash -c 'zcat test-primary4/linux-5.4.tar.gz | pv -L 256K | tar xf - -C test-primary4' 2024-09-12T14:01:56.457 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Mirroring enabled 2024-09-12T14:01:56.461 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2650 2024-09-12T14:01:56.462 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:01:56.462 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T14:01:56.462 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:01:56.462 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:01:56.463 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T14:01:56.463 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T14:01:56.484 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 info mirror/image-primary5 2024-09-12T14:01:56.484 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:01:56.591 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:rbd image 'image-primary5': 2024-09-12T14:01:56.592 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: size 10 GiB in 2560 objects 2024-09-12T14:01:56.592 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: order 22 (4 MiB objects) 2024-09-12T14:01:56.592 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: snapshot_count: 1 2024-09-12T14:01:56.592 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: id: 3859f5490c54 2024-09-12T14:01:56.592 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: block_name_prefix: rbd_data.3859f5490c54 2024-09-12T14:01:56.592 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: format: 2 2024-09-12T14:01:56.592 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: features: layering, exclusive-lock, object-map, fast-diff 2024-09-12T14:01:56.592 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: op_features: 2024-09-12T14:01:56.592 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: flags: 2024-09-12T14:01:56.593 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: create_timestamp: Thu Sep 12 14:01:55 2024 2024-09-12T14:01:56.593 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: access_timestamp: Thu Sep 12 14:01:55 2024 2024-09-12T14:01:56.593 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: modify_timestamp: Thu Sep 12 14:01:55 2024 2024-09-12T14:01:56.593 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring state: enabled 2024-09-12T14:01:56.593 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring mode: snapshot 2024-09-12T14:01:56.593 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring global id: 399c5b9a-89f0-4680-96c5-f2cac59115cf 2024-09-12T14:01:56.593 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring primary: true 2024-09-12T14:01:56.602 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \n\b\d ]] 2024-09-12T14:01:56.602 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \k\r\b\d ]] 2024-09-12T14:01:56.603 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo rbd --cluster cluster1 device map -t krbd mirror/image-primary5 2024-09-12T14:01:56.734 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEV=/dev/rbd4 2024-09-12T14:01:56.734 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEVS+=($DEV) 2024-09-12T14:01:56.735 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo mkfs.ext4 /dev/rbd4 2024-09-12T14:01:56.755 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:mke2fs 1.46.5 (30-Dec-2021) 2024-09-12T14:01:56.766 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Discarding device blocks: 0/2621440 done 2024-09-12T14:01:56.767 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Creating filesystem with 2621440 4k blocks and 655360 inodes 2024-09-12T14:01:56.767 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Filesystem UUID: 893d58b3-738c-4245-9265-5a40cb16ee6e 2024-09-12T14:01:56.767 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Superblock backups stored on blocks: 2024-09-12T14:01:56.767 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 2024-09-12T14:01:56.767 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 2024-09-12T14:01:56.767 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Allocating group tables: 0/80 done 2024-09-12T14:01:56.767 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Writing inode tables: 0/80 done 2024-09-12T14:01:56.772 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Creating journal (16384 blocks): done 2024-09-12T14:01:56.827 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Writing superblocks and filesystem accounting information: 0/80 done 2024-09-12T14:01:56.827 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 2024-09-12T14:01:56.829 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mkdir test-primary5 2024-09-12T14:01:56.832 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo mount /dev/rbd4 test-primary5 2024-09-12T14:01:56.892 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ whoami 2024-09-12T14:01:56.894 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo chown ubuntu test-primary5 2024-09-12T14:01:56.919 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ SNAP_PIDS+=($!) 2024-09-12T14:01:56.919 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ take_mirror_snapshots cluster1 mirror image-primary5 2024-09-12T14:01:56.920 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:01:56.920 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:01:56.920 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T14:01:56.920 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:01:56.920 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T14:01:56.920 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:01:56.921 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:01:56.921 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ WORKLOAD_PIDS+=($!) 2024-09-12T14:01:56.921 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T14:01:56.921 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T14:01:56.921 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T14:01:56.921 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ IMG=image-primary6 2024-09-12T14:01:56.921 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ MNTPT=test-primary6 2024-09-12T14:01:56.921 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ slow_untar_workload test-primary5 2024-09-12T14:01:56.921 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ create_image_and_enable_mirror cluster1 mirror image-primary6 snapshot 10G 2024-09-12T14:01:56.921 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:01:56.921 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T14:01:56.921 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:01:56.921 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T14:01:56.922 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T14:01:56.922 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T14:01:56.922 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mode=snapshot 2024-09-12T14:01:56.922 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n snapshot ']' 2024-09-12T14:01:56.922 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T14:01:56.922 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mountpt=test-primary5 2024-09-12T14:01:56.922 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ create_image cluster1 mirror image-primary6 10G 2024-09-12T14:01:56.922 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ cp linux-5.4.tar.gz test-primary5 2024-09-12T14:01:56.922 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:01:56.922 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T14:01:56.922 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:01:56.923 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T14:01:56.923 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T14:01:56.923 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T14:01:56.923 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local size=128 2024-09-12T14:01:56.923 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n 10G ']' 2024-09-12T14:01:56.923 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ size=10G 2024-09-12T14:01:56.923 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T14:01:56.923 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 create --size 10G --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image-primary6 2024-09-12T14:01:56.995 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ is_pool_mirror_mode_image mirror 2024-09-12T14:01:56.995 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:01:56.995 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' image = image ']' 2024-09-12T14:01:56.995 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:01:56.996 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ enable_mirror cluster1 mirror image-primary6 snapshot 2024-09-12T14:01:56.996 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:01:56.996 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:01:56.996 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T14:01:56.996 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mode=snapshot 2024-09-12T14:01:56.996 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 mirror image enable mirror/image-primary6 snapshot 2024-09-12T14:01:57.212 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local ret=0 2024-09-12T14:01:57.212 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ timeout 5m bash -c 'zcat test-primary5/linux-5.4.tar.gz | pv -L 256K | tar xf - -C test-primary5' 2024-09-12T14:01:57.455 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:01:57.455 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T14:01:57.455 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:01:57.455 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:01:57.455 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T14:01:57.455 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T14:01:57.470 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Mirroring enabled 2024-09-12T14:01:57.496 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2652 2024-09-12T14:01:57.504 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2653 2024-09-12T14:01:57.511 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:01:57.518 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 info mirror/image-primary6 2024-09-12T14:01:57.519 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:01:57.610 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:rbd image 'image-primary6': 2024-09-12T14:01:57.610 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: size 10 GiB in 2560 objects 2024-09-12T14:01:57.610 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: order 22 (4 MiB objects) 2024-09-12T14:01:57.610 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: snapshot_count: 1 2024-09-12T14:01:57.610 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: id: 386a5838c8e1 2024-09-12T14:01:57.610 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: block_name_prefix: rbd_data.386a5838c8e1 2024-09-12T14:01:57.610 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: format: 2 2024-09-12T14:01:57.610 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: features: layering, exclusive-lock, object-map, fast-diff 2024-09-12T14:01:57.611 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: op_features: 2024-09-12T14:01:57.611 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: flags: 2024-09-12T14:01:57.611 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: create_timestamp: Thu Sep 12 14:01:56 2024 2024-09-12T14:01:57.611 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: access_timestamp: Thu Sep 12 14:01:56 2024 2024-09-12T14:01:57.611 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: modify_timestamp: Thu Sep 12 14:01:56 2024 2024-09-12T14:01:57.611 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring state: enabled 2024-09-12T14:01:57.611 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring mode: snapshot 2024-09-12T14:01:57.611 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring global id: f8d03c2d-535f-4038-a91f-c159711e8e6c 2024-09-12T14:01:57.611 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring primary: true 2024-09-12T14:01:57.616 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \n\b\d ]] 2024-09-12T14:01:57.616 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \k\r\b\d ]] 2024-09-12T14:01:57.617 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo rbd --cluster cluster1 device map -t krbd mirror/image-primary6 2024-09-12T14:01:57.775 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEV=/dev/rbd5 2024-09-12T14:01:57.775 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEVS+=($DEV) 2024-09-12T14:01:57.776 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo mkfs.ext4 /dev/rbd5 2024-09-12T14:01:57.796 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:mke2fs 1.46.5 (30-Dec-2021) 2024-09-12T14:01:57.807 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Discarding device blocks: 0/2621440 done 2024-09-12T14:01:57.807 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Creating filesystem with 2621440 4k blocks and 655360 inodes 2024-09-12T14:01:57.807 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Filesystem UUID: 574a33a4-b3d0-4918-98d5-82b11d2f7ea5 2024-09-12T14:01:57.807 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Superblock backups stored on blocks: 2024-09-12T14:01:57.807 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 2024-09-12T14:01:57.808 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 2024-09-12T14:01:57.808 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Allocating group tables: 0/80 done 2024-09-12T14:01:57.808 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Writing inode tables: 0/80 done 2024-09-12T14:01:57.813 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Creating journal (16384 blocks): done 2024-09-12T14:01:57.924 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Writing superblocks and filesystem accounting information: 0/80 done 2024-09-12T14:01:57.924 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 2024-09-12T14:01:57.927 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mkdir test-primary6 2024-09-12T14:01:57.930 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo mount /dev/rbd5 test-primary6 2024-09-12T14:01:58.013 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ whoami 2024-09-12T14:01:58.014 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo chown ubuntu test-primary6 2024-09-12T14:01:58.040 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ SNAP_PIDS+=($!) 2024-09-12T14:01:58.040 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ take_mirror_snapshots cluster1 mirror image-primary6 2024-09-12T14:01:58.040 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:01:58.040 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:01:58.041 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T14:01:58.041 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:01:58.041 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T14:01:58.041 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:01:58.041 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:01:58.041 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T14:01:58.041 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T14:01:58.041 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ WORKLOAD_PIDS+=($!) 2024-09-12T14:01:58.041 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T14:01:58.041 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ slow_untar_workload test-primary6 2024-09-12T14:01:58.041 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ IMG=image-primary7 2024-09-12T14:01:58.041 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ MNTPT=test-primary7 2024-09-12T14:01:58.041 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ create_image_and_enable_mirror cluster1 mirror image-primary7 snapshot 10G 2024-09-12T14:01:58.042 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mountpt=test-primary6 2024-09-12T14:01:58.042 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ cp linux-5.4.tar.gz test-primary6 2024-09-12T14:01:58.042 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:01:58.042 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T14:01:58.042 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:01:58.042 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T14:01:58.042 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T14:01:58.042 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T14:01:58.042 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mode=snapshot 2024-09-12T14:01:58.042 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n snapshot ']' 2024-09-12T14:01:58.042 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T14:01:58.043 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ create_image cluster1 mirror image-primary7 10G 2024-09-12T14:01:58.043 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:01:58.043 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T14:01:58.043 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:01:58.043 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T14:01:58.043 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T14:01:58.043 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T14:01:58.043 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local size=128 2024-09-12T14:01:58.043 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n 10G ']' 2024-09-12T14:01:58.043 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ size=10G 2024-09-12T14:01:58.043 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T14:01:58.043 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 create --size 10G --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image-primary7 2024-09-12T14:01:58.148 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ is_pool_mirror_mode_image mirror 2024-09-12T14:01:58.149 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:01:58.149 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' image = image ']' 2024-09-12T14:01:58.149 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:01:58.149 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ enable_mirror cluster1 mirror image-primary7 snapshot 2024-09-12T14:01:58.149 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:01:58.149 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:01:58.149 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T14:01:58.149 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mode=snapshot 2024-09-12T14:01:58.149 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 mirror image enable mirror/image-primary7 snapshot 2024-09-12T14:01:58.349 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local ret=0 2024-09-12T14:01:58.349 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ timeout 5m bash -c 'zcat test-primary6/linux-5.4.tar.gz | pv -L 256K | tar xf - -C test-primary6' 2024-09-12T14:01:58.474 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:01:58.474 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T14:01:58.474 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:01:58.474 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:01:58.475 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T14:01:58.475 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T14:01:58.548 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Mirroring enabled 2024-09-12T14:01:58.572 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2655 2024-09-12T14:01:58.572 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2656 2024-09-12T14:01:58.588 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:01:58.595 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 info mirror/image-primary7 2024-09-12T14:01:58.602 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:01:58.678 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:rbd image 'image-primary7': 2024-09-12T14:01:58.678 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: size 10 GiB in 2560 objects 2024-09-12T14:01:58.678 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: order 22 (4 MiB objects) 2024-09-12T14:01:58.678 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: snapshot_count: 1 2024-09-12T14:01:58.678 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: id: 387c36bcce19 2024-09-12T14:01:58.678 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: block_name_prefix: rbd_data.387c36bcce19 2024-09-12T14:01:58.679 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: format: 2 2024-09-12T14:01:58.679 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: features: layering, exclusive-lock, object-map, fast-diff 2024-09-12T14:01:58.679 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: op_features: 2024-09-12T14:01:58.679 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: flags: 2024-09-12T14:01:58.679 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: create_timestamp: Thu Sep 12 14:01:58 2024 2024-09-12T14:01:58.679 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: access_timestamp: Thu Sep 12 14:01:58 2024 2024-09-12T14:01:58.679 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: modify_timestamp: Thu Sep 12 14:01:58 2024 2024-09-12T14:01:58.679 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring state: enabled 2024-09-12T14:01:58.679 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring mode: snapshot 2024-09-12T14:01:58.679 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring global id: 5795f6f3-43c3-4e14-89d9-f4798e240cc6 2024-09-12T14:01:58.679 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring primary: true 2024-09-12T14:01:58.683 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \n\b\d ]] 2024-09-12T14:01:58.683 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \k\r\b\d ]] 2024-09-12T14:01:58.684 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo rbd --cluster cluster1 device map -t krbd mirror/image-primary7 2024-09-12T14:01:58.944 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEV=/dev/rbd6 2024-09-12T14:01:58.944 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEVS+=($DEV) 2024-09-12T14:01:58.944 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo mkfs.ext4 /dev/rbd6 2024-09-12T14:01:58.965 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:mke2fs 1.46.5 (30-Dec-2021) 2024-09-12T14:01:58.977 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Discarding device blocks: 0/2621440 done 2024-09-12T14:01:58.978 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Creating filesystem with 2621440 4k blocks and 655360 inodes 2024-09-12T14:01:58.978 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Filesystem UUID: 2ce674a1-17c2-422f-b386-66c0874217c8 2024-09-12T14:01:58.978 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Superblock backups stored on blocks: 2024-09-12T14:01:58.978 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 2024-09-12T14:01:58.978 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 2024-09-12T14:01:58.978 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Allocating group tables: 0/80 done 2024-09-12T14:01:58.978 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Writing inode tables: 0/80 done 2024-09-12T14:01:58.984 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Creating journal (16384 blocks): done 2024-09-12T14:01:59.086 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Writing superblocks and filesystem accounting information: 0/80 done 2024-09-12T14:01:59.086 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 2024-09-12T14:01:59.089 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mkdir test-primary7 2024-09-12T14:01:59.091 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo mount /dev/rbd6 test-primary7 2024-09-12T14:01:59.181 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ whoami 2024-09-12T14:01:59.183 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo chown ubuntu test-primary7 2024-09-12T14:01:59.207 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ SNAP_PIDS+=($!) 2024-09-12T14:01:59.207 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ take_mirror_snapshots cluster1 mirror image-primary7 2024-09-12T14:01:59.207 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:01:59.208 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:01:59.208 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T14:01:59.208 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:01:59.208 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T14:01:59.208 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ WORKLOAD_PIDS+=($!) 2024-09-12T14:01:59.208 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:01:59.208 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:01:59.208 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T14:01:59.208 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T14:01:59.208 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T14:01:59.208 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ IMG=image-primary8 2024-09-12T14:01:59.210 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ MNTPT=test-primary8 2024-09-12T14:01:59.210 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ slow_untar_workload test-primary7 2024-09-12T14:01:59.210 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ create_image_and_enable_mirror cluster1 mirror image-primary8 snapshot 10G 2024-09-12T14:01:59.210 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:01:59.210 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T14:01:59.210 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:01:59.210 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T14:01:59.210 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T14:01:59.210 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T14:01:59.210 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mode=snapshot 2024-09-12T14:01:59.210 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n snapshot ']' 2024-09-12T14:01:59.211 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T14:01:59.211 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ create_image cluster1 mirror image-primary8 10G 2024-09-12T14:01:59.211 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mountpt=test-primary7 2024-09-12T14:01:59.211 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:01:59.211 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T14:01:59.211 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:01:59.211 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T14:01:59.211 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T14:01:59.211 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T14:01:59.211 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local size=128 2024-09-12T14:01:59.211 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n 10G ']' 2024-09-12T14:01:59.211 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ cp linux-5.4.tar.gz test-primary7 2024-09-12T14:01:59.212 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ size=10G 2024-09-12T14:01:59.212 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T14:01:59.212 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 create --size 10G --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image-primary8 2024-09-12T14:01:59.320 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ is_pool_mirror_mode_image mirror 2024-09-12T14:01:59.320 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:01:59.320 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' image = image ']' 2024-09-12T14:01:59.320 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:01:59.320 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ enable_mirror cluster1 mirror image-primary8 snapshot 2024-09-12T14:01:59.320 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:01:59.320 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:01:59.320 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T14:01:59.321 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mode=snapshot 2024-09-12T14:01:59.321 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 mirror image enable mirror/image-primary8 snapshot 2024-09-12T14:01:59.488 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:01:59.489 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T14:01:59.489 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:01:59.489 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:01:59.489 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T14:01:59.489 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T14:01:59.502 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2659 2024-09-12T14:01:59.536 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2658 2024-09-12T14:01:59.559 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:01:59.582 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:01:59.680 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local ret=0 2024-09-12T14:01:59.681 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ timeout 5m bash -c 'zcat test-primary7/linux-5.4.tar.gz | pv -L 256K | tar xf - -C test-primary7' 2024-09-12T14:02:00.457 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Mirroring enabled 2024-09-12T14:02:00.471 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2661 2024-09-12T14:02:00.487 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 info mirror/image-primary8 2024-09-12T14:02:00.490 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:02:00.515 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:02:00.515 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T14:02:00.515 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:02:00.515 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:02:00.515 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T14:02:00.515 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T14:02:00.522 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:02:00.522 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T14:02:00.522 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:02:00.522 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:02:00.522 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T14:02:00.522 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T14:02:00.598 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:rbd image 'image-primary8': 2024-09-12T14:02:00.598 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: size 10 GiB in 2560 objects 2024-09-12T14:02:00.598 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: order 22 (4 MiB objects) 2024-09-12T14:02:00.598 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: snapshot_count: 1 2024-09-12T14:02:00.598 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: id: 388b83bad88e 2024-09-12T14:02:00.599 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: block_name_prefix: rbd_data.388b83bad88e 2024-09-12T14:02:00.599 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: format: 2 2024-09-12T14:02:00.599 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: features: layering, exclusive-lock, object-map, fast-diff 2024-09-12T14:02:00.599 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: op_features: 2024-09-12T14:02:00.599 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: flags: 2024-09-12T14:02:00.599 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: create_timestamp: Thu Sep 12 14:01:59 2024 2024-09-12T14:02:00.599 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: access_timestamp: Thu Sep 12 14:01:59 2024 2024-09-12T14:02:00.599 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: modify_timestamp: Thu Sep 12 14:01:59 2024 2024-09-12T14:02:00.599 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring state: enabled 2024-09-12T14:02:00.599 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring mode: snapshot 2024-09-12T14:02:00.599 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring global id: fa74ee32-e225-4343-82da-f93063849113 2024-09-12T14:02:00.599 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring primary: true 2024-09-12T14:02:00.615 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \n\b\d ]] 2024-09-12T14:02:00.615 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \k\r\b\d ]] 2024-09-12T14:02:00.621 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo rbd --cluster cluster1 device map -t krbd mirror/image-primary8 2024-09-12T14:02:00.779 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEV=/dev/rbd7 2024-09-12T14:02:00.779 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEVS+=($DEV) 2024-09-12T14:02:00.779 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo mkfs.ext4 /dev/rbd7 2024-09-12T14:02:00.799 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:mke2fs 1.46.5 (30-Dec-2021) 2024-09-12T14:02:00.814 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Discarding device blocks: 0/2621440 done 2024-09-12T14:02:00.814 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Creating filesystem with 2621440 4k blocks and 655360 inodes 2024-09-12T14:02:00.814 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Filesystem UUID: 2fae3840-025a-41d0-a100-43a53bdcfb3d 2024-09-12T14:02:00.814 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Superblock backups stored on blocks: 2024-09-12T14:02:00.814 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 2024-09-12T14:02:00.814 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 2024-09-12T14:02:00.814 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Allocating group tables: 0/80 done 2024-09-12T14:02:00.815 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Writing inode tables: 0/80 done 2024-09-12T14:02:00.819 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Creating journal (16384 blocks): done 2024-09-12T14:02:00.990 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Writing superblocks and filesystem accounting information: 0/80 done 2024-09-12T14:02:00.990 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 2024-09-12T14:02:00.993 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mkdir test-primary8 2024-09-12T14:02:00.995 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo mount /dev/rbd7 test-primary8 2024-09-12T14:02:01.052 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ whoami 2024-09-12T14:02:01.054 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo chown ubuntu test-primary8 2024-09-12T14:02:01.079 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ SNAP_PIDS+=($!) 2024-09-12T14:02:01.079 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ take_mirror_snapshots cluster1 mirror image-primary8 2024-09-12T14:02:01.079 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ WORKLOAD_PIDS+=($!) 2024-09-12T14:02:01.079 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:02:01.079 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T14:02:01.079 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:02:01.079 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T14:02:01.080 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ IMG=image-primary9 2024-09-12T14:02:01.080 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ MNTPT=test-primary9 2024-09-12T14:02:01.080 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ slow_untar_workload test-primary8 2024-09-12T14:02:01.080 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:02:01.080 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T14:02:01.080 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ create_image_and_enable_mirror cluster1 mirror image-primary9 snapshot 10G 2024-09-12T14:02:01.080 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:02:01.080 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:02:01.080 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:02:01.080 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T14:02:01.080 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T14:02:01.080 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:02:01.080 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T14:02:01.081 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T14:02:01.081 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T14:02:01.081 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mode=snapshot 2024-09-12T14:02:01.081 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T14:02:01.081 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n snapshot ']' 2024-09-12T14:02:01.081 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T14:02:01.081 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ create_image cluster1 mirror image-primary9 10G 2024-09-12T14:02:01.081 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mountpt=test-primary8 2024-09-12T14:02:01.081 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ cp linux-5.4.tar.gz test-primary8 2024-09-12T14:02:01.081 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:02:01.082 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T14:02:01.082 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:02:01.082 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T14:02:01.082 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T14:02:01.082 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T14:02:01.082 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local size=128 2024-09-12T14:02:01.082 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n 10G ']' 2024-09-12T14:02:01.082 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ size=10G 2024-09-12T14:02:01.082 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T14:02:01.082 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 create --size 10G --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image-primary9 2024-09-12T14:02:01.150 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ is_pool_mirror_mode_image mirror 2024-09-12T14:02:01.150 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:02:01.150 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' image = image ']' 2024-09-12T14:02:01.150 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:02:01.150 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ enable_mirror cluster1 mirror image-primary9 snapshot 2024-09-12T14:02:01.151 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:02:01.151 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:02:01.151 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T14:02:01.151 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mode=snapshot 2024-09-12T14:02:01.151 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 mirror image enable mirror/image-primary9 snapshot 2024-09-12T14:02:01.367 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local ret=0 2024-09-12T14:02:01.367 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ timeout 5m bash -c 'zcat test-primary8/linux-5.4.tar.gz | pv -L 256K | tar xf - -C test-primary8' 2024-09-12T14:02:01.469 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2663 2024-09-12T14:02:01.476 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2664 2024-09-12T14:02:01.482 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Mirroring enabled 2024-09-12T14:02:01.486 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2662 2024-09-12T14:02:01.494 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:02:01.510 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 info mirror/image-primary9 2024-09-12T14:02:01.514 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:02:01.516 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:02:01.590 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:02:01.592 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T14:02:01.592 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:02:01.593 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:02:01.593 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T14:02:01.593 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T14:02:01.605 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:02:01.605 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T14:02:01.605 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:02:01.605 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:02:01.605 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T14:02:01.605 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T14:02:01.610 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:rbd image 'image-primary9': 2024-09-12T14:02:01.610 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: size 10 GiB in 2560 objects 2024-09-12T14:02:01.610 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: order 22 (4 MiB objects) 2024-09-12T14:02:01.610 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: snapshot_count: 1 2024-09-12T14:02:01.610 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: id: 38a054fc692f 2024-09-12T14:02:01.610 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: block_name_prefix: rbd_data.38a054fc692f 2024-09-12T14:02:01.610 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: format: 2 2024-09-12T14:02:01.610 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: features: layering, exclusive-lock, object-map, fast-diff 2024-09-12T14:02:01.610 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: op_features: 2024-09-12T14:02:01.611 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: flags: 2024-09-12T14:02:01.611 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: create_timestamp: Thu Sep 12 14:02:01 2024 2024-09-12T14:02:01.611 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: access_timestamp: Thu Sep 12 14:02:01 2024 2024-09-12T14:02:01.611 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: modify_timestamp: Thu Sep 12 14:02:01 2024 2024-09-12T14:02:01.611 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring state: enabled 2024-09-12T14:02:01.611 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring mode: snapshot 2024-09-12T14:02:01.611 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring global id: 5991a19a-9e33-46ee-965b-85f990022e98 2024-09-12T14:02:01.611 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring primary: true 2024-09-12T14:02:01.631 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \n\b\d ]] 2024-09-12T14:02:01.631 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \k\r\b\d ]] 2024-09-12T14:02:01.633 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo rbd --cluster cluster1 device map -t krbd mirror/image-primary9 2024-09-12T14:02:01.789 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEV=/dev/rbd8 2024-09-12T14:02:01.790 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEVS+=($DEV) 2024-09-12T14:02:01.790 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo mkfs.ext4 /dev/rbd8 2024-09-12T14:02:01.810 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:mke2fs 1.46.5 (30-Dec-2021) 2024-09-12T14:02:01.823 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Discarding device blocks: 0/2621440 done 2024-09-12T14:02:01.824 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Creating filesystem with 2621440 4k blocks and 655360 inodes 2024-09-12T14:02:01.824 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Filesystem UUID: a9260072-5b2a-4c3c-8fa3-ea9f2cb2f10e 2024-09-12T14:02:01.824 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Superblock backups stored on blocks: 2024-09-12T14:02:01.824 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 2024-09-12T14:02:01.824 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 2024-09-12T14:02:01.824 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Allocating group tables: 0/80 done 2024-09-12T14:02:01.824 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Writing inode tables: 0/80 done 2024-09-12T14:02:01.829 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Creating journal (16384 blocks): done 2024-09-12T14:02:01.988 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Writing superblocks and filesystem accounting information: 0/80 done 2024-09-12T14:02:01.988 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 2024-09-12T14:02:01.991 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mkdir test-primary9 2024-09-12T14:02:01.994 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo mount /dev/rbd8 test-primary9 2024-09-12T14:02:02.053 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ whoami 2024-09-12T14:02:02.055 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo chown ubuntu test-primary9 2024-09-12T14:02:02.080 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ SNAP_PIDS+=($!) 2024-09-12T14:02:02.080 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ take_mirror_snapshots cluster1 mirror image-primary9 2024-09-12T14:02:02.080 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:02:02.080 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:02:02.080 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T14:02:02.080 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:02:02.080 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T14:02:02.080 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ WORKLOAD_PIDS+=($!) 2024-09-12T14:02:02.080 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:02:02.080 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:02:02.080 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T14:02:02.080 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T14:02:02.081 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T14:02:02.081 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ IMG=image-primary10 2024-09-12T14:02:02.081 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ MNTPT=test-primary10 2024-09-12T14:02:02.081 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ create_image_and_enable_mirror cluster1 mirror image-primary10 snapshot 10G 2024-09-12T14:02:02.081 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:02:02.081 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T14:02:02.081 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:02:02.081 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T14:02:02.081 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T14:02:02.081 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T14:02:02.081 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mode=snapshot 2024-09-12T14:02:02.082 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n snapshot ']' 2024-09-12T14:02:02.082 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T14:02:02.082 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ create_image cluster1 mirror image-primary10 10G 2024-09-12T14:02:02.082 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ slow_untar_workload test-primary9 2024-09-12T14:02:02.082 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:02:02.082 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T14:02:02.082 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:02:02.082 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T14:02:02.082 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T14:02:02.082 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T14:02:02.082 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local size=128 2024-09-12T14:02:02.082 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n 10G ']' 2024-09-12T14:02:02.083 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ size=10G 2024-09-12T14:02:02.083 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T14:02:02.083 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 create --size 10G --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image-primary10 2024-09-12T14:02:02.083 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mountpt=test-primary9 2024-09-12T14:02:02.083 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ cp linux-5.4.tar.gz test-primary9 2024-09-12T14:02:02.153 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ is_pool_mirror_mode_image mirror 2024-09-12T14:02:02.153 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:02:02.153 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' image = image ']' 2024-09-12T14:02:02.154 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:02:02.154 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ enable_mirror cluster1 mirror image-primary10 snapshot 2024-09-12T14:02:02.154 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:02:02.154 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:02:02.154 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T14:02:02.154 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mode=snapshot 2024-09-12T14:02:02.154 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 mirror image enable mirror/image-primary10 snapshot 2024-09-12T14:02:02.368 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local ret=0 2024-09-12T14:02:02.368 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ timeout 5m bash -c 'zcat test-primary9/linux-5.4.tar.gz | pv -L 256K | tar xf - -C test-primary9' 2024-09-12T14:02:02.475 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2667 2024-09-12T14:02:02.476 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Mirroring enabled 2024-09-12T14:02:02.479 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2666 2024-09-12T14:02:02.495 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2668 2024-09-12T14:02:02.497 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:02:02.510 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 info mirror/image-primary10 2024-09-12T14:02:02.510 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:02:02.527 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:02:02.563 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:02:02.563 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T14:02:02.564 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:02:02.564 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:02:02.564 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T14:02:02.564 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T14:02:02.596 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:02:02.596 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T14:02:02.598 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:02:02.598 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:02:02.598 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T14:02:02.598 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T14:02:02.676 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:rbd image 'image-primary10': 2024-09-12T14:02:02.676 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: size 10 GiB in 2560 objects 2024-09-12T14:02:02.676 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: order 22 (4 MiB objects) 2024-09-12T14:02:02.676 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: snapshot_count: 1 2024-09-12T14:02:02.676 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: id: 38b540b32504 2024-09-12T14:02:02.676 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: block_name_prefix: rbd_data.38b540b32504 2024-09-12T14:02:02.676 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: format: 2 2024-09-12T14:02:02.677 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: features: layering, exclusive-lock, object-map, fast-diff 2024-09-12T14:02:02.677 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: op_features: 2024-09-12T14:02:02.677 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: flags: 2024-09-12T14:02:02.677 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: create_timestamp: Thu Sep 12 14:02:02 2024 2024-09-12T14:02:02.677 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: access_timestamp: Thu Sep 12 14:02:02 2024 2024-09-12T14:02:02.677 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: modify_timestamp: Thu Sep 12 14:02:02 2024 2024-09-12T14:02:02.677 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring state: enabled 2024-09-12T14:02:02.677 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring mode: snapshot 2024-09-12T14:02:02.677 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring global id: b0e88823-56d3-40a7-88c0-8d4c22f008ee 2024-09-12T14:02:02.677 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring primary: true 2024-09-12T14:02:02.694 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \n\b\d ]] 2024-09-12T14:02:02.694 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \k\r\b\d ]] 2024-09-12T14:02:02.695 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo rbd --cluster cluster1 device map -t krbd mirror/image-primary10 2024-09-12T14:02:02.870 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEV=/dev/rbd9 2024-09-12T14:02:02.870 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEVS+=($DEV) 2024-09-12T14:02:02.870 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo mkfs.ext4 /dev/rbd9 2024-09-12T14:02:02.892 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:mke2fs 1.46.5 (30-Dec-2021) 2024-09-12T14:02:02.908 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Discarding device blocks: 0/2621440 done 2024-09-12T14:02:02.908 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Creating filesystem with 2621440 4k blocks and 655360 inodes 2024-09-12T14:02:02.908 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Filesystem UUID: 43031585-19bc-4d80-8199-2674f5a954d9 2024-09-12T14:02:02.908 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Superblock backups stored on blocks: 2024-09-12T14:02:02.908 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 2024-09-12T14:02:02.908 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 2024-09-12T14:02:02.908 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Allocating group tables: 0/80 done 2024-09-12T14:02:02.908 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Writing inode tables: 0/80 done 2024-09-12T14:02:02.913 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Creating journal (16384 blocks): done 2024-09-12T14:02:03.033 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Writing superblocks and filesystem accounting information: 0/80 done 2024-09-12T14:02:03.033 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 2024-09-12T14:02:03.036 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mkdir test-primary10 2024-09-12T14:02:03.038 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo mount /dev/rbd9 test-primary10 2024-09-12T14:02:03.095 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ whoami 2024-09-12T14:02:03.096 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo chown ubuntu test-primary10 2024-09-12T14:02:03.121 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ SNAP_PIDS+=($!) 2024-09-12T14:02:03.121 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ take_mirror_snapshots cluster1 mirror image-primary10 2024-09-12T14:02:03.122 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:02:03.122 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:02:03.122 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T14:02:03.122 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:02:03.122 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T14:02:03.122 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:02:03.122 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:02:03.122 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T14:02:03.122 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T14:02:03.122 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ WORKLOAD_PIDS+=($!) 2024-09-12T14:02:03.122 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${SNAP_PIDS[@]} 2024-09-12T14:02:03.122 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 130355 2024-09-12T14:02:03.122 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ slow_untar_workload test-primary10 2024-09-12T14:02:03.123 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mountpt=test-primary10 2024-09-12T14:02:03.123 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ cp linux-5.4.tar.gz test-primary10 2024-09-12T14:02:03.443 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local ret=0 2024-09-12T14:02:03.445 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ timeout 5m bash -c 'zcat test-primary10/linux-5.4.tar.gz | pv -L 256K | tar xf - -C test-primary10' 2024-09-12T14:02:03.447 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2672 2024-09-12T14:02:03.460 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2670 2024-09-12T14:02:03.467 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2671 2024-09-12T14:02:03.477 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:02:03.484 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:02:03.496 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:02:03.496 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T14:02:03.496 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:02:03.496 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:02:03.496 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T14:02:03.496 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T14:02:03.519 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:02:04.498 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:02:04.498 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T14:02:04.498 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:02:04.498 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:02:04.498 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T14:02:04.499 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T14:02:04.520 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:02:04.520 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T14:02:04.520 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:02:04.520 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:02:04.520 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T14:02:04.520 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T14:02:04.522 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:02:04.522 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T14:02:04.522 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:02:04.522 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:02:04.522 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T14:02:04.522 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T14:02:04.701 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2673 2024-09-12T14:02:04.750 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:02:05.507 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:02:05.507 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T14:02:05.507 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:02:05.507 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:02:05.507 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T14:02:05.507 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T14:02:05.515 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:02:05.516 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T14:02:05.516 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:02:05.516 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:02:05.516 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T14:02:05.516 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T14:02:05.528 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2675 2024-09-12T14:02:05.532 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:02:05.532 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T14:02:05.533 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:02:05.533 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:02:05.534 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2674 2024-09-12T14:02:05.535 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T14:02:05.535 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T14:02:05.547 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2676 2024-09-12T14:02:05.592 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:02:05.604 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:02:05.614 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:02:06.485 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:02:06.485 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T14:02:06.485 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:02:06.485 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:02:06.485 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T14:02:06.485 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T14:02:06.496 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:02:06.496 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T14:02:06.497 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:02:06.497 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:02:06.497 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T14:02:06.497 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T14:02:06.528 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:02:06.528 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T14:02:06.528 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:02:06.528 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:02:06.528 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T14:02:06.528 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T14:02:06.671 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2677 2024-09-12T14:02:06.697 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2678 2024-09-12T14:02:06.702 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2679 2024-09-12T14:02:06.727 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:02:06.788 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:02:06.802 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:02:07.529 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2680 2024-09-12T14:02:07.529 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2681 2024-09-12T14:02:07.632 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2682 2024-09-12T14:02:07.681 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:02:07.692 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:02:07.709 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:02:07.756 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:02:07.757 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T14:02:07.757 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:02:07.757 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:02:07.757 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T14:02:07.757 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T14:02:08.509 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2683 2024-09-12T14:02:08.539 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:02:08.607 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:02:08.607 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T14:02:08.607 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:02:08.607 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:02:08.607 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T14:02:08.607 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T14:02:08.622 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:02:08.622 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T14:02:08.622 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:02:08.622 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:02:08.622 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T14:02:08.622 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T14:02:08.627 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:02:08.628 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T14:02:08.628 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:02:08.628 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:02:08.628 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T14:02:08.629 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T14:02:09.478 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2684 2024-09-12T14:02:09.500 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2685 2024-09-12T14:02:09.501 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2686 2024-09-12T14:02:09.518 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:02:09.540 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:02:09.550 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:02:09.732 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:02:09.733 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T14:02:09.733 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:02:09.733 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:02:09.733 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T14:02:09.733 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T14:02:09.792 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:02:09.792 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T14:02:09.792 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:02:09.792 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:02:09.792 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T14:02:09.792 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T14:02:09.809 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:02:09.809 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T14:02:09.809 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:02:09.809 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:02:09.809 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T14:02:09.809 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T14:02:10.526 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2689 2024-09-12T14:02:10.529 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2688 2024-09-12T14:02:10.553 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2687 2024-09-12T14:02:10.565 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:02:10.571 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:02:10.588 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:02:10.688 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:02:10.688 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T14:02:10.688 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:02:10.688 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:02:10.688 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T14:02:10.689 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T14:02:10.696 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:02:10.697 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T14:02:10.698 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:02:10.698 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:02:10.698 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T14:02:10.698 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T14:02:10.718 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:02:10.718 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T14:02:10.718 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:02:10.718 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:02:10.718 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T14:02:10.718 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T14:02:11.542 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:02:11.550 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T14:02:11.550 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:02:11.550 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:02:11.550 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T14:02:11.550 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T14:02:11.583 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2690 2024-09-12T14:02:11.620 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2692 2024-09-12T14:02:11.680 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:02:11.681 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2691 2024-09-12T14:02:11.709 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:02:11.730 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:02:12.526 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:02:12.529 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T14:02:12.529 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:02:12.529 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:02:12.529 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T14:02:12.529 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T14:02:12.544 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:02:12.544 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T14:02:12.544 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:02:12.544 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:02:12.544 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T14:02:12.544 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T14:02:12.555 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2693 2024-09-12T14:02:12.555 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:02:12.556 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T14:02:12.556 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:02:12.556 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:02:12.556 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T14:02:12.556 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T14:02:12.601 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:02:13.456 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2694 2024-09-12T14:02:13.485 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:02:13.486 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2695 2024-09-12T14:02:13.500 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:02:13.572 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:02:13.572 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T14:02:13.572 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:02:13.572 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:02:13.572 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T14:02:13.572 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T14:02:13.577 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:02:13.577 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T14:02:13.577 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:02:13.577 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:02:13.577 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T14:02:13.577 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T14:02:13.591 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:02:13.591 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T14:02:13.591 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:02:13.592 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:02:13.592 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T14:02:13.592 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T14:02:14.482 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2698 2024-09-12T14:02:14.487 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2696 2024-09-12T14:02:14.523 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:02:14.541 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2697 2024-09-12T14:02:14.553 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:02:14.596 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:02:14.690 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:02:14.690 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T14:02:14.691 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:02:14.691 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:02:14.691 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T14:02:14.691 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T14:02:14.712 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:02:14.712 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T14:02:14.712 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:02:14.712 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:02:14.712 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T14:02:14.713 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T14:02:14.734 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:02:14.734 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T14:02:14.734 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:02:14.734 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:02:14.734 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T14:02:14.734 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T14:02:15.518 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T14:02:15.519+0000 7f67dc600640 -1 rbd::mirror::image_replayer::snapshot::Replayer: 0x56055b999c00 handle_unlink_peer: failed to unlink local peer from remote image: (22) Invalid argument 2024-09-12T14:02:15.568 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2701 2024-09-12T14:02:15.591 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:02:15.605 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:02:15.605 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T14:02:15.605 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:02:15.605 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:02:15.605 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T14:02:15.605 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T14:02:15.609 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2700 2024-09-12T14:02:15.626 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:02:16.478 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2699 2024-09-12T14:02:16.489 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:02:16.490 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T14:02:16.490 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:02:16.490 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:02:16.490 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T14:02:16.490 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T14:02:16.505 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:02:16.505 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T14:02:16.505 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:02:16.505 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:02:16.505 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T14:02:16.505 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T14:02:16.526 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:02:17.506 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2702 2024-09-12T14:02:17.530 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:02:17.531 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T14:02:17.531 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:02:17.531 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:02:17.531 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T14:02:17.531 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T14:02:17.560 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:02:17.561 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T14:02:17.561 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:02:17.561 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:02:17.561 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T14:02:17.561 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T14:02:17.573 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:02:17.604 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:02:17.604 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T14:02:17.604 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:02:17.604 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:02:17.604 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T14:02:17.604 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T14:02:17.673 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2704 2024-09-12T14:02:17.771 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:02:18.458 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2703 2024-09-12T14:02:18.526 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:02:18.594 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:02:18.594 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T14:02:18.595 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:02:18.595 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:02:18.595 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T14:02:18.595 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T14:02:18.630 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:02:18.630 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T14:02:18.630 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:02:18.630 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:02:18.630 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T14:02:18.630 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T14:02:18.713 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2708 2024-09-12T14:02:18.791 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:02:19.522 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2705 2024-09-12T14:02:19.531 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:02:19.532 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T14:02:19.532 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:02:19.532 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:02:19.532 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T14:02:19.532 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T14:02:19.581 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2706 2024-09-12T14:02:19.586 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:02:19.627 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:02:19.645 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2710 2024-09-12T14:02:19.692 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:02:20.505 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2707 2024-09-12T14:02:20.535 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:02:20.578 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:02:20.578 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T14:02:20.578 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:02:20.578 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:02:20.578 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T14:02:20.578 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T14:02:20.609 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2711 2024-09-12T14:02:20.623 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:02:20.773 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:02:20.774 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T14:02:20.774 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:02:20.774 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:02:20.774 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T14:02:20.774 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T14:02:21.504 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2709 2024-09-12T14:02:21.529 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:02:21.530 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T14:02:21.530 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:02:21.530 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:02:21.530 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T14:02:21.530 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T14:02:21.554 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:02:21.623 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2712 2024-09-12T14:02:21.643 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:02:21.794 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:02:21.794 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T14:02:21.794 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:02:21.794 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:02:21.794 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T14:02:21.794 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T14:02:22.591 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2713 2024-09-12T14:02:22.592 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:02:22.592 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T14:02:22.592 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:02:22.592 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:02:22.592 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T14:02:22.592 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T14:02:22.605 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:02:22.637 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:02:22.638 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T14:02:22.638 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:02:22.638 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:02:22.638 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T14:02:22.638 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T14:02:22.645 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2714 2024-09-12T14:02:22.670 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:02:22.697 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:02:22.697 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T14:02:22.697 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:02:22.697 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:02:22.697 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T14:02:22.697 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T14:02:23.538 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:02:23.538 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T14:02:23.538 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:02:23.539 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:02:23.539 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T14:02:23.539 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T14:02:23.623 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2715 2024-09-12T14:02:23.626 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:02:23.627 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T14:02:23.627 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:02:23.628 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:02:23.628 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T14:02:23.628 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T14:02:23.653 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:02:23.690 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2716 2024-09-12T14:02:23.725 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:02:24.559 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:02:24.560 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T14:02:24.560 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:02:24.560 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:02:24.560 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T14:02:24.560 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T14:02:24.607 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2717 2024-09-12T14:02:24.651 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:02:24.651 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T14:02:24.651 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:02:24.651 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:02:24.652 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T14:02:24.652 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T14:02:24.670 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:02:24.680 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2719 2024-09-12T14:02:24.714 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:02:25.546 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2718 2024-09-12T14:02:25.583 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2720 2024-09-12T14:02:25.590 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:02:25.610 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:02:25.610 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T14:02:25.610 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:02:25.610 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:02:25.610 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T14:02:25.610 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T14:02:25.646 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:02:25.674 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:02:25.674 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T14:02:25.674 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:02:25.675 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:02:25.675 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T14:02:25.675 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T14:02:25.688 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2721 2024-09-12T14:02:25.703 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:02:26.596 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2722 2024-09-12T14:02:26.642 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:02:26.659 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:02:26.659 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T14:02:26.659 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:02:26.659 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:02:26.659 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T14:02:26.659 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T14:02:26.729 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:02:26.730 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T14:02:26.730 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:02:26.730 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:02:26.730 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T14:02:26.730 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T14:02:27.677 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:02:27.678 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T14:02:27.678 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:02:27.678 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:02:27.678 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T14:02:27.678 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T14:02:27.721 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:02:27.722 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T14:02:27.722 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:02:27.722 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:02:27.722 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T14:02:27.722 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T14:02:28.455 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2723 2024-09-12T14:02:28.477 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2724 2024-09-12T14:02:28.530 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:02:28.530 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:02:28.538 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2726 2024-09-12T14:02:28.587 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:02:28.593 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:02:28.594 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T14:02:28.594 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:02:28.594 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:02:28.594 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T14:02:28.594 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T14:02:28.649 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:02:28.649 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T14:02:28.649 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:02:28.649 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:02:28.649 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T14:02:28.649 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T14:02:28.706 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:02:28.706 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T14:02:28.707 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:02:28.707 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:02:28.707 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T14:02:28.707 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T14:02:29.512 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2727 2024-09-12T14:02:29.528 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2728 2024-09-12T14:02:29.533 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2729 2024-09-12T14:02:29.548 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:02:29.567 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:02:29.576 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:02:29.646 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:02:29.647 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T14:02:29.647 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:02:29.647 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:02:29.647 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T14:02:29.647 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T14:02:30.442 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2725 2024-09-12T14:02:30.496 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:02:30.504 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2731 2024-09-12T14:02:30.522 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:02:31.441 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2730 2024-09-12T14:02:31.489 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:02:31.498 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2732 2024-09-12T14:02:31.514 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:02:31.533 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:02:31.534 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T14:02:31.534 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:02:31.534 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:02:31.534 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T14:02:31.534 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T14:02:31.535 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:02:31.535 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T14:02:31.535 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:02:31.535 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:02:31.535 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T14:02:31.535 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T14:02:31.592 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:02:31.593 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T14:02:31.594 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:02:31.594 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:02:31.597 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T14:02:31.597 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T14:02:32.515 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2733 2024-09-12T14:02:32.522 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2734 2024-09-12T14:02:32.538 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:02:32.541 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:02:32.553 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:02:32.553 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T14:02:32.553 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:02:32.553 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:02:32.553 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T14:02:32.553 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T14:02:32.572 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:02:32.572 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T14:02:32.572 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:02:32.573 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:02:32.573 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T14:02:32.573 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T14:02:32.580 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:02:32.580 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T14:02:32.581 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:02:32.581 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:02:32.581 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T14:02:32.581 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T14:02:33.499 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:02:33.499 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T14:02:33.500 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:02:33.500 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:02:33.500 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T14:02:33.500 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T14:02:33.527 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:02:33.527 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T14:02:33.527 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:02:33.527 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:02:33.527 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T14:02:33.528 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T14:02:34.476 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2735 2024-09-12T14:02:34.492 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:02:34.492 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T14:02:34.492 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:02:34.492 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:02:34.492 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T14:02:34.492 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T14:02:34.507 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2737 2024-09-12T14:02:34.517 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:02:34.518 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T14:02:34.519 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:02:34.519 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:02:34.519 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T14:02:34.519 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T14:02:34.539 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:02:34.544 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2738 2024-09-12T14:02:34.573 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:02:34.601 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:02:34.609 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2739 2024-09-12T14:02:34.639 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:02:35.498 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2736 2024-09-12T14:02:35.545 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:02:35.546 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T14:02:35.546 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:02:35.546 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:02:35.546 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T14:02:35.546 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T14:02:35.553 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:02:35.553 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T14:02:35.554 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:02:35.554 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:02:35.554 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T14:02:35.554 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T14:02:35.569 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:02:35.653 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2741 2024-09-12T14:02:35.698 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:02:36.470 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2740 2024-09-12T14:02:36.563 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:02:36.574 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2742 2024-09-12T14:02:36.596 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:02:37.537 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2744 2024-09-12T14:02:37.545 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:02:37.545 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T14:02:37.545 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:02:37.545 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:02:37.545 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T14:02:37.545 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T14:02:37.577 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:02:37.577 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T14:02:37.577 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:02:37.577 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:02:37.577 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T14:02:37.577 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T14:02:37.584 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2743 2024-09-12T14:02:37.601 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:02:37.605 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:02:37.605 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T14:02:37.605 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:02:37.605 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:02:37.605 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T14:02:37.605 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T14:02:37.613 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:02:37.641 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:02:37.641 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T14:02:37.641 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:02:37.641 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:02:37.641 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T14:02:37.641 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T14:02:38.492 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2748 2024-09-12T14:02:38.511 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:02:38.574 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:02:38.575 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T14:02:38.575 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:02:38.576 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:02:38.576 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T14:02:38.576 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T14:02:38.701 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:02:38.701 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T14:02:38.701 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:02:38.701 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:02:38.701 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T14:02:38.702 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T14:02:39.564 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2749 2024-09-12T14:02:39.566 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:02:39.566 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T14:02:39.567 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:02:39.567 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:02:39.567 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T14:02:39.567 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T14:02:39.599 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:02:39.599 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T14:02:39.599 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:02:39.599 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:02:39.599 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T14:02:39.599 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T14:02:39.600 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:02:40.474 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2746 2024-09-12T14:02:40.481 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2745 2024-09-12T14:02:40.498 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2747 2024-09-12T14:02:40.527 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:02:40.530 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:02:40.560 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:02:40.595 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2751 2024-09-12T14:02:40.605 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:02:40.605 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T14:02:40.606 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:02:40.606 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:02:40.606 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T14:02:40.606 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T14:02:40.616 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:02:40.616 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T14:02:40.617 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:02:40.617 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:02:40.617 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T14:02:40.617 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T14:02:40.650 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:02:41.493 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2750 2024-09-12T14:02:41.513 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:02:41.513 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T14:02:41.513 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:02:41.514 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:02:41.514 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T14:02:41.514 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T14:02:41.559 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:02:42.490 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2752 2024-09-12T14:02:42.554 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:02:42.604 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:02:42.604 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T14:02:42.604 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:02:42.604 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:02:42.604 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T14:02:42.604 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T14:02:43.435 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2753 2024-09-12T14:02:43.452 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2754 2024-09-12T14:02:43.498 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:02:43.519 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:02:43.530 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:02:43.531 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T14:02:43.531 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:02:43.531 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:02:43.531 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T14:02:43.531 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T14:02:43.533 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:02:43.533 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T14:02:43.533 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:02:43.533 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:02:43.533 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T14:02:43.534 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T14:02:43.561 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:02:43.562 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T14:02:43.562 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:02:43.562 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:02:43.562 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T14:02:43.562 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T14:02:43.653 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:02:43.653 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T14:02:43.653 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:02:43.653 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:02:43.653 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T14:02:43.653 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T14:02:44.502 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2755 2024-09-12T14:02:44.549 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:02:44.562 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:02:44.562 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T14:02:44.562 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:02:44.563 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:02:44.563 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T14:02:44.563 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T14:02:44.630 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2757 2024-09-12T14:02:44.650 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2758 2024-09-12T14:02:44.660 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2759 2024-09-12T14:02:44.707 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:02:44.709 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:02:44.732 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:02:45.525 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2756 2024-09-12T14:02:45.556 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:02:45.556 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T14:02:45.557 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:02:45.557 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:02:45.557 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T14:02:45.557 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T14:02:45.582 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:02:45.647 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2761 2024-09-12T14:02:45.681 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:02:46.501 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:02:46.501 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T14:02:46.501 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:02:46.501 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:02:46.501 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T14:02:46.501 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T14:02:46.521 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:02:46.522 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T14:02:46.522 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:02:46.522 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:02:46.522 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T14:02:46.522 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T14:02:46.525 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2760 2024-09-12T14:02:46.573 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:02:46.658 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2762 2024-09-12T14:02:46.676 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:02:47.551 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:02:47.551 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T14:02:47.552 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:02:47.552 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:02:47.552 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T14:02:47.552 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T14:02:47.556 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2763 2024-09-12T14:02:47.560 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2764 2024-09-12T14:02:47.565 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:02:47.570 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:02:47.710 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:02:47.710 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T14:02:47.710 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:02:47.710 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:02:47.710 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T14:02:47.710 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T14:02:47.712 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:02:47.712 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T14:02:47.712 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:02:47.712 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:02:47.712 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T14:02:47.712 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T14:02:47.734 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:02:47.734 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T14:02:47.734 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:02:47.735 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:02:47.735 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T14:02:47.735 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T14:02:48.512 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2765 2024-09-12T14:02:48.523 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:02:48.585 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:02:48.585 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T14:02:48.586 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:02:48.586 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:02:48.586 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T14:02:48.586 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T14:02:48.684 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:02:48.684 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T14:02:48.684 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:02:48.684 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:02:48.684 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T14:02:48.684 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T14:02:49.504 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2766 2024-09-12T14:02:49.516 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2767 2024-09-12T14:02:49.559 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:02:49.560 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:02:49.568 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2769 2024-09-12T14:02:49.576 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:02:49.576 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T14:02:49.576 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:02:49.576 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:02:49.576 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T14:02:49.576 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T14:02:49.582 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:02:49.678 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:02:49.678 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T14:02:49.679 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:02:49.679 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:02:49.679 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T14:02:49.679 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T14:02:50.521 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2768 2024-09-12T14:02:50.559 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:02:50.568 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:02:50.568 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T14:02:50.568 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:02:50.568 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:02:50.568 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T14:02:50.568 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T14:02:50.573 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:02:50.573 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T14:02:50.573 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:02:50.574 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:02:50.574 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T14:02:50.574 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T14:02:50.624 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2771 2024-09-12T14:02:50.648 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:02:51.521 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2770 2024-09-12T14:02:51.527 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:02:51.527 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T14:02:51.527 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:02:51.527 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:02:51.527 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T14:02:51.527 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T14:02:51.607 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:02:52.530 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2772 2024-09-12T14:02:52.562 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:02:52.562 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T14:02:52.563 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:02:52.563 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:02:52.563 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T14:02:52.563 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T14:02:52.563 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:02:52.564 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T14:02:52.564 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:02:52.564 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:02:52.564 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T14:02:52.564 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T14:02:52.585 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:02:52.586 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T14:02:52.586 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:02:52.586 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:02:52.586 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T14:02:52.586 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T14:02:52.596 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:02:53.457 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2774 2024-09-12T14:02:53.490 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2773 2024-09-12T14:02:53.523 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:02:53.562 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:02:53.562 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T14:02:53.562 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:02:53.562 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:02:53.562 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T14:02:53.562 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T14:02:53.590 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:02:53.651 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:02:53.651 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T14:02:53.651 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:02:53.651 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:02:53.651 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T14:02:53.651 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T14:02:54.566 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2775 2024-09-12T14:02:54.610 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:02:54.610 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T14:02:54.610 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:02:54.610 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:02:54.610 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T14:02:54.610 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T14:02:54.636 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:02:54.747 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2779 2024-09-12T14:02:54.761 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:02:55.546 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2776 2024-09-12T14:02:55.550 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2777 2024-09-12T14:02:55.571 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2778 2024-09-12T14:02:55.598 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:02:55.599 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T14:02:55.599 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:02:55.599 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:02:55.599 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T14:02:55.599 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T14:02:55.624 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:02:55.657 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:02:55.676 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:02:55.864 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2781 2024-09-12T14:02:55.901 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:02:56.526 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:02:56.526 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T14:02:56.527 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:02:56.527 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:02:56.527 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T14:02:56.527 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T14:02:56.539 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2780 2024-09-12T14:02:56.564 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:02:56.592 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:02:56.592 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T14:02:56.593 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:02:56.593 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:02:56.593 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T14:02:56.593 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T14:02:56.594 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2782 2024-09-12T14:02:56.603 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:02:57.568 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2784 2024-09-12T14:02:57.573 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2783 2024-09-12T14:02:57.584 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:02:57.586 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:02:57.639 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:02:57.639 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T14:02:57.639 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:02:57.639 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:02:57.639 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T14:02:57.639 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T14:02:57.764 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:02:57.764 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T14:02:57.764 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:02:57.764 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:02:57.764 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T14:02:57.764 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T14:02:58.480 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2785 2024-09-12T14:02:58.490 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:02:58.627 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:02:58.627 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T14:02:58.628 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:02:58.628 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:02:58.628 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T14:02:58.628 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T14:02:58.659 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:02:58.660 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T14:02:58.660 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:02:58.660 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:02:58.660 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T14:02:58.660 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T14:02:58.678 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:02:58.679 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T14:02:58.679 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:02:58.679 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:02:58.679 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T14:02:58.679 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T14:02:58.903 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:02:58.903 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T14:02:58.904 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:02:58.904 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:02:58.904 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T14:02:58.904 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T14:02:59.463 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2786 2024-09-12T14:02:59.490 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2789 2024-09-12T14:02:59.491 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:02:59.492 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2788 2024-09-12T14:02:59.511 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2787 2024-09-12T14:02:59.520 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:02:59.522 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:02:59.533 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:02:59.566 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:02:59.567 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T14:02:59.567 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:02:59.567 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:02:59.567 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T14:02:59.567 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T14:02:59.606 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:02:59.606 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T14:02:59.606 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:02:59.606 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:02:59.606 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T14:02:59.606 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T14:03:00.525 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2791 2024-09-12T14:03:00.553 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:03:00.585 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:03:00.585 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T14:03:00.585 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:03:00.585 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:03:00.586 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T14:03:00.586 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T14:03:00.589 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:03:00.589 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T14:03:00.589 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:03:00.589 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:03:00.589 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T14:03:00.589 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T14:03:01.445 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2790 2024-09-12T14:03:01.485 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2792 2024-09-12T14:03:01.487 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:03:01.493 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:03:01.494 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T14:03:01.494 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:03:01.494 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:03:01.494 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T14:03:01.494 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T14:03:01.522 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:03:02.474 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2794 2024-09-12T14:03:02.486 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2793 2024-09-12T14:03:02.496 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:03:02.496 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T14:03:02.496 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:03:02.496 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:03:02.496 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T14:03:02.496 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T14:03:02.503 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:03:02.505 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:03:02.527 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:03:02.528 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:03:02.528 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T14:03:02.528 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:03:02.528 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:03:02.528 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T14:03:02.528 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T14:03:02.530 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T14:03:02.530 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:03:02.530 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:03:02.530 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T14:03:02.530 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T14:03:02.537 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:03:02.537 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T14:03:02.537 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:03:02.537 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:03:02.537 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T14:03:02.537 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T14:03:03.481 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2795 2024-09-12T14:03:03.500 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:03:03.556 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:03:03.556 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T14:03:03.556 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:03:03.556 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:03:03.556 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T14:03:03.556 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T14:03:04.492 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:03:04.493 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T14:03:04.493 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:03:04.493 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:03:04.493 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T14:03:04.493 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T14:03:04.519 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2797 2024-09-12T14:03:04.527 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:03:04.527 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T14:03:04.527 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:03:04.527 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:03:04.527 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T14:03:04.527 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T14:03:04.569 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2799 2024-09-12T14:03:04.573 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2798 2024-09-12T14:03:04.591 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:03:04.593 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2796 2024-09-12T14:03:04.596 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:03:04.615 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:03:04.620 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:03:05.509 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:03:05.509 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T14:03:05.509 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:03:05.509 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:03:05.509 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T14:03:05.509 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T14:03:05.511 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:03:05.511 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T14:03:05.511 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:03:05.511 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:03:05.511 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T14:03:05.511 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T14:03:05.513 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2802 2024-09-12T14:03:05.529 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:03:06.474 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2800 2024-09-12T14:03:06.504 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:03:06.504 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T14:03:06.505 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:03:06.505 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:03:06.505 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T14:03:06.505 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T14:03:06.528 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:03:06.535 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2801 2024-09-12T14:03:06.562 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:03:07.514 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2804 2024-09-12T14:03:07.522 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2803 2024-09-12T14:03:07.533 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:03:07.534 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:03:07.595 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:03:07.595 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T14:03:07.596 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:03:07.596 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:03:07.596 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T14:03:07.596 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T14:03:07.600 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:03:07.600 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T14:03:07.600 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:03:07.600 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:03:07.601 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T14:03:07.601 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T14:03:07.620 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:03:07.620 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T14:03:07.620 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:03:07.620 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:03:07.621 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T14:03:07.621 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T14:03:07.624 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:03:07.624 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T14:03:07.624 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:03:07.624 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:03:07.624 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T14:03:07.624 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T14:03:08.493 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2805 2024-09-12T14:03:08.517 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:03:08.533 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:03:08.533 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T14:03:08.533 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:03:08.533 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:03:08.533 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T14:03:08.533 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T14:03:09.532 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:03:09.532 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T14:03:09.532 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:03:09.532 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:03:09.532 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T14:03:09.532 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T14:03:09.554 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2806 2024-09-12T14:03:09.566 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:03:09.566 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T14:03:09.566 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:03:09.566 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:03:09.566 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T14:03:09.566 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T14:03:09.570 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2809 2024-09-12T14:03:09.601 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:03:09.613 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:03:10.479 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2808 2024-09-12T14:03:10.491 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2807 2024-09-12T14:03:10.524 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:03:10.537 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:03:10.537 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T14:03:10.537 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:03:10.537 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:03:10.537 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T14:03:10.537 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T14:03:10.538 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:03:10.538 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T14:03:10.538 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:03:10.538 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:03:10.538 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T14:03:10.538 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T14:03:10.542 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:03:10.575 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2811 2024-09-12T14:03:10.592 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:03:11.510 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2810 2024-09-12T14:03:11.520 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:03:11.520 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T14:03:11.520 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:03:11.520 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:03:11.520 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T14:03:11.521 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T14:03:11.571 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2812 2024-09-12T14:03:11.571 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:03:11.587 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:03:12.536 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2813 2024-09-12T14:03:12.537 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2814 2024-09-12T14:03:12.552 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:03:12.568 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:03:12.604 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:03:12.604 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T14:03:12.604 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:03:12.604 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:03:12.604 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T14:03:12.604 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T14:03:12.616 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:03:12.616 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T14:03:12.617 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:03:12.617 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:03:12.617 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T14:03:12.617 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T14:03:13.493 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2815 2024-09-12T14:03:13.514 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:03:13.527 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:03:13.527 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T14:03:13.527 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:03:13.527 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:03:13.527 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T14:03:13.527 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T14:03:13.546 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:03:13.546 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T14:03:13.546 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:03:13.546 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:03:13.546 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T14:03:13.546 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T14:03:13.595 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:03:13.595 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T14:03:13.595 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:03:13.595 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:03:13.595 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T14:03:13.595 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T14:03:14.545 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2816 2024-09-12T14:03:14.551 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2817 2024-09-12T14:03:14.571 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:03:14.574 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2819 2024-09-12T14:03:14.575 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:03:14.575 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:03:14.575 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T14:03:14.575 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:03:14.575 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:03:14.575 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T14:03:14.575 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T14:03:14.578 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2818 2024-09-12T14:03:14.591 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:03:14.591 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T14:03:14.591 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:03:14.591 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:03:14.591 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T14:03:14.591 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T14:03:14.593 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:03:14.604 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:03:15.550 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2821 2024-09-12T14:03:15.556 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:03:15.556 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T14:03:15.556 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:03:15.556 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:03:15.556 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T14:03:15.556 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T14:03:15.559 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:03:15.571 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:03:15.571 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T14:03:15.572 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:03:15.572 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:03:15.572 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T14:03:15.572 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T14:03:16.514 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2820 2024-09-12T14:03:16.517 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:03:16.517 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T14:03:16.517 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:03:16.517 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:03:16.517 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T14:03:16.517 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T14:03:16.555 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:03:17.513 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2822 2024-09-12T14:03:17.551 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:03:17.555 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T14:03:17.556+0000 7f67dc600640 -1 rbd::mirror::image_replayer::snapshot::Replayer: 0x56055b7dd800 handle_unlink_peer: failed to unlink local peer from remote image: (22) Invalid argument 2024-09-12T14:03:17.573 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:03:17.573 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T14:03:17.573 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:03:17.574 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:03:17.574 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T14:03:17.574 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T14:03:17.577 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:03:17.577 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T14:03:17.577 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:03:17.577 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:03:17.577 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T14:03:17.578 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T14:03:17.596 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:03:17.596 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T14:03:17.596 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:03:17.596 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:03:17.596 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T14:03:17.596 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T14:03:17.608 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:03:17.608 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T14:03:17.608 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:03:17.608 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:03:17.608 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T14:03:17.608 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T14:03:18.472 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2823 2024-09-12T14:03:18.485 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2824 2024-09-12T14:03:18.522 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:03:18.523 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:03:18.561 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:03:18.562 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T14:03:18.562 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:03:18.562 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:03:18.562 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T14:03:18.562 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T14:03:19.527 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2825 2024-09-12T14:03:19.558 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:03:19.558 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T14:03:19.559 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:03:19.559 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:03:19.559 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T14:03:19.559 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T14:03:19.568 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T14:03:19.569+0000 7f67dd000640 -1 rbd::mirror::image_replayer::snapshot::Replayer: 0x560556cd3800 handle_unlink_peer: failed to unlink local peer from remote image: (22) Invalid argument 2024-09-12T14:03:19.581 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T14:03:19.582+0000 7f67dc600640 -1 rbd::mirror::image_replayer::snapshot::Replayer: 0x560556a8b400 handle_unlink_peer: failed to unlink local peer from remote image: (22) Invalid argument 2024-09-12T14:03:19.606 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:03:19.609 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T14:03:19.610+0000 7f67dd000640 -1 rbd::mirror::image_replayer::snapshot::Replayer: 0x56055ba71c00 handle_unlink_peer: failed to unlink local peer from remote image: (22) Invalid argument 2024-09-12T14:03:20.524 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2827 2024-09-12T14:03:20.527 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2828 2024-09-12T14:03:20.532 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2829 2024-09-12T14:03:20.545 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2826 2024-09-12T14:03:20.554 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:03:20.554 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T14:03:20.554 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:03:20.554 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:03:20.555 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T14:03:20.555 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T14:03:20.568 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:03:20.583 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:03:20.586 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:03:20.590 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:03:20.640 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2831 2024-09-12T14:03:20.660 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:03:21.516 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2830 2024-09-12T14:03:21.525 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:03:21.525 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:03:21.525 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T14:03:21.525 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T14:03:21.525 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:03:21.526 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:03:21.526 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:03:21.526 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T14:03:21.526 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:03:21.526 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T14:03:21.526 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T14:03:21.526 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T14:03:21.546 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:03:21.567 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2832 2024-09-12T14:03:21.577 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:03:22.548 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2833 2024-09-12T14:03:22.553 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2834 2024-09-12T14:03:22.562 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:03:22.564 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:03:22.609 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:03:22.609 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T14:03:22.609 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:03:22.609 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:03:22.609 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T14:03:22.610 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T14:03:23.485 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2835 2024-09-12T14:03:23.497 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:03:23.571 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:03:23.571 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T14:03:23.571 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:03:23.571 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:03:23.572 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T14:03:23.572 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T14:03:23.586 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:03:23.586 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T14:03:23.586 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:03:23.586 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:03:23.587 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T14:03:23.587 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T14:03:23.588 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:03:23.588 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T14:03:23.589 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:03:23.589 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:03:23.589 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T14:03:23.589 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T14:03:23.592 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:03:23.593 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T14:03:23.593 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:03:23.593 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:03:23.593 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T14:03:23.593 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T14:03:23.663 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:03:23.663 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T14:03:23.663 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:03:23.663 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:03:23.663 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T14:03:23.663 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T14:03:24.549 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:03:24.549 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T14:03:24.549 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:03:24.549 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:03:24.549 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T14:03:24.549 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T14:03:24.565 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2838 2024-09-12T14:03:24.566 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2836 2024-09-12T14:03:24.570 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2837 2024-09-12T14:03:24.576 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2839 2024-09-12T14:03:24.580 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:03:24.580 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T14:03:24.580 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:03:24.580 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:03:24.580 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T14:03:24.580 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T14:03:24.588 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:03:24.590 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:03:24.595 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:03:24.596 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:03:25.565 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:03:25.565 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T14:03:25.565 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:03:25.565 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:03:25.565 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T14:03:25.565 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T14:03:25.566 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:03:25.566 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T14:03:25.567 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:03:25.567 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:03:25.567 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T14:03:25.567 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T14:03:25.584 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2841 2024-09-12T14:03:25.596 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:03:26.500 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:03:26.501 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T14:03:26.501 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:03:26.501 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:03:26.501 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T14:03:26.501 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T14:03:26.551 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2840 2024-09-12T14:03:26.588 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:03:27.553 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2842 2024-09-12T14:03:27.591 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:03:27.591 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T14:03:27.592 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:03:27.592 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:03:27.592 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T14:03:27.592 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T14:03:27.593 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:03:27.593 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T14:03:27.593 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:03:27.593 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:03:27.593 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T14:03:27.593 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T14:03:27.594 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:03:27.597 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:03:27.597 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T14:03:27.597 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:03:27.597 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:03:27.598 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T14:03:27.598 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T14:03:27.598 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:03:27.598 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T14:03:27.598 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:03:27.599 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:03:27.599 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T14:03:27.599 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T14:03:28.459 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2844 2024-09-12T14:03:28.480 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2843 2024-09-12T14:03:28.513 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:03:28.521 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:03:28.549 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2848 2024-09-12T14:03:28.555 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2849 2024-09-12T14:03:28.568 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:03:28.570 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2846 2024-09-12T14:03:28.575 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:03:28.579 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:03:28.598 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:03:28.599 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T14:03:28.599 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:03:28.599 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:03:28.599 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T14:03:28.599 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T14:03:29.551 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2845 2024-09-12T14:03:29.580 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:03:29.590 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:03:29.590 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T14:03:29.590 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:03:29.590 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:03:29.590 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T14:03:29.590 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T14:03:30.556 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2847 2024-09-12T14:03:30.594 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:03:30.597 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:03:30.597 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T14:03:30.597 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:03:30.597 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:03:30.597 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T14:03:30.597 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T14:03:30.620 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2851 2024-09-12T14:03:30.636 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:03:31.516 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:03:31.517 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T14:03:31.517 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:03:31.517 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:03:31.517 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T14:03:31.517 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T14:03:31.524 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:03:31.524 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T14:03:31.524 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:03:31.524 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:03:31.525 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T14:03:31.525 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T14:03:31.552 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2850 2024-09-12T14:03:31.572 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:03:31.572 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T14:03:31.572 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:03:31.572 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:03:31.572 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T14:03:31.572 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T14:03:31.574 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:03:31.577 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:03:31.577 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T14:03:31.577 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:03:31.578 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:03:31.578 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T14:03:31.578 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T14:03:31.582 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:03:31.582 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T14:03:31.582 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:03:31.582 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:03:31.582 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T14:03:31.582 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T14:03:31.599 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2852 2024-09-12T14:03:31.615 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:03:32.583 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:03:32.583 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T14:03:32.583 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:03:32.583 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:03:32.583 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T14:03:32.584 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T14:03:32.592 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2856 2024-09-12T14:03:32.595 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2857 2024-09-12T14:03:32.596 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2854 2024-09-12T14:03:32.602 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2855 2024-09-12T14:03:32.610 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:03:32.617 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:03:32.618 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:03:32.619 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2853 2024-09-12T14:03:32.624 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:03:32.628 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:03:33.484 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2858 2024-09-12T14:03:33.497 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:03:33.598 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:03:33.598 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T14:03:33.598 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:03:33.598 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:03:33.598 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T14:03:33.598 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T14:03:33.639 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:03:33.639 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T14:03:33.639 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:03:33.639 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:03:33.639 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T14:03:33.639 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T14:03:34.577 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:03:34.578 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T14:03:34.578 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:03:34.578 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:03:34.578 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T14:03:34.578 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T14:03:34.585 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2859 2024-09-12T14:03:34.600 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:03:34.618 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:03:34.618 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T14:03:34.618 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:03:34.618 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:03:34.618 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T14:03:34.619 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T14:03:35.614 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:03:35.614 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T14:03:35.615 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:03:35.615 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:03:35.615 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T14:03:35.615 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T14:03:35.620 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2861 2024-09-12T14:03:35.621 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:03:35.621 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T14:03:35.621 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:03:35.621 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:03:35.621 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T14:03:35.622 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T14:03:35.622 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:03:35.622 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T14:03:35.622 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:03:35.622 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:03:35.622 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T14:03:35.622 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T14:03:35.627 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:03:35.627 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T14:03:35.627 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:03:35.627 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:03:35.627 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T14:03:35.627 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T14:03:35.631 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:03:35.631 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T14:03:35.631 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:03:35.631 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:03:35.631 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T14:03:35.632 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T14:03:35.632 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:03:36.501 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:03:36.501 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T14:03:36.501 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:03:36.502 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:03:36.502 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T14:03:36.502 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T14:03:36.601 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2860 2024-09-12T14:03:36.654 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:03:36.688 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2863 2024-09-12T14:03:36.691 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2864 2024-09-12T14:03:36.694 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2865 2024-09-12T14:03:36.701 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:03:36.717 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:03:36.725 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:03:37.604 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:03:37.604 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T14:03:37.604 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:03:37.604 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:03:37.604 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T14:03:37.604 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T14:03:37.610 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2862 2024-09-12T14:03:37.668 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:03:38.457 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2867 2024-09-12T14:03:38.461 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2866 2024-09-12T14:03:38.505 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:03:38.511 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:03:38.516 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2868 2024-09-12T14:03:38.537 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:03:38.636 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:03:38.636 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T14:03:38.636 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:03:38.636 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:03:38.637 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T14:03:38.637 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T14:03:39.466 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2869 2024-09-12T14:03:39.483 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:03:39.657 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:03:39.657 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T14:03:39.657 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:03:39.657 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:03:39.657 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T14:03:39.657 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T14:03:39.704 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:03:39.704 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T14:03:39.704 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:03:39.705 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:03:39.705 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T14:03:39.705 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T14:03:39.719 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:03:39.719 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T14:03:39.719 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:03:39.719 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:03:39.719 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T14:03:39.719 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T14:03:39.728 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:03:39.728 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T14:03:39.728 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:03:39.728 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:03:39.728 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T14:03:39.729 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T14:03:40.512 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2871 2024-09-12T14:03:40.531 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:03:40.670 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:03:40.671 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T14:03:40.671 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:03:40.671 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:03:40.671 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T14:03:40.671 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T14:03:41.450 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2870 2024-09-12T14:03:41.456 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2873 2024-09-12T14:03:41.462 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2872 2024-09-12T14:03:41.465 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2874 2024-09-12T14:03:41.493 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:03:41.512 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:03:41.512 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T14:03:41.512 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:03:41.512 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:03:41.512 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T14:03:41.512 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T14:03:41.516 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:03:41.516 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T14:03:41.516 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:03:41.516 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:03:41.516 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T14:03:41.517 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T14:03:41.519 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:03:41.520 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:03:41.534 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:03:41.541 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:03:41.541 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T14:03:41.541 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:03:41.541 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:03:41.541 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T14:03:41.541 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T14:03:41.558 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2875 2024-09-12T14:03:41.575 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:03:42.486 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:03:42.486 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T14:03:42.487 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:03:42.487 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:03:42.487 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T14:03:42.487 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T14:03:42.487 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2878 2024-09-12T14:03:42.492 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2876 2024-09-12T14:03:42.511 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:03:42.521 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:03:43.535 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:03:43.535 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T14:03:43.535 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:03:43.535 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:03:43.535 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T14:03:43.535 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T14:03:44.483 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2877 2024-09-12T14:03:44.496 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:03:44.496 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T14:03:44.496 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:03:44.496 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:03:44.497 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T14:03:44.497 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T14:03:44.522 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:03:44.525 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:03:44.525 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T14:03:44.525 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:03:44.525 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:03:44.525 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T14:03:44.526 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T14:03:44.528 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:03:44.528 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T14:03:44.528 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:03:44.528 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:03:44.528 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T14:03:44.529 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T14:03:44.539 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:03:44.539 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T14:03:44.539 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:03:44.540 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:03:44.540 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T14:03:44.540 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T14:03:44.582 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:03:44.582 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T14:03:44.582 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:03:44.583 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:03:44.583 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T14:03:44.583 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T14:03:45.495 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2879 2024-09-12T14:03:45.515 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:03:45.515 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T14:03:45.515 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:03:45.515 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:03:45.515 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T14:03:45.515 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T14:03:45.524 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:03:45.524 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T14:03:45.524 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:03:45.524 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:03:45.524 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T14:03:45.524 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T14:03:45.536 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:03:45.585 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2881 2024-09-12T14:03:45.596 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:03:46.475 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2880 2024-09-12T14:03:46.492 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2884 2024-09-12T14:03:46.495 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2883 2024-09-12T14:03:46.497 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2882 2024-09-12T14:03:46.541 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:03:46.541 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:03:46.542 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:03:46.562 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:03:47.482 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2885 2024-09-12T14:03:47.519 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:03:47.525 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:03:47.525 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T14:03:47.525 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:03:47.525 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:03:47.525 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T14:03:47.525 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T14:03:48.461 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2887 2024-09-12T14:03:48.467 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2886 2024-09-12T14:03:48.498 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:03:48.498 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:03:48.520 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2888 2024-09-12T14:03:48.536 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:03:48.538 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:03:48.538 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T14:03:48.538 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:03:48.538 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:03:48.538 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T14:03:48.539 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T14:03:48.599 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:03:48.600 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T14:03:48.600 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:03:48.600 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:03:48.600 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T14:03:48.600 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T14:03:49.525 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2889 2024-09-12T14:03:49.544 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:03:49.544 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T14:03:49.544 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:03:49.544 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:03:49.545 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T14:03:49.545 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T14:03:49.545 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:03:49.545 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T14:03:49.545 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:03:49.545 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:03:49.545 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T14:03:49.545 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T14:03:49.545 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:03:49.546 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T14:03:49.546 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:03:49.546 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:03:49.546 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T14:03:49.546 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T14:03:49.557 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:03:49.565 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:03:49.565 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T14:03:49.565 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:03:49.565 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:03:49.565 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T14:03:49.565 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T14:03:50.523 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:03:50.523 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T14:03:50.523 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:03:50.523 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:03:50.523 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T14:03:50.523 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T14:03:50.561 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2893 2024-09-12T14:03:50.586 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:03:51.502 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:03:51.502 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T14:03:51.502 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:03:51.502 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:03:51.502 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T14:03:51.502 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T14:03:51.503 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:03:51.503 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T14:03:51.503 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:03:51.503 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:03:51.503 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T14:03:51.503 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T14:03:51.506 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2890 2024-09-12T14:03:51.541 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:03:51.541 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T14:03:51.541 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:03:51.541 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:03:51.542 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T14:03:51.542 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T14:03:51.588 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:03:51.686 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2895 2024-09-12T14:03:51.699 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:03:52.514 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2891 2024-09-12T14:03:52.520 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2894 2024-09-12T14:03:52.524 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2892 2024-09-12T14:03:52.561 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:03:52.561 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T14:03:52.561 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:03:52.562 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:03:52.562 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T14:03:52.562 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T14:03:52.562 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:03:52.576 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:03:52.578 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:03:52.605 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2896 2024-09-12T14:03:52.615 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2897 2024-09-12T14:03:52.616 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:03:52.625 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:03:53.478 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2898 2024-09-12T14:03:53.507 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:03:53.589 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:03:53.589 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T14:03:53.589 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:03:53.589 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:03:53.589 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T14:03:53.590 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T14:03:54.591 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:03:54.591 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T14:03:54.591 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:03:54.591 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:03:54.591 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T14:03:54.591 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T14:03:54.702 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:03:54.702 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T14:03:54.702 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:03:54.702 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:03:54.702 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T14:03:54.702 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T14:03:55.526 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2899 2024-09-12T14:03:55.565 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:03:55.565 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T14:03:55.566 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:03:55.566 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:03:55.566 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T14:03:55.566 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T14:03:55.581 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:03:55.581 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T14:03:55.582 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:03:55.582 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:03:55.582 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T14:03:55.582 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T14:03:55.583 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:03:55.583 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T14:03:55.583 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:03:55.583 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:03:55.583 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T14:03:55.583 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T14:03:55.588 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:03:55.618 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:03:55.619 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T14:03:55.619 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:03:55.619 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:03:55.619 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T14:03:55.619 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T14:03:55.625 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2901 2024-09-12T14:03:55.628 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:03:55.628 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T14:03:55.628 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:03:55.628 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:03:55.628 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T14:03:55.628 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T14:03:55.638 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:03:56.512 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:03:56.512 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T14:03:56.512 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:03:56.512 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:03:56.512 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T14:03:56.512 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T14:03:56.530 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2900 2024-09-12T14:03:56.598 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2902 2024-09-12T14:03:56.598 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:03:56.631 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:03:57.587 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2907 2024-09-12T14:03:57.640 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:03:58.488 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2905 2024-09-12T14:03:58.552 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2908 2024-09-12T14:03:58.556 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:03:58.592 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:03:58.592 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T14:03:58.592 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:03:58.593 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:03:58.593 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T14:03:58.593 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T14:03:58.603 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:03:58.642 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:03:58.642 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T14:03:58.642 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:03:58.642 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:03:58.642 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T14:03:58.642 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T14:03:59.546 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2904 2024-09-12T14:03:59.589 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:03:59.602 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:03:59.602 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T14:03:59.602 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:03:59.602 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:03:59.602 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T14:03:59.602 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T14:03:59.633 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:03:59.634 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T14:03:59.634 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:03:59.634 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:03:59.634 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T14:03:59.634 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T14:03:59.653 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2909 2024-09-12T14:03:59.677 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:04:00.537 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2903 2024-09-12T14:04:00.540 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2906 2024-09-12T14:04:00.572 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:04:00.582 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:04:00.634 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2911 2024-09-12T14:04:00.644 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:04:00.644 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T14:04:00.644 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:04:00.644 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:04:00.644 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T14:04:00.644 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T14:04:00.654 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:04:01.551 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2910 2024-09-12T14:04:01.559 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:04:01.559 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T14:04:01.559 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:04:01.559 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:04:01.559 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T14:04:01.559 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T14:04:01.577 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T14:04:01.576+0000 7f67dc600640 -1 rbd::mirror::image_replayer::snapshot::Replayer: 0x56055bb77000 handle_unlink_peer: failed to unlink local peer from remote image: (22) Invalid argument 2024-09-12T14:04:01.588 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:04:01.607 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:04:01.607 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T14:04:01.607 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:04:01.607 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:04:01.607 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T14:04:01.607 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T14:04:02.544 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2912 2024-09-12T14:04:02.585 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:04:02.592 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:04:02.592 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T14:04:02.592 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:04:02.592 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:04:02.592 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T14:04:02.592 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T14:04:02.629 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2914 2024-09-12T14:04:02.652 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:04:02.680 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:04:02.681 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T14:04:02.681 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:04:02.681 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:04:02.681 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T14:04:02.681 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T14:04:03.470 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2913 2024-09-12T14:04:03.515 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2915 2024-09-12T14:04:03.519 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:04:03.537 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:04:03.574 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:04:03.575 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T14:04:03.575 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:04:03.575 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:04:03.575 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T14:04:03.575 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T14:04:03.585 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:04:03.585 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T14:04:03.585 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:04:03.585 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:04:03.585 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T14:04:03.586 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T14:04:03.658 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:04:03.658 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T14:04:03.658 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:04:03.658 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:04:03.658 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T14:04:03.658 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T14:04:04.591 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:04:04.592 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T14:04:04.592 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:04:04.592 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:04:04.592 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T14:04:04.592 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T14:04:04.600 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T14:04:04.600+0000 7f67dc600640 -1 rbd::mirror::image_replayer::snapshot::Replayer: 0x560556e54000 handle_unlink_peer: failed to unlink local peer from remote image: (22) Invalid argument 2024-09-12T14:04:04.636 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2918 2024-09-12T14:04:04.659 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:04:04.672 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2919 2024-09-12T14:04:04.688 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:04:05.567 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2916 2024-09-12T14:04:05.573 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2917 2024-09-12T14:04:05.588 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:04:05.588 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T14:04:05.588 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:04:05.588 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:04:05.588 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T14:04:05.588 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T14:04:05.613 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:04:05.628 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:04:05.656 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:04:05.656 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T14:04:05.656 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:04:05.657 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:04:05.657 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T14:04:05.657 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T14:04:05.663 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2921 2024-09-12T14:04:05.673 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:04:06.522 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:04:06.523 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T14:04:06.523 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:04:06.523 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:04:06.523 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T14:04:06.523 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T14:04:06.540 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:04:06.541 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T14:04:06.541 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:04:06.541 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:04:06.541 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T14:04:06.541 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T14:04:06.569 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2920 2024-09-12T14:04:06.594 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:04:06.615 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2922 2024-09-12T14:04:06.633 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:04:07.618 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2924 2024-09-12T14:04:07.636 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:04:07.662 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:04:07.662 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T14:04:07.662 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:04:07.662 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:04:07.662 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T14:04:07.662 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T14:04:07.691 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:04:07.691 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T14:04:07.691 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:04:07.691 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:04:07.691 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T14:04:07.692 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T14:04:08.471 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2923 2024-09-12T14:04:08.507 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2925 2024-09-12T14:04:08.515 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:04:08.526 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:04:08.617 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:04:08.617 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T14:04:08.617 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:04:08.617 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:04:08.617 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T14:04:08.617 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T14:04:08.631 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:04:08.631 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T14:04:08.631 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:04:08.631 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:04:08.631 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T14:04:08.632 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T14:04:08.676 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:04:08.676 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T14:04:08.676 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:04:08.676 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:04:08.676 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T14:04:08.676 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T14:04:09.504 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2927 2024-09-12T14:04:09.527 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2926 2024-09-12T14:04:09.549 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:04:09.553 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2928 2024-09-12T14:04:09.558 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:04:09.558 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2929 2024-09-12T14:04:09.584 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:04:09.589 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:04:09.598 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:04:09.598 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T14:04:09.598 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:04:09.598 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:04:09.598 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T14:04:09.598 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T14:04:09.636 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:04:09.636 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T14:04:09.636 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:04:09.636 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:04:09.636 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T14:04:09.636 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T14:04:10.612 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2932 2024-09-12T14:04:10.624 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2931 2024-09-12T14:04:10.630 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:04:10.633 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:04:10.639 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:04:10.639 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T14:04:10.639 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:04:10.640 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:04:10.640 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T14:04:10.640 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T14:04:11.514 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:04:11.514 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T14:04:11.514 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:04:11.514 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:04:11.514 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T14:04:11.514 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T14:04:11.527 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:04:11.528 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T14:04:11.528 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:04:11.528 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:04:11.528 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T14:04:11.528 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T14:04:11.573 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2930 2024-09-12T14:04:11.606 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:04:12.555 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:04:12.555 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T14:04:12.555 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:04:12.555 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:04:12.555 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T14:04:12.555 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T14:04:12.562 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:04:12.562 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T14:04:12.562 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:04:12.562 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:04:12.562 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T14:04:12.562 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T14:04:12.587 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:04:12.588 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T14:04:12.588 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:04:12.588 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:04:12.588 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T14:04:12.588 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T14:04:12.593 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:04:12.593 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T14:04:12.593 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:04:12.593 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:04:12.593 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T14:04:12.593 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T14:04:12.599 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T14:04:12.600+0000 7f67dc600640 -1 rbd::mirror::image_replayer::snapshot::Replayer: 0x5605563f0400 handle_unlink_peer: failed to unlink local peer from remote image: (22) Invalid argument 2024-09-12T14:04:12.634 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2934 2024-09-12T14:04:12.654 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:04:13.456 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2933 2024-09-12T14:04:13.504 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2935 2024-09-12T14:04:13.507 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:04:13.526 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2938 2024-09-12T14:04:13.534 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:04:13.548 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:04:13.633 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:04:13.633 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T14:04:13.634 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:04:13.634 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:04:13.634 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T14:04:13.634 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T14:04:13.636 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:04:13.636 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T14:04:13.637 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:04:13.637 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:04:13.637 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T14:04:13.637 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T14:04:14.504 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2936 2024-09-12T14:04:14.523 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2939 2024-09-12T14:04:14.538 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2937 2024-09-12T14:04:14.546 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:04:14.554 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:04:14.559 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2940 2024-09-12T14:04:14.563 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:04:14.585 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:04:14.610 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:04:14.610 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T14:04:14.610 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:04:14.610 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:04:14.610 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T14:04:14.610 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T14:04:15.620 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2942 2024-09-12T14:04:15.629 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:04:15.657 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:04:15.658 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T14:04:15.658 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:04:15.658 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:04:15.658 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T14:04:15.658 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T14:04:16.512 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:04:16.512 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T14:04:16.512 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:04:16.513 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:04:16.513 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T14:04:16.513 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T14:04:16.538 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:04:16.539 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T14:04:16.539 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:04:16.539 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:04:16.539 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T14:04:16.539 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T14:04:16.551 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:04:16.552 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T14:04:16.552 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:04:16.552 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:04:16.552 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T14:04:16.552 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T14:04:16.591 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2941 2024-09-12T14:04:16.622 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:04:17.550 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${SNAP_PIDS[@]} 2024-09-12T14:04:17.550 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 130487 2024-09-12T14:04:17.559 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:04:17.559 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T14:04:17.560 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:04:17.560 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:04:17.560 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T14:04:17.560 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T14:04:17.566 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:04:17.566 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T14:04:17.567 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:04:17.567 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:04:17.567 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T14:04:17.567 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T14:04:17.629 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T14:04:17.629+0000 7f67dc600640 -1 rbd::mirror::image_replayer::snapshot::Replayer: 0x56055bb4e800 handle_unlink_peer: failed to unlink local peer from remote image: (22) Invalid argument 2024-09-12T14:04:17.644 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2946 2024-09-12T14:04:17.650 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2945 2024-09-12T14:04:17.662 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:04:17.668 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:04:18.455 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2943 2024-09-12T14:04:18.485 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2944 2024-09-12T14:04:18.496 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:04:18.511 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:04:18.633 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${SNAP_PIDS[@]} 2024-09-12T14:04:18.633 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 130619 2024-09-12T14:04:18.633 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${SNAP_PIDS[@]} 2024-09-12T14:04:18.633 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 130751 2024-09-12T14:04:19.489 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2947 2024-09-12T14:04:19.497 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2948 2024-09-12T14:04:19.507 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:04:19.507 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:04:19.626 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:04:19.626 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T14:04:19.626 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:04:19.626 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:04:19.626 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T14:04:19.626 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T14:04:20.611 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2949 2024-09-12T14:04:20.625 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:04:20.670 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:04:20.670 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T14:04:20.671 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:04:20.671 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:04:20.671 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T14:04:20.671 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T14:04:21.499 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${SNAP_PIDS[@]} 2024-09-12T14:04:21.500 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 130903 2024-09-12T14:04:21.516 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${SNAP_PIDS[@]} 2024-09-12T14:04:21.516 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 131056 2024-09-12T14:04:21.516 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${SNAP_PIDS[@]} 2024-09-12T14:04:21.516 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 131209 2024-09-12T14:04:22.510 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:04:22.511 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:04:22.511 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T14:04:22.511 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T14:04:22.511 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:04:22.511 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:04:22.511 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:04:22.511 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:04:22.511 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T14:04:22.511 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T14:04:22.511 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T14:04:22.511 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T14:04:22.636 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2950 2024-09-12T14:04:22.657 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:04:23.632 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${SNAP_PIDS[@]} 2024-09-12T14:04:23.633 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 131403 2024-09-12T14:04:25.465 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2952 2024-09-12T14:04:25.471 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2951 2024-09-12T14:04:25.505 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:04:25.512 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:04:25.660 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${SNAP_PIDS[@]} 2024-09-12T14:04:25.661 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 131577 2024-09-12T14:04:28.508 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${SNAP_PIDS[@]} 2024-09-12T14:04:28.509 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 131751 2024-09-12T14:04:28.515 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ (( RET != 0 )) 2024-09-12T14:04:28.515 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${WORKLOAD_PIDS[@]} 2024-09-12T14:04:28.515 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 130356 2024-09-12T14:06:53.283 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ ret=124 2024-09-12T14:06:53.283 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ (( ret != 124 )) 2024-09-12T14:06:53.283 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${WORKLOAD_PIDS[@]} 2024-09-12T14:06:53.283 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 130488 2024-09-12T14:06:54.109 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ ret=124 2024-09-12T14:06:54.109 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ (( ret != 124 )) 2024-09-12T14:06:54.109 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${WORKLOAD_PIDS[@]} 2024-09-12T14:06:54.109 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 130620 2024-09-12T14:06:55.137 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ ret=124 2024-09-12T14:06:55.137 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ (( ret != 124 )) 2024-09-12T14:06:55.137 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${WORKLOAD_PIDS[@]} 2024-09-12T14:06:55.137 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 130752 2024-09-12T14:06:56.141 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ ret=124 2024-09-12T14:06:56.141 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ (( ret != 124 )) 2024-09-12T14:06:56.142 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${WORKLOAD_PIDS[@]} 2024-09-12T14:06:56.142 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 130904 2024-09-12T14:06:57.216 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ ret=124 2024-09-12T14:06:57.216 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ (( ret != 124 )) 2024-09-12T14:06:57.216 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${WORKLOAD_PIDS[@]} 2024-09-12T14:06:57.216 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 131057 2024-09-12T14:06:58.352 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ ret=124 2024-09-12T14:06:58.353 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ (( ret != 124 )) 2024-09-12T14:06:58.353 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${WORKLOAD_PIDS[@]} 2024-09-12T14:06:58.353 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 131210 2024-09-12T14:06:59.685 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ ret=124 2024-09-12T14:06:59.685 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ (( ret != 124 )) 2024-09-12T14:06:59.686 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${WORKLOAD_PIDS[@]} 2024-09-12T14:06:59.686 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 131404 2024-09-12T14:07:01.370 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ ret=124 2024-09-12T14:07:01.371 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ (( ret != 124 )) 2024-09-12T14:07:01.371 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${WORKLOAD_PIDS[@]} 2024-09-12T14:07:01.371 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 131578 2024-09-12T14:07:02.371 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ ret=124 2024-09-12T14:07:02.372 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ (( ret != 124 )) 2024-09-12T14:07:02.372 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${WORKLOAD_PIDS[@]} 2024-09-12T14:07:02.372 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 131752 2024-09-12T14:07:03.448 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ ret=124 2024-09-12T14:07:03.449 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ (( ret != 124 )) 2024-09-12T14:07:03.449 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ (( RET != 0 )) 2024-09-12T14:07:03.449 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T14:07:03.450 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ COMPARE_PIDS+=($!) 2024-09-12T14:07:03.450 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T14:07:03.450 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ compare_demoted_promoted_image 1 2024-09-12T14:07:03.450 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local dev=/dev/rbd0 2024-09-12T14:07:03.450 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ COMPARE_PIDS+=($!) 2024-09-12T14:07:03.451 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local img=image-primary1 2024-09-12T14:07:03.451 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mntpt=test-primary1 2024-09-12T14:07:03.451 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local demote_md5 promote_md5 2024-09-12T14:07:03.451 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo umount test-primary1 2024-09-12T14:07:03.451 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T14:07:03.451 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ compare_demoted_promoted_image 2 2024-09-12T14:07:03.451 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local dev=/dev/rbd1 2024-09-12T14:07:03.451 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local img=image-primary2 2024-09-12T14:07:03.451 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mntpt=test-primary2 2024-09-12T14:07:03.451 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local demote_md5 promote_md5 2024-09-12T14:07:03.451 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo umount test-primary2 2024-09-12T14:07:03.452 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ COMPARE_PIDS+=($!) 2024-09-12T14:07:03.452 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T14:07:03.452 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ compare_demoted_promoted_image 3 2024-09-12T14:07:03.453 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local dev=/dev/rbd2 2024-09-12T14:07:03.453 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local img=image-primary3 2024-09-12T14:07:03.453 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mntpt=test-primary3 2024-09-12T14:07:03.453 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local demote_md5 promote_md5 2024-09-12T14:07:03.453 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo umount test-primary3 2024-09-12T14:07:03.453 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ COMPARE_PIDS+=($!) 2024-09-12T14:07:03.453 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T14:07:03.453 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ compare_demoted_promoted_image 4 2024-09-12T14:07:03.454 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local dev=/dev/rbd3 2024-09-12T14:07:03.454 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local img=image-primary4 2024-09-12T14:07:03.454 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mntpt=test-primary4 2024-09-12T14:07:03.454 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local demote_md5 promote_md5 2024-09-12T14:07:03.454 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo umount test-primary4 2024-09-12T14:07:03.454 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ COMPARE_PIDS+=($!) 2024-09-12T14:07:03.454 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T14:07:03.454 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ compare_demoted_promoted_image 5 2024-09-12T14:07:03.455 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ COMPARE_PIDS+=($!) 2024-09-12T14:07:03.455 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local dev=/dev/rbd4 2024-09-12T14:07:03.455 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local img=image-primary5 2024-09-12T14:07:03.455 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mntpt=test-primary5 2024-09-12T14:07:03.455 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local demote_md5 promote_md5 2024-09-12T14:07:03.455 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T14:07:03.455 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo umount test-primary5 2024-09-12T14:07:03.455 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ compare_demoted_promoted_image 6 2024-09-12T14:07:03.455 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local dev=/dev/rbd5 2024-09-12T14:07:03.455 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local img=image-primary6 2024-09-12T14:07:03.455 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mntpt=test-primary6 2024-09-12T14:07:03.455 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local demote_md5 promote_md5 2024-09-12T14:07:03.456 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo umount test-primary6 2024-09-12T14:07:03.456 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ COMPARE_PIDS+=($!) 2024-09-12T14:07:03.456 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T14:07:03.456 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ compare_demoted_promoted_image 7 2024-09-12T14:07:03.456 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ COMPARE_PIDS+=($!) 2024-09-12T14:07:03.456 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T14:07:03.456 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local dev=/dev/rbd6 2024-09-12T14:07:03.457 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local img=image-primary7 2024-09-12T14:07:03.457 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mntpt=test-primary7 2024-09-12T14:07:03.457 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local demote_md5 promote_md5 2024-09-12T14:07:03.457 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo umount test-primary7 2024-09-12T14:07:03.457 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ compare_demoted_promoted_image 8 2024-09-12T14:07:03.457 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local dev=/dev/rbd7 2024-09-12T14:07:03.458 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ compare_demoted_promoted_image 9 2024-09-12T14:07:03.458 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local dev=/dev/rbd8 2024-09-12T14:07:03.458 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local img=image-primary9 2024-09-12T14:07:03.458 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mntpt=test-primary9 2024-09-12T14:07:03.458 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local demote_md5 promote_md5 2024-09-12T14:07:03.458 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo umount test-primary9 2024-09-12T14:07:03.458 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local img=image-primary8 2024-09-12T14:07:03.459 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ COMPARE_PIDS+=($!) 2024-09-12T14:07:03.459 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mntpt=test-primary8 2024-09-12T14:07:03.459 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local demote_md5 promote_md5 2024-09-12T14:07:03.459 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo umount test-primary8 2024-09-12T14:07:03.459 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T14:07:03.461 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ COMPARE_PIDS+=($!) 2024-09-12T14:07:03.461 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ compare_demoted_promoted_image 10 2024-09-12T14:07:03.461 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${COMPARE_PIDS[@]} 2024-09-12T14:07:03.461 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 137665 2024-09-12T14:07:03.466 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local dev=/dev/rbd9 2024-09-12T14:07:03.466 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local img=image-primary10 2024-09-12T14:07:03.466 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mntpt=test-primary10 2024-09-12T14:07:03.466 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local demote_md5 promote_md5 2024-09-12T14:07:03.466 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo umount test-primary10 2024-09-12T14:07:03.967 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ awk '{print $1}' 2024-09-12T14:07:03.968 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo md5sum /dev/rbd1 2024-09-12T14:07:04.024 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo md5sum /dev/rbd0 2024-09-12T14:07:04.024 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ awk '{print $1}' 2024-09-12T14:07:04.044 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ awk '{print $1}' 2024-09-12T14:07:04.057 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo md5sum /dev/rbd3 2024-09-12T14:07:04.100 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo md5sum /dev/rbd2 2024-09-12T14:07:04.100 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ awk '{print $1}' 2024-09-12T14:07:04.117 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo md5sum /dev/rbd6 2024-09-12T14:07:04.118 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ awk '{print $1}' 2024-09-12T14:07:04.251 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo md5sum /dev/rbd7 2024-09-12T14:07:04.253 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ awk '{print $1}' 2024-09-12T14:07:04.255 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo md5sum /dev/rbd5 2024-09-12T14:07:04.256 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ awk '{print $1}' 2024-09-12T14:07:04.301 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo md5sum /dev/rbd4 2024-09-12T14:07:04.301 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo md5sum /dev/rbd8 2024-09-12T14:07:04.302 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ awk '{print $1}' 2024-09-12T14:07:04.306 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ awk '{print $1}' 2024-09-12T14:07:04.308 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ awk '{print $1}' 2024-09-12T14:07:04.308 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo md5sum /dev/rbd9 2024-09-12T14:07:57.329 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ demote_md5=726359b6503be8b926d2ecbe06764c40 2024-09-12T14:07:57.330 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo rbd --cluster cluster1 device unmap -t krbd mirror/image-primary2 2024-09-12T14:07:57.702 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ demote_image cluster1 mirror image-primary2 2024-09-12T14:07:57.702 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:07:57.702 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:07:57.702 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T14:07:57.702 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 mirror image demote mirror/image-primary2 2024-09-12T14:07:57.709 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ demote_md5=89cebaf0f069635630e707839fd61396 2024-09-12T14:07:57.709 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo rbd --cluster cluster1 device unmap -t krbd mirror/image-primary7 2024-09-12T14:07:57.965 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ demote_image cluster1 mirror image-primary7 2024-09-12T14:07:57.965 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:07:57.965 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:07:57.965 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T14:07:57.965 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 mirror image demote mirror/image-primary7 2024-09-12T14:07:58.786 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ demote_md5=42539596e7b0352ed1e2ab5083d992fe 2024-09-12T14:07:58.786 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo rbd --cluster cluster1 device unmap -t krbd mirror/image-primary8 2024-09-12T14:07:58.960 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ demote_md5=3318816507f6cb2bf4e2cb61d48dd326 2024-09-12T14:07:58.960 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo rbd --cluster cluster1 device unmap -t krbd mirror/image-primary4 2024-09-12T14:07:59.100 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ demote_image cluster1 mirror image-primary8 2024-09-12T14:07:59.100 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:07:59.100 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:07:59.100 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T14:07:59.100 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 mirror image demote mirror/image-primary8 2024-09-12T14:07:59.467 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ demote_image cluster1 mirror image-primary4 2024-09-12T14:07:59.467 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:07:59.468 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:07:59.468 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T14:07:59.468 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 mirror image demote mirror/image-primary4 2024-09-12T14:07:59.666 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ demote_md5=761401645037fbc384db7229420cff4e 2024-09-12T14:07:59.667 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo rbd --cluster cluster1 device unmap -t krbd mirror/image-primary10 2024-09-12T14:07:59.696 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Image demoted to non-primary 2024-09-12T14:07:59.699 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Image demoted to non-primary 2024-09-12T14:07:59.714 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Image demoted to non-primary 2024-09-12T14:07:59.724 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary7 up+unknown 2024-09-12T14:07:59.724 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:07:59.724 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:07:59.724 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T14:07:59.724 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:07:59.724 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:07:59.725 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:07:59.725 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:07:59.725 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T14:07:59.726 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary2 up+unknown 2024-09-12T14:07:59.726 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:07:59.726 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:07:59.726 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T14:07:59.726 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:07:59.726 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:07:59.726 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:07:59.726 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:07:59.726 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T14:07:59.739 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ demote_md5=a58afde9a16731c66f673f9349cf3686 2024-09-12T14:07:59.739 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo rbd --cluster cluster1 device unmap -t krbd mirror/image-primary3 2024-09-12T14:07:59.741 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary4 up+unknown 2024-09-12T14:07:59.741 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:07:59.742 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:07:59.742 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T14:07:59.742 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:07:59.742 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:07:59.742 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:07:59.742 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:07:59.742 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T14:07:59.908 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ demote_image cluster1 mirror image-primary10 2024-09-12T14:07:59.910 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:07:59.911 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:07:59.914 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T14:07:59.915 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 mirror image demote mirror/image-primary10 2024-09-12T14:07:59.929 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ demote_image cluster1 mirror image-primary3 2024-09-12T14:07:59.930 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:07:59.930 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:07:59.930 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T14:07:59.930 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 mirror image demote mirror/image-primary3 2024-09-12T14:08:00.642 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ demote_md5=08b3821aa5466400f387d2934beb2fef 2024-09-12T14:08:00.642 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo rbd --cluster cluster1 device unmap -t krbd mirror/image-primary9 2024-09-12T14:08:00.675 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Image demoted to non-primary 2024-09-12T14:08:00.691 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary8 up+unknown 2024-09-12T14:08:00.691 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:08:00.691 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:08:00.693 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T14:08:00.693 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:08:00.693 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:08:00.693 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:08:00.693 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:08:00.694 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T14:08:00.700 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Image demoted to non-primary 2024-09-12T14:08:00.712 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary10 up+unknown 2024-09-12T14:08:00.712 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:08:00.712 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:08:00.712 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T14:08:00.712 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:08:00.712 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:08:00.713 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:08:00.713 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:08:00.713 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T14:08:00.728 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary7 up+unknown '' '' 2024-09-12T14:08:00.729 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:08:00.729 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:08:00.729 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T14:08:00.729 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:08:00.729 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:08:00.729 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:08:00.729 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:08:00.730 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:08:00.730 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary7 2024-09-12T14:08:00.730 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary2 up+unknown '' '' 2024-09-12T14:08:00.730 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:08:00.731 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:08:00.731 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T14:08:00.731 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:08:00.731 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:08:00.731 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:08:00.731 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:08:00.735 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:08:00.735 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary2 2024-09-12T14:08:00.745 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary4 up+unknown '' '' 2024-09-12T14:08:00.745 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:08:00.745 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:08:00.745 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T14:08:00.745 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:08:00.746 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:08:00.746 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:08:00.746 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:08:00.746 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:08:00.747 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary4 2024-09-12T14:08:00.748 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ demote_md5=527686362ecdfb4b7f3c247431caa979 2024-09-12T14:08:00.749 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo rbd --cluster cluster1 device unmap -t krbd mirror/image-primary5 2024-09-12T14:08:00.789 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ demote_image cluster1 mirror image-primary9 2024-09-12T14:08:00.789 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:08:00.789 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:08:00.790 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T14:08:00.790 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 mirror image demote mirror/image-primary9 2024-09-12T14:08:00.864 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary2: 2024-09-12T14:08:00.864 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: ccaafd72-518c-4588-86d7-96c322bcf1bd 2024-09-12T14:08:00.864 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T14:08:00.864 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T14:08:00.864 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T14:08:00.864 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 14:07:47' 2024-09-12T14:08:00.864 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T14:08:00.876 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T14:08:00.876 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:08:00.876 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 2 2024-09-12T14:08:00.886 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary7: 2024-09-12T14:08:00.886 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 5795f6f3-43c3-4e14-89d9-f4798e240cc6 2024-09-12T14:08:00.886 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T14:08:00.887 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T14:08:00.887 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T14:08:00.887 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 14:07:47' 2024-09-12T14:08:00.887 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T14:08:00.898 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ demote_image cluster1 mirror image-primary5 2024-09-12T14:08:00.898 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:08:00.899 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:08:00.899 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T14:08:00.899 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 mirror image demote mirror/image-primary5 2024-09-12T14:08:00.900 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary4: 2024-09-12T14:08:00.900 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: a6cc92cf-b177-4aea-95c5-b61892fd9062 2024-09-12T14:08:00.901 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T14:08:00.901 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T14:08:00.901 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T14:08:00.901 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 14:07:47' 2024-09-12T14:08:00.901 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T14:08:00.905 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T14:08:00.905 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:08:00.905 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 2 2024-09-12T14:08:00.913 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T14:08:00.913 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:08:00.913 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 2 2024-09-12T14:08:01.181 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ demote_md5=0393d04de9574824e5fa7aefcef31dd8 2024-09-12T14:08:01.182 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo rbd --cluster cluster1 device unmap -t krbd mirror/image-primary6 2024-09-12T14:08:01.340 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ demote_image cluster1 mirror image-primary6 2024-09-12T14:08:01.340 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:08:01.341 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:08:01.341 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T14:08:01.341 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 mirror image demote mirror/image-primary6 2024-09-12T14:08:01.377 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ demote_md5=f23f150e00f53f7158113196b6b4bd34 2024-09-12T14:08:01.377 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo rbd --cluster cluster1 device unmap -t krbd mirror/image-primary1 2024-09-12T14:08:01.475 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ demote_image cluster1 mirror image-primary1 2024-09-12T14:08:01.475 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:08:01.475 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:08:01.475 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T14:08:01.475 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 mirror image demote mirror/image-primary1 2024-09-12T14:08:01.700 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary8 up+unknown '' '' 2024-09-12T14:08:01.701 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:08:01.701 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:08:01.701 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T14:08:01.701 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:08:01.701 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:08:01.701 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:08:01.701 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:08:01.701 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:08:01.702 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary8 2024-09-12T14:08:01.704 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Image demoted to non-primary 2024-09-12T14:08:01.715 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary10 up+unknown '' '' 2024-09-12T14:08:01.715 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:08:01.715 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:08:01.715 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T14:08:01.715 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:08:01.715 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:08:01.715 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:08:01.715 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:08:01.716 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:08:01.716 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary10 2024-09-12T14:08:01.716 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary9 up+unknown 2024-09-12T14:08:01.716 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:08:01.717 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:08:01.717 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T14:08:01.717 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:08:01.717 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:08:01.717 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:08:01.717 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:08:01.717 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T14:08:01.771 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary8: 2024-09-12T14:08:01.772 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: fa74ee32-e225-4343-82da-f93063849113 2024-09-12T14:08:01.772 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T14:08:01.772 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T14:08:01.772 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T14:08:01.772 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 14:07:47' 2024-09-12T14:08:01.772 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T14:08:01.774 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T14:08:01.774 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:08:01.774 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 2 2024-09-12T14:08:01.788 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary10: 2024-09-12T14:08:01.788 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: b0e88823-56d3-40a7-88c0-8d4c22f008ee 2024-09-12T14:08:01.788 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T14:08:01.789 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T14:08:01.789 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T14:08:01.789 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 14:07:47' 2024-09-12T14:08:01.789 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T14:08:01.791 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T14:08:01.791 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:08:01.791 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 2 2024-09-12T14:08:02.691 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Image demoted to non-primary 2024-09-12T14:08:02.692 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Image demoted to non-primary 2024-09-12T14:08:02.706 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary1 up+unknown 2024-09-12T14:08:02.707 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:08:02.707 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:08:02.707 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T14:08:02.707 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:08:02.707 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:08:02.707 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:08:02.707 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:08:02.707 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T14:08:02.711 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary5 up+unknown 2024-09-12T14:08:02.711 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:08:02.711 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:08:02.711 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T14:08:02.711 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:08:02.712 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:08:02.712 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:08:02.712 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:08:02.712 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T14:08:02.719 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary9 up+unknown '' '' 2024-09-12T14:08:02.719 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:08:02.719 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:08:02.719 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T14:08:02.719 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:08:02.719 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:08:02.719 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:08:02.719 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:08:02.720 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:08:02.720 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary9 2024-09-12T14:08:02.797 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary9: 2024-09-12T14:08:02.797 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 5991a19a-9e33-46ee-965b-85f990022e98 2024-09-12T14:08:02.798 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T14:08:02.798 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T14:08:02.798 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T14:08:02.798 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 14:07:47' 2024-09-12T14:08:02.798 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T14:08:02.800 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T14:08:02.800 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:08:02.800 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 2 2024-09-12T14:08:02.884 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary2 up+unknown '' '' 2024-09-12T14:08:02.884 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:08:02.884 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:08:02.885 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T14:08:02.885 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:08:02.885 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:08:02.885 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:08:02.885 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:08:02.886 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:08:02.887 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary2 2024-09-12T14:08:02.910 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary7 up+unknown '' '' 2024-09-12T14:08:02.911 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:08:02.911 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:08:02.911 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T14:08:02.911 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:08:02.911 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:08:02.911 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:08:02.911 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:08:02.912 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:08:02.912 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary7 2024-09-12T14:08:02.922 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary4 up+unknown '' '' 2024-09-12T14:08:02.922 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:08:02.922 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:08:02.923 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T14:08:02.923 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:08:02.923 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:08:02.923 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:08:02.923 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:08:02.924 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:08:02.924 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary4 2024-09-12T14:08:03.000 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary2: 2024-09-12T14:08:03.000 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: ccaafd72-518c-4588-86d7-96c322bcf1bd 2024-09-12T14:08:03.000 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T14:08:03.000 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T14:08:03.000 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T14:08:03.000 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 14:07:47' 2024-09-12T14:08:03.001 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T14:08:03.003 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T14:08:03.003 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:08:03.003 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 4 2024-09-12T14:08:03.018 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary7: 2024-09-12T14:08:03.018 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 5795f6f3-43c3-4e14-89d9-f4798e240cc6 2024-09-12T14:08:03.018 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T14:08:03.019 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T14:08:03.019 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T14:08:03.019 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 14:07:47' 2024-09-12T14:08:03.019 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T14:08:03.021 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T14:08:03.021 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:08:03.021 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 4 2024-09-12T14:08:03.023 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary4: 2024-09-12T14:08:03.023 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: a6cc92cf-b177-4aea-95c5-b61892fd9062 2024-09-12T14:08:03.023 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T14:08:03.023 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T14:08:03.023 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T14:08:03.023 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 14:07:47' 2024-09-12T14:08:03.023 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T14:08:03.025 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T14:08:03.026 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:08:03.026 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 4 2024-09-12T14:08:03.529 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Image demoted to non-primary 2024-09-12T14:08:03.546 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary3 up+unknown 2024-09-12T14:08:03.546 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:08:03.546 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:08:03.547 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T14:08:03.547 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:08:03.547 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:08:03.547 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:08:03.547 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:08:03.547 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T14:08:03.709 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary1 up+unknown '' '' 2024-09-12T14:08:03.709 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:08:03.709 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:08:03.709 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T14:08:03.709 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:08:03.709 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:08:03.710 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:08:03.710 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:08:03.710 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:08:03.710 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary1 2024-09-12T14:08:03.713 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary5 up+unknown '' '' 2024-09-12T14:08:03.714 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:08:03.714 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:08:03.714 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T14:08:03.714 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:08:03.714 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:08:03.714 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:08:03.714 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:08:03.715 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:08:03.716 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary5 2024-09-12T14:08:03.777 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary8 up+unknown '' '' 2024-09-12T14:08:03.777 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:08:03.777 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:08:03.777 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T14:08:03.777 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:08:03.777 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:08:03.777 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:08:03.778 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:08:03.778 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:08:03.778 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary8 2024-09-12T14:08:03.784 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary1: 2024-09-12T14:08:03.784 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 9bf3eb7e-eb1b-464c-b684-743ee2313f71 2024-09-12T14:08:03.784 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T14:08:03.784 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T14:08:03.784 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T14:08:03.784 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 14:07:47' 2024-09-12T14:08:03.784 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T14:08:03.787 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T14:08:03.787 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:08:03.787 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 2 2024-09-12T14:08:03.791 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary5: 2024-09-12T14:08:03.791 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 399c5b9a-89f0-4680-96c5-f2cac59115cf 2024-09-12T14:08:03.791 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T14:08:03.791 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T14:08:03.791 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T14:08:03.791 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 14:07:47' 2024-09-12T14:08:03.791 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T14:08:03.793 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary10 up+unknown '' '' 2024-09-12T14:08:03.793 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:08:03.793 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:08:03.793 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T14:08:03.794 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:08:03.794 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:08:03.794 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:08:03.794 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:08:03.794 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T14:08:03.795 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:08:03.795 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 2 2024-09-12T14:08:03.795 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:08:03.795 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary10 2024-09-12T14:08:03.853 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary8: 2024-09-12T14:08:03.853 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: fa74ee32-e225-4343-82da-f93063849113 2024-09-12T14:08:03.853 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T14:08:03.853 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T14:08:03.853 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T14:08:03.853 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 14:07:47' 2024-09-12T14:08:03.854 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T14:08:03.856 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T14:08:03.856 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:08:03.856 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 4 2024-09-12T14:08:03.865 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary10: 2024-09-12T14:08:03.865 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: b0e88823-56d3-40a7-88c0-8d4c22f008ee 2024-09-12T14:08:03.865 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T14:08:03.865 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T14:08:03.865 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T14:08:03.865 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 14:07:47' 2024-09-12T14:08:03.865 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T14:08:03.868 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T14:08:03.868 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:08:03.868 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 4 2024-09-12T14:08:04.515 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Image demoted to non-primary 2024-09-12T14:08:04.527 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary6 up+unknown 2024-09-12T14:08:04.527 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:08:04.527 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:08:04.527 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T14:08:04.527 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:08:04.527 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:08:04.527 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:08:04.527 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:08:04.527 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T14:08:04.549 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary3 up+unknown '' '' 2024-09-12T14:08:04.549 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:08:04.549 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:08:04.549 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T14:08:04.549 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:08:04.549 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:08:04.549 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:08:04.550 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:08:04.550 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:08:04.550 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary3 2024-09-12T14:08:04.625 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary3: 2024-09-12T14:08:04.626 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: d356f103-adde-4c7a-855e-96419b23abef 2024-09-12T14:08:04.626 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T14:08:04.626 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T14:08:04.626 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T14:08:04.626 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 14:07:47' 2024-09-12T14:08:04.626 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T14:08:04.628 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T14:08:04.628 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:08:04.629 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 2 2024-09-12T14:08:04.803 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary9 up+unknown '' '' 2024-09-12T14:08:04.803 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:08:04.803 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:08:04.803 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T14:08:04.803 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:08:04.803 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:08:04.804 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:08:04.804 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:08:04.804 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:08:04.804 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary9 2024-09-12T14:08:04.879 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary9: 2024-09-12T14:08:04.879 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 5991a19a-9e33-46ee-965b-85f990022e98 2024-09-12T14:08:04.879 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T14:08:04.879 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T14:08:04.879 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T14:08:04.879 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 14:07:47' 2024-09-12T14:08:04.879 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T14:08:04.881 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T14:08:04.881 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:08:04.881 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 4 2024-09-12T14:08:05.530 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary6 up+unknown '' '' 2024-09-12T14:08:05.530 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:08:05.530 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:08:05.530 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T14:08:05.531 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:08:05.531 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:08:05.531 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:08:05.531 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:08:05.531 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:08:05.531 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary6 2024-09-12T14:08:05.614 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary6: 2024-09-12T14:08:05.614 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: f8d03c2d-535f-4038-a91f-c159711e8e6c 2024-09-12T14:08:05.614 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T14:08:05.614 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T14:08:05.614 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T14:08:05.614 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 14:07:47' 2024-09-12T14:08:05.614 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T14:08:05.616 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T14:08:05.617 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:08:05.617 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 2 2024-09-12T14:08:05.789 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary1 up+unknown '' '' 2024-09-12T14:08:05.789 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:08:05.789 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:08:05.789 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T14:08:05.789 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:08:05.789 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:08:05.789 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:08:05.790 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:08:05.790 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:08:05.790 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary1 2024-09-12T14:08:05.796 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary5 up+unknown '' '' 2024-09-12T14:08:05.796 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:08:05.796 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:08:05.796 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T14:08:05.796 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:08:05.796 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:08:05.796 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:08:05.797 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:08:05.797 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:08:05.797 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary5 2024-09-12T14:08:05.863 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary1: 2024-09-12T14:08:05.864 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 9bf3eb7e-eb1b-464c-b684-743ee2313f71 2024-09-12T14:08:05.864 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T14:08:05.864 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T14:08:05.864 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T14:08:05.864 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 14:07:47' 2024-09-12T14:08:05.864 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T14:08:05.867 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T14:08:05.867 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:08:05.867 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 4 2024-09-12T14:08:05.871 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary5: 2024-09-12T14:08:05.871 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 399c5b9a-89f0-4680-96c5-f2cac59115cf 2024-09-12T14:08:05.871 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T14:08:05.871 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T14:08:05.871 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T14:08:05.871 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 14:07:47' 2024-09-12T14:08:05.872 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T14:08:05.874 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T14:08:05.874 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:08:05.874 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 4 2024-09-12T14:08:06.631 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary3 up+unknown '' '' 2024-09-12T14:08:06.631 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:08:06.631 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:08:06.631 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T14:08:06.631 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:08:06.631 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:08:06.631 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:08:06.632 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:08:06.632 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:08:06.632 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary3 2024-09-12T14:08:06.709 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary3: 2024-09-12T14:08:06.709 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: d356f103-adde-4c7a-855e-96419b23abef 2024-09-12T14:08:06.709 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T14:08:06.709 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T14:08:06.709 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T14:08:06.709 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 14:07:47' 2024-09-12T14:08:06.709 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T14:08:06.711 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T14:08:06.712 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:08:06.712 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 4 2024-09-12T14:08:07.005 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary2 up+unknown '' '' 2024-09-12T14:08:07.005 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:08:07.005 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:08:07.006 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T14:08:07.006 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:08:07.006 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:08:07.006 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:08:07.006 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:08:07.006 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:08:07.006 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary2 2024-09-12T14:08:07.023 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary7 up+unknown '' '' 2024-09-12T14:08:07.024 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:08:07.024 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:08:07.024 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T14:08:07.024 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:08:07.024 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:08:07.024 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:08:07.024 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:08:07.024 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:08:07.025 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary7 2024-09-12T14:08:07.028 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary4 up+unknown '' '' 2024-09-12T14:08:07.028 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:08:07.028 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:08:07.028 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T14:08:07.028 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:08:07.028 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:08:07.028 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:08:07.028 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:08:07.029 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:08:07.029 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary4 2024-09-12T14:08:07.084 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary2: 2024-09-12T14:08:07.085 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: ccaafd72-518c-4588-86d7-96c322bcf1bd 2024-09-12T14:08:07.085 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T14:08:07.085 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T14:08:07.085 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T14:08:07.085 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 14:07:47' 2024-09-12T14:08:07.085 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T14:08:07.088 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T14:08:07.088 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:08:07.088 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 8 2024-09-12T14:08:07.104 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary7: 2024-09-12T14:08:07.104 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 5795f6f3-43c3-4e14-89d9-f4798e240cc6 2024-09-12T14:08:07.105 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T14:08:07.105 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T14:08:07.105 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T14:08:07.105 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 14:07:47' 2024-09-12T14:08:07.105 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T14:08:07.107 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary4: 2024-09-12T14:08:07.107 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: a6cc92cf-b177-4aea-95c5-b61892fd9062 2024-09-12T14:08:07.107 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T14:08:07.108 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T14:08:07.108 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T14:08:07.108 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 14:07:47' 2024-09-12T14:08:07.108 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T14:08:07.108 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T14:08:07.108 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:08:07.108 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 8 2024-09-12T14:08:07.110 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T14:08:07.110 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:08:07.111 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 8 2024-09-12T14:08:07.619 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary6 up+unknown '' '' 2024-09-12T14:08:07.619 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:08:07.619 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:08:07.620 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T14:08:07.620 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:08:07.620 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:08:07.620 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:08:07.620 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:08:07.620 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:08:07.620 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary6 2024-09-12T14:08:07.704 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary6: 2024-09-12T14:08:07.704 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: f8d03c2d-535f-4038-a91f-c159711e8e6c 2024-09-12T14:08:07.704 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T14:08:07.704 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T14:08:07.704 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T14:08:07.704 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 14:07:47' 2024-09-12T14:08:07.704 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T14:08:07.706 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T14:08:07.707 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:08:07.707 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 4 2024-09-12T14:08:07.859 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary8 up+unknown '' '' 2024-09-12T14:08:07.859 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:08:07.859 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:08:07.859 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T14:08:07.860 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:08:07.860 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:08:07.860 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:08:07.860 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:08:07.860 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:08:07.860 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary8 2024-09-12T14:08:07.870 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary10 up+unknown '' '' 2024-09-12T14:08:07.871 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:08:07.871 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:08:07.871 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T14:08:07.871 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:08:07.871 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:08:07.871 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:08:07.871 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:08:07.872 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:08:07.872 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary10 2024-09-12T14:08:07.940 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary8: 2024-09-12T14:08:07.941 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: fa74ee32-e225-4343-82da-f93063849113 2024-09-12T14:08:07.941 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T14:08:07.941 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T14:08:07.941 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T14:08:07.941 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 14:07:47' 2024-09-12T14:08:07.941 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T14:08:07.943 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T14:08:07.943 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:08:07.943 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 8 2024-09-12T14:08:07.950 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary10: 2024-09-12T14:08:07.950 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: b0e88823-56d3-40a7-88c0-8d4c22f008ee 2024-09-12T14:08:07.950 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T14:08:07.950 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T14:08:07.950 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T14:08:07.950 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 14:07:47' 2024-09-12T14:08:07.950 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T14:08:07.953 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T14:08:07.953 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:08:07.953 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 8 2024-09-12T14:08:08.884 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary9 up+unknown '' '' 2024-09-12T14:08:08.884 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:08:08.884 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:08:08.884 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T14:08:08.884 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:08:08.885 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:08:08.885 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:08:08.885 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:08:08.885 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:08:08.885 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary9 2024-09-12T14:08:08.976 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary9: 2024-09-12T14:08:08.976 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 5991a19a-9e33-46ee-965b-85f990022e98 2024-09-12T14:08:08.976 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T14:08:08.976 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T14:08:08.976 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T14:08:08.976 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 14:07:47' 2024-09-12T14:08:08.976 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T14:08:08.979 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T14:08:08.979 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:08:08.979 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 8 2024-09-12T14:08:09.869 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary1 up+unknown '' '' 2024-09-12T14:08:09.869 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:08:09.869 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:08:09.870 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T14:08:09.870 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:08:09.870 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:08:09.870 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:08:09.870 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:08:09.870 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:08:09.870 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary1 2024-09-12T14:08:09.876 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary5 up+unknown '' '' 2024-09-12T14:08:09.876 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:08:09.877 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:08:09.877 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T14:08:09.877 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:08:09.877 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:08:09.877 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:08:09.877 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:08:09.877 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:08:09.877 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary5 2024-09-12T14:08:09.954 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary1: 2024-09-12T14:08:09.954 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 9bf3eb7e-eb1b-464c-b684-743ee2313f71 2024-09-12T14:08:09.955 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T14:08:09.955 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T14:08:09.955 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T14:08:09.955 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 14:07:47' 2024-09-12T14:08:09.955 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T14:08:09.957 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T14:08:09.957 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:08:09.957 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 8 2024-09-12T14:08:09.959 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary5: 2024-09-12T14:08:09.959 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 399c5b9a-89f0-4680-96c5-f2cac59115cf 2024-09-12T14:08:09.960 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T14:08:09.960 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T14:08:09.960 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T14:08:09.960 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 14:07:47' 2024-09-12T14:08:09.960 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T14:08:09.962 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T14:08:09.962 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:08:09.963 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 8 2024-09-12T14:08:10.714 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary3 up+unknown '' '' 2024-09-12T14:08:10.714 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:08:10.715 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:08:10.715 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T14:08:10.715 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:08:10.715 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:08:10.715 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:08:10.715 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:08:10.715 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:08:10.715 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary3 2024-09-12T14:08:10.798 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary3: 2024-09-12T14:08:10.799 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: d356f103-adde-4c7a-855e-96419b23abef 2024-09-12T14:08:10.799 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T14:08:10.799 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T14:08:10.799 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T14:08:10.799 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 14:07:47' 2024-09-12T14:08:10.799 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T14:08:10.801 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T14:08:10.802 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:08:10.802 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 8 2024-09-12T14:08:11.709 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary6 up+unknown '' '' 2024-09-12T14:08:11.709 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:08:11.710 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:08:11.710 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T14:08:11.710 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:08:11.710 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:08:11.710 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:08:11.710 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:08:11.710 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:08:11.710 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary6 2024-09-12T14:08:11.797 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary6: 2024-09-12T14:08:11.797 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: f8d03c2d-535f-4038-a91f-c159711e8e6c 2024-09-12T14:08:11.797 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T14:08:11.797 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T14:08:11.798 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T14:08:11.798 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 14:07:47' 2024-09-12T14:08:11.798 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T14:08:11.800 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T14:08:11.800 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:08:11.800 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 8 2024-09-12T14:08:15.090 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary2 up+unknown '' '' 2024-09-12T14:08:15.091 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:08:15.091 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:08:15.091 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T14:08:15.091 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:08:15.091 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:08:15.091 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:08:15.091 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:08:15.091 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:08:15.091 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary2 2024-09-12T14:08:15.110 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary7 up+unknown '' '' 2024-09-12T14:08:15.110 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:08:15.110 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:08:15.110 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T14:08:15.110 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:08:15.110 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:08:15.110 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:08:15.110 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:08:15.111 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:08:15.111 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary7 2024-09-12T14:08:15.112 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary4 up+unknown '' '' 2024-09-12T14:08:15.112 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:08:15.112 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:08:15.112 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T14:08:15.112 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:08:15.112 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:08:15.112 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:08:15.113 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:08:15.113 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:08:15.113 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary4 2024-09-12T14:08:15.173 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary2: 2024-09-12T14:08:15.173 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: ccaafd72-518c-4588-86d7-96c322bcf1bd 2024-09-12T14:08:15.173 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T14:08:15.173 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T14:08:15.173 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T14:08:15.174 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 14:07:47' 2024-09-12T14:08:15.174 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T14:08:15.176 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T14:08:15.176 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:08:15.176 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 8 2024-09-12T14:08:15.191 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary7: 2024-09-12T14:08:15.191 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 5795f6f3-43c3-4e14-89d9-f4798e240cc6 2024-09-12T14:08:15.191 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T14:08:15.191 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T14:08:15.191 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T14:08:15.191 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 14:07:47' 2024-09-12T14:08:15.191 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T14:08:15.191 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary4: 2024-09-12T14:08:15.191 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: a6cc92cf-b177-4aea-95c5-b61892fd9062 2024-09-12T14:08:15.192 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T14:08:15.192 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T14:08:15.192 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T14:08:15.192 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 14:07:47' 2024-09-12T14:08:15.192 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T14:08:15.194 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T14:08:15.194 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:08:15.194 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T14:08:15.194 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 8 2024-09-12T14:08:15.194 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:08:15.194 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 8 2024-09-12T14:08:15.946 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary8 up+unknown '' '' 2024-09-12T14:08:15.946 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:08:15.946 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:08:15.946 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T14:08:15.946 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:08:15.946 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:08:15.946 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:08:15.947 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:08:15.947 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:08:15.947 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary8 2024-09-12T14:08:15.955 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary10 up+unknown '' '' 2024-09-12T14:08:15.955 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:08:15.955 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:08:15.955 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T14:08:15.955 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:08:15.955 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:08:15.956 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:08:15.956 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:08:15.956 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:08:15.956 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary10 2024-09-12T14:08:16.029 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary8: 2024-09-12T14:08:16.030 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: fa74ee32-e225-4343-82da-f93063849113 2024-09-12T14:08:16.030 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T14:08:16.030 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T14:08:16.030 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T14:08:16.030 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 14:07:47' 2024-09-12T14:08:16.030 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T14:08:16.032 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T14:08:16.032 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:08:16.033 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 8 2024-09-12T14:08:16.034 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary10: 2024-09-12T14:08:16.034 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: b0e88823-56d3-40a7-88c0-8d4c22f008ee 2024-09-12T14:08:16.035 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T14:08:16.035 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T14:08:16.035 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T14:08:16.035 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 14:07:47' 2024-09-12T14:08:16.035 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T14:08:16.037 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T14:08:16.037 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:08:16.038 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 8 2024-09-12T14:08:16.982 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary9 up+unknown '' '' 2024-09-12T14:08:16.982 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:08:16.982 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:08:16.982 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T14:08:16.982 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:08:16.982 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:08:16.982 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:08:16.982 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:08:16.983 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:08:16.983 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary9 2024-09-12T14:08:17.065 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary9: 2024-09-12T14:08:17.065 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 5991a19a-9e33-46ee-965b-85f990022e98 2024-09-12T14:08:17.065 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T14:08:17.065 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T14:08:17.065 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T14:08:17.065 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 14:07:47' 2024-09-12T14:08:17.065 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T14:08:17.068 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T14:08:17.068 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:08:17.068 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 8 2024-09-12T14:08:17.960 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary1 up+unknown '' '' 2024-09-12T14:08:17.960 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:08:17.960 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:08:17.960 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T14:08:17.960 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:08:17.961 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:08:17.961 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:08:17.961 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:08:17.961 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:08:17.961 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary1 2024-09-12T14:08:17.965 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary5 up+unknown '' '' 2024-09-12T14:08:17.965 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:08:17.965 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:08:17.965 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T14:08:17.965 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:08:17.965 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:08:17.966 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:08:17.966 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:08:17.966 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:08:17.966 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary5 2024-09-12T14:08:18.042 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary1: 2024-09-12T14:08:18.042 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 9bf3eb7e-eb1b-464c-b684-743ee2313f71 2024-09-12T14:08:18.042 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+unknown 2024-09-12T14:08:18.042 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: remote image is not primary 2024-09-12T14:08:18.043 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T14:08:18.043 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 14:08:17' 2024-09-12T14:08:18.043 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T14:08:18.045 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary5: 2024-09-12T14:08:18.045 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 399c5b9a-89f0-4680-96c5-f2cac59115cf 2024-09-12T14:08:18.045 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+unknown 2024-09-12T14:08:18.045 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: remote image is not primary 2024-09-12T14:08:18.045 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T14:08:18.045 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 14:08:17' 2024-09-12T14:08:18.045 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T14:08:18.045 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+unknown 2024-09-12T14:08:18.046 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T14:08:18.047 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+unknown 2024-09-12T14:08:18.047 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T14:08:18.048 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: remote image is not primary 2024-09-12T14:08:18.048 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T14:08:18.048 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T14:08:18.049 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T14:08:18.050 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: remote image is not primary 2024-09-12T14:08:18.051 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T14:08:18.051 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T14:08:18.051 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T14:08:18.051 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T14:08:18.052 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T14:08:18.054 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T14:08:18.054 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T14:08:18.054 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T14:08:18.054 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T14:08:18.055 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T14:08:18.056 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T14:08:18.057 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T14:08:18.058 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T14:08:18.059 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 14:08:17' 2024-09-12T14:08:18.059 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary1 up+unknown '2024-09-12 14:08:17' 2024-09-12T14:08:18.059 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:08:18.059 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:08:18.059 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T14:08:18.059 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:08:18.059 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 14:08:17' 2024-09-12T14:08:18.059 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:08:18.059 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:08:18.059 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-09-12T14:08:18.062 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 14:08:17' 2024-09-12T14:08:18.062 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary5 up+unknown '2024-09-12 14:08:17' 2024-09-12T14:08:18.062 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:08:18.062 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:08:18.062 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T14:08:18.062 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:08:18.062 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 14:08:17' 2024-09-12T14:08:18.062 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:08:18.063 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:08:18.063 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-09-12T14:08:18.193 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041934232mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary19bf3eb7e-eb1b-464c-b684-743ee2313f71up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:08:17image-primary10b0e88823-56d3-40a7-88c0-8d4c22f008eeup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:08:17image-primary2ccaafd72-518c-4588-86d7-96c322bcf1bdup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:08:17image-primary3d356f103-adde-4c7a-855e-96419b23abefup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:08:17image-primary4a6cc92cf-b177-4aea-95c5-b61892fd9062up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:08:17image-primary5399c5b9a-89f0-4680-96c5-f2cac59115cfup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:08:17image-primary6f8d03c2d-535f-4038-a91f-c159711e8e6cup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:08:17image-primary75795f6f3-43c3-4e14-89d9-f4798e240cc6up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:08:17image-primary8fa74ee32-e225-4343-82da-f93063849113up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:08:17image-primary95991a19a-9e33-46ee-965b-85f990022e98up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:08:17' 2024-09-12T14:08:18.194 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T14:08:18.194 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary1'\'']/last_update' 2024-09-12T14:08:18.195 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041934232mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary19bf3eb7e-eb1b-464c-b684-743ee2313f71up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:08:17image-primary10b0e88823-56d3-40a7-88c0-8d4c22f008eeup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:08:17image-primary2ccaafd72-518c-4588-86d7-96c322bcf1bdup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:08:17image-primary3d356f103-adde-4c7a-855e-96419b23abefup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:08:17image-primary4a6cc92cf-b177-4aea-95c5-b61892fd9062up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:08:17image-primary5399c5b9a-89f0-4680-96c5-f2cac59115cfup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:08:17image-primary6f8d03c2d-535f-4038-a91f-c159711e8e6cup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:08:17image-primary75795f6f3-43c3-4e14-89d9-f4798e240cc6up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:08:17image-primary8fa74ee32-e225-4343-82da-f93063849113up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:08:17image-primary95991a19a-9e33-46ee-965b-85f990022e98up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:08:17' 2024-09-12T14:08:18.195 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T14:08:18.196 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary5'\'']/last_update' 2024-09-12T14:08:18.297 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 14:08:17' 2024-09-12T14:08:18.297 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 14:08:17' 2024-09-12T14:08:18.298 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary5'\'']/state' 2024-09-12T14:08:18.298 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary1'\'']/state' 2024-09-12T14:08:18.301 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+unknown 2024-09-12T14:08:18.301 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+unknown 2024-09-12T14:08:18.302 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T14:08:18.302 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T14:08:18.302 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+unknown 2024-09-12T14:08:18.302 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+unknown 2024-09-12T14:08:18.304 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T14:08:18.304 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T14:08:18.305 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:08:18.305 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:08:18.305 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster2 mirror image-primary5 up+unknown 2024-09-12T14:08:18.306 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T14:08:18.306 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:08:18.306 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T14:08:18.306 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:08:18.306 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:08:18.306 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:08:18.306 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:08:18.306 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T14:08:18.306 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:08:18.306 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:08:18.306 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster2 mirror image-primary1 up+unknown 2024-09-12T14:08:18.306 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T14:08:18.306 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:08:18.306 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T14:08:18.307 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:08:18.307 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:08:18.307 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:08:18.307 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:08:18.307 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T14:08:18.804 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary3 up+unknown '' '' 2024-09-12T14:08:18.804 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:08:18.804 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:08:18.804 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T14:08:18.804 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:08:18.805 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:08:18.805 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:08:18.805 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:08:18.805 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:08:18.805 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary3 2024-09-12T14:08:18.879 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary3: 2024-09-12T14:08:18.879 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: d356f103-adde-4c7a-855e-96419b23abef 2024-09-12T14:08:18.879 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+unknown 2024-09-12T14:08:18.879 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: remote image is not primary 2024-09-12T14:08:18.879 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T14:08:18.879 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 14:08:17' 2024-09-12T14:08:18.879 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T14:08:18.881 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+unknown 2024-09-12T14:08:18.881 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T14:08:18.884 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: remote image is not primary 2024-09-12T14:08:18.884 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T14:08:18.885 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T14:08:18.885 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T14:08:18.887 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T14:08:18.887 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T14:08:18.890 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T14:08:18.890 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T14:08:18.891 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T14:08:18.895 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 14:08:17' 2024-09-12T14:08:18.895 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary3 up+unknown '2024-09-12 14:08:17' 2024-09-12T14:08:18.895 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:08:18.895 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:08:18.895 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T14:08:18.895 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:08:18.895 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 14:08:17' 2024-09-12T14:08:18.895 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:08:18.896 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:08:18.896 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-09-12T14:08:19.003 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041934232mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary19bf3eb7e-eb1b-464c-b684-743ee2313f71up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:08:17image-primary10b0e88823-56d3-40a7-88c0-8d4c22f008eeup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:08:17image-primary2ccaafd72-518c-4588-86d7-96c322bcf1bdup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:08:17image-primary3d356f103-adde-4c7a-855e-96419b23abefup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:08:17image-primary4a6cc92cf-b177-4aea-95c5-b61892fd9062up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:08:17image-primary5399c5b9a-89f0-4680-96c5-f2cac59115cfup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:08:17image-primary6f8d03c2d-535f-4038-a91f-c159711e8e6cup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:08:17image-primary75795f6f3-43c3-4e14-89d9-f4798e240cc6up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:08:17image-primary8fa74ee32-e225-4343-82da-f93063849113up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:08:17image-primary95991a19a-9e33-46ee-965b-85f990022e98up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:08:17' 2024-09-12T14:08:19.003 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T14:08:19.004 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary3'\'']/last_update' 2024-09-12T14:08:19.007 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 14:08:17' 2024-09-12T14:08:19.008 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary3'\'']/state' 2024-09-12T14:08:19.012 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+unknown 2024-09-12T14:08:19.012 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T14:08:19.013 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+unknown 2024-09-12T14:08:19.015 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T14:08:19.016 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:08:19.016 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:08:19.016 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster2 mirror image-primary3 up+unknown 2024-09-12T14:08:19.016 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T14:08:19.016 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:08:19.016 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T14:08:19.016 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:08:19.016 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:08:19.016 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:08:19.016 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:08:19.016 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T14:08:19.308 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster2 mirror image-primary5 up+unknown '' '' 2024-09-12T14:08:19.308 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T14:08:19.308 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:08:19.308 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T14:08:19.308 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:08:19.308 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:08:19.309 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:08:19.309 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:08:19.309 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster2 mirror image-primary1 up+unknown '' '' 2024-09-12T14:08:19.309 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T14:08:19.309 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:08:19.309 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T14:08:19.309 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:08:19.309 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:08:19.309 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:08:19.309 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:08:19.309 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:08:19.309 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster2 mirror image status mirror/image-primary5 2024-09-12T14:08:19.310 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:08:19.310 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster2 mirror image status mirror/image-primary1 2024-09-12T14:08:19.377 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary5: 2024-09-12T14:08:19.377 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 399c5b9a-89f0-4680-96c5-f2cac59115cf 2024-09-12T14:08:19.378 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+unknown 2024-09-12T14:08:19.378 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: remote image is not primary 2024-09-12T14:08:19.378 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T14:08:19.378 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 14:08:17' 2024-09-12T14:08:19.378 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T14:08:19.379 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary1: 2024-09-12T14:08:19.379 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 9bf3eb7e-eb1b-464c-b684-743ee2313f71 2024-09-12T14:08:19.379 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+unknown 2024-09-12T14:08:19.379 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: remote image is not primary 2024-09-12T14:08:19.379 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T14:08:19.379 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 14:08:17' 2024-09-12T14:08:19.379 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T14:08:19.380 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+unknown 2024-09-12T14:08:19.380 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+unknown 2024-09-12T14:08:19.381 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T14:08:19.381 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T14:08:19.383 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: remote image is not primary 2024-09-12T14:08:19.383 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: remote image is not primary 2024-09-12T14:08:19.384 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T14:08:19.384 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T14:08:19.384 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T14:08:19.384 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T14:08:19.385 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T14:08:19.385 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T14:08:19.386 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T14:08:19.386 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T14:08:19.387 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T14:08:19.387 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T14:08:19.389 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T14:08:19.389 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T14:08:19.389 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T14:08:19.389 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T14:08:19.390 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T14:08:19.390 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T14:08:19.394 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 14:08:17' 2024-09-12T14:08:19.394 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster2 mirror image-primary5 up+unknown '2024-09-12 14:08:17' 2024-09-12T14:08:19.394 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T14:08:19.394 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:08:19.394 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T14:08:19.394 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:08:19.394 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 14:08:17' 2024-09-12T14:08:19.394 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:08:19.394 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 14:08:17' 2024-09-12T14:08:19.394 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster2 mirror image-primary1 up+unknown '2024-09-12 14:08:17' 2024-09-12T14:08:19.394 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T14:08:19.395 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:08:19.395 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T14:08:19.395 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:08:19.395 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 14:08:17' 2024-09-12T14:08:19.395 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:08:19.395 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:08:19.395 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2024-09-12T14:08:19.395 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:08:19.395 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2024-09-12T14:08:19.526 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041924231mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary19bf3eb7e-eb1b-464c-b684-743ee2313f71up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:08:17image-primary10b0e88823-56d3-40a7-88c0-8d4c22f008eeup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:08:17image-primary2ccaafd72-518c-4588-86d7-96c322bcf1bdup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:08:17image-primary3d356f103-adde-4c7a-855e-96419b23abefup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:08:17image-primary4a6cc92cf-b177-4aea-95c5-b61892fd9062up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:08:17image-primary5399c5b9a-89f0-4680-96c5-f2cac59115cfup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:08:17image-primary6f8d03c2d-535f-4038-a91f-c159711e8e6cup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:08:17image-primary75795f6f3-43c3-4e14-89d9-f4798e240cc6up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:08:17image-primary8fa74ee32-e225-4343-82da-f93063849113up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:08:17image-primary95991a19a-9e33-46ee-965b-85f990022e98up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:08:17' 2024-09-12T14:08:19.526 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T14:08:19.526 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary5'\'']/last_update' 2024-09-12T14:08:19.527 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041924231mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary19bf3eb7e-eb1b-464c-b684-743ee2313f71up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:08:17image-primary10b0e88823-56d3-40a7-88c0-8d4c22f008eeup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:08:17image-primary2ccaafd72-518c-4588-86d7-96c322bcf1bdup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:08:17image-primary3d356f103-adde-4c7a-855e-96419b23abefup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:08:17image-primary4a6cc92cf-b177-4aea-95c5-b61892fd9062up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:08:17image-primary5399c5b9a-89f0-4680-96c5-f2cac59115cfup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:08:17image-primary6f8d03c2d-535f-4038-a91f-c159711e8e6cup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:08:17image-primary75795f6f3-43c3-4e14-89d9-f4798e240cc6up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:08:17image-primary8fa74ee32-e225-4343-82da-f93063849113up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:08:17image-primary95991a19a-9e33-46ee-965b-85f990022e98up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:08:17' 2024-09-12T14:08:19.527 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T14:08:19.528 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary1'\'']/last_update' 2024-09-12T14:08:19.530 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 14:08:17' 2024-09-12T14:08:19.531 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary5'\'']/state' 2024-09-12T14:08:19.531 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 14:08:17' 2024-09-12T14:08:19.531 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary1'\'']/state' 2024-09-12T14:08:19.534 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+unknown 2024-09-12T14:08:19.535 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T14:08:19.535 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+unknown 2024-09-12T14:08:19.536 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+unknown 2024-09-12T14:08:19.537 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T14:08:19.537 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+unknown 2024-09-12T14:08:19.538 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T14:08:19.539 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:08:19.539 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:08:19.539 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ promote_image cluster2 mirror image-primary5 2024-09-12T14:08:19.540 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T14:08:19.540 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:08:19.540 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T14:08:19.540 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local force= 2024-09-12T14:08:19.540 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 mirror image promote mirror/image-primary5 2024-09-12T14:08:19.541 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T14:08:19.542 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:08:19.542 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:08:19.543 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ promote_image cluster2 mirror image-primary1 2024-09-12T14:08:19.543 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T14:08:19.543 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:08:19.543 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T14:08:19.543 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local force= 2024-09-12T14:08:19.543 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 mirror image promote mirror/image-primary1 2024-09-12T14:08:19.800 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Image promoted to primary 2024-09-12T14:08:19.803 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary6 up+unknown '' '' 2024-09-12T14:08:19.803 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:08:19.803 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:08:19.803 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T14:08:19.804 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:08:19.804 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:08:19.804 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:08:19.804 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:08:19.805 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:08:19.805 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary6 2024-09-12T14:08:19.806 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Image promoted to primary 2024-09-12T14:08:19.814 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \n\b\d ]] 2024-09-12T14:08:19.815 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \k\r\b\d ]] 2024-09-12T14:08:19.815 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo rbd --cluster cluster2 device map -t krbd mirror/image-primary5 2024-09-12T14:08:19.822 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \n\b\d ]] 2024-09-12T14:08:19.822 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \k\r\b\d ]] 2024-09-12T14:08:19.822 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo rbd --cluster cluster2 device map -t krbd mirror/image-primary1 2024-09-12T14:08:19.887 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary6: 2024-09-12T14:08:19.887 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: f8d03c2d-535f-4038-a91f-c159711e8e6c 2024-09-12T14:08:19.887 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+unknown 2024-09-12T14:08:19.887 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: remote image is not primary 2024-09-12T14:08:19.887 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T14:08:19.887 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 14:08:17' 2024-09-12T14:08:19.887 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T14:08:19.890 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+unknown 2024-09-12T14:08:19.890 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T14:08:19.893 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: remote image is not primary 2024-09-12T14:08:19.894 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T14:08:19.894 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T14:08:19.897 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T14:08:19.897 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T14:08:19.900 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T14:08:19.900 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T14:08:19.900 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T14:08:19.901 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T14:08:19.905 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 14:08:17' 2024-09-12T14:08:19.905 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary6 up+unknown '2024-09-12 14:08:17' 2024-09-12T14:08:19.905 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:08:19.905 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:08:19.905 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T14:08:19.905 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:08:19.905 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 14:08:17' 2024-09-12T14:08:19.905 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:08:19.906 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:08:19.906 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-09-12T14:08:19.961 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ dev=/dev/rbd1 2024-09-12T14:08:19.962 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ dev=/dev/rbd0 2024-09-12T14:08:19.963 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo md5sum /dev/rbd1 2024-09-12T14:08:19.963 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ awk '{print $1}' 2024-09-12T14:08:19.964 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo md5sum /dev/rbd0 2024-09-12T14:08:19.965 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ awk '{print $1}' 2024-09-12T14:08:20.018 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster2 mirror image-primary3 up+unknown '' '' 2024-09-12T14:08:20.018 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T14:08:20.018 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:08:20.019 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T14:08:20.019 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:08:20.019 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:08:20.019 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:08:20.019 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:08:20.019 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:08:20.019 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster2 mirror image status mirror/image-primary3 2024-09-12T14:08:20.041 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041934232mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary19bf3eb7e-eb1b-464c-b684-743ee2313f71up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:08:17image-primary10b0e88823-56d3-40a7-88c0-8d4c22f008eeup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:08:17image-primary2ccaafd72-518c-4588-86d7-96c322bcf1bdup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:08:17image-primary3d356f103-adde-4c7a-855e-96419b23abefup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:08:17image-primary4a6cc92cf-b177-4aea-95c5-b61892fd9062up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:08:17image-primary5399c5b9a-89f0-4680-96c5-f2cac59115cfup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:08:17image-primary6f8d03c2d-535f-4038-a91f-c159711e8e6cup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:08:17image-primary75795f6f3-43c3-4e14-89d9-f4798e240cc6up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:08:17image-primary8fa74ee32-e225-4343-82da-f93063849113up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:08:17image-primary95991a19a-9e33-46ee-965b-85f990022e98up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:08:17' 2024-09-12T14:08:20.042 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T14:08:20.042 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary6'\'']/last_update' 2024-09-12T14:08:20.046 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 14:08:17' 2024-09-12T14:08:20.047 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary6'\'']/state' 2024-09-12T14:08:20.051 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+unknown 2024-09-12T14:08:20.051 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T14:08:20.052 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+unknown 2024-09-12T14:08:20.057 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T14:08:20.058 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:08:20.058 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:08:20.058 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster2 mirror image-primary6 up+unknown 2024-09-12T14:08:20.058 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T14:08:20.058 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:08:20.058 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T14:08:20.058 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:08:20.058 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:08:20.058 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:08:20.058 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:08:20.059 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T14:08:20.096 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary3: 2024-09-12T14:08:20.097 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: d356f103-adde-4c7a-855e-96419b23abef 2024-09-12T14:08:20.097 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+unknown 2024-09-12T14:08:20.097 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: remote image is not primary 2024-09-12T14:08:20.097 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T14:08:20.097 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 14:08:17' 2024-09-12T14:08:20.097 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T14:08:20.099 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+unknown 2024-09-12T14:08:20.099 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T14:08:20.102 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: remote image is not primary 2024-09-12T14:08:20.102 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T14:08:20.103 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T14:08:20.103 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T14:08:20.105 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T14:08:20.105 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T14:08:20.108 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T14:08:20.108 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T14:08:20.109 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T14:08:20.113 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 14:08:17' 2024-09-12T14:08:20.113 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster2 mirror image-primary3 up+unknown '2024-09-12 14:08:17' 2024-09-12T14:08:20.113 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T14:08:20.113 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:08:20.113 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T14:08:20.113 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:08:20.113 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 14:08:17' 2024-09-12T14:08:20.113 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:08:20.114 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:08:20.114 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2024-09-12T14:08:20.237 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041924231mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary19bf3eb7e-eb1b-464c-b684-743ee2313f71up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:08:17image-primary10b0e88823-56d3-40a7-88c0-8d4c22f008eeup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:08:17image-primary2ccaafd72-518c-4588-86d7-96c322bcf1bdup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:08:17image-primary3d356f103-adde-4c7a-855e-96419b23abefup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:08:17image-primary4a6cc92cf-b177-4aea-95c5-b61892fd9062up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:08:17image-primary5399c5b9a-89f0-4680-96c5-f2cac59115cfup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:08:17image-primary6f8d03c2d-535f-4038-a91f-c159711e8e6cup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:08:17image-primary75795f6f3-43c3-4e14-89d9-f4798e240cc6up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:08:17image-primary8fa74ee32-e225-4343-82da-f93063849113up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:08:17image-primary95991a19a-9e33-46ee-965b-85f990022e98up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:08:17' 2024-09-12T14:08:20.237 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T14:08:20.238 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary3'\'']/last_update' 2024-09-12T14:08:20.242 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 14:08:17' 2024-09-12T14:08:20.242 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary3'\'']/state' 2024-09-12T14:08:20.246 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+unknown 2024-09-12T14:08:20.247 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T14:08:20.248 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+unknown 2024-09-12T14:08:20.250 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T14:08:20.251 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:08:20.251 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:08:20.251 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ promote_image cluster2 mirror image-primary3 2024-09-12T14:08:20.251 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T14:08:20.251 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:08:20.251 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T14:08:20.251 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local force= 2024-09-12T14:08:20.251 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 mirror image promote mirror/image-primary3 2024-09-12T14:08:20.812 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Image promoted to primary 2024-09-12T14:08:20.825 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \n\b\d ]] 2024-09-12T14:08:20.825 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \k\r\b\d ]] 2024-09-12T14:08:20.825 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo rbd --cluster cluster2 device map -t krbd mirror/image-primary3 2024-09-12T14:08:20.958 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ dev=/dev/rbd2 2024-09-12T14:08:20.959 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo md5sum /dev/rbd2 2024-09-12T14:08:20.959 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ awk '{print $1}' 2024-09-12T14:08:21.057 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster2 mirror image-primary6 up+unknown '' '' 2024-09-12T14:08:21.057 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T14:08:21.058 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:08:21.058 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T14:08:21.058 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:08:21.058 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:08:21.058 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:08:21.058 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:08:21.058 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:08:21.058 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster2 mirror image status mirror/image-primary6 2024-09-12T14:08:21.135 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary6: 2024-09-12T14:08:21.135 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: f8d03c2d-535f-4038-a91f-c159711e8e6c 2024-09-12T14:08:21.135 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+unknown 2024-09-12T14:08:21.135 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: remote image is not primary 2024-09-12T14:08:21.135 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T14:08:21.135 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 14:08:17' 2024-09-12T14:08:21.135 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T14:08:21.137 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+unknown 2024-09-12T14:08:21.138 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T14:08:21.141 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: remote image is not primary 2024-09-12T14:08:21.141 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T14:08:21.142 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T14:08:21.142 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T14:08:21.145 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T14:08:21.145 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T14:08:21.148 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T14:08:21.148 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T14:08:21.149 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T14:08:21.153 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 14:08:17' 2024-09-12T14:08:21.153 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster2 mirror image-primary6 up+unknown '2024-09-12 14:08:17' 2024-09-12T14:08:21.153 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T14:08:21.153 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:08:21.153 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T14:08:21.153 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:08:21.153 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 14:08:17' 2024-09-12T14:08:21.153 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:08:21.154 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:08:21.154 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2024-09-12T14:08:21.288 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041924231mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary19bf3eb7e-eb1b-464c-b684-743ee2313f71up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:08:17image-primary10b0e88823-56d3-40a7-88c0-8d4c22f008eeup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:08:17image-primary2ccaafd72-518c-4588-86d7-96c322bcf1bdup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:08:17image-primary3d356f103-adde-4c7a-855e-96419b23abefup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:08:17image-primary4a6cc92cf-b177-4aea-95c5-b61892fd9062up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:08:17image-primary5399c5b9a-89f0-4680-96c5-f2cac59115cfup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:08:17image-primary6f8d03c2d-535f-4038-a91f-c159711e8e6cup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:08:17image-primary75795f6f3-43c3-4e14-89d9-f4798e240cc6up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:08:17image-primary8fa74ee32-e225-4343-82da-f93063849113up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:08:17image-primary95991a19a-9e33-46ee-965b-85f990022e98up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:08:17' 2024-09-12T14:08:21.288 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T14:08:21.288 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary6'\'']/last_update' 2024-09-12T14:08:21.293 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 14:08:17' 2024-09-12T14:08:21.294 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary6'\'']/state' 2024-09-12T14:08:21.298 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+unknown 2024-09-12T14:08:21.299 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T14:08:21.299 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+unknown 2024-09-12T14:08:21.301 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T14:08:21.302 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:08:21.302 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:08:21.302 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ promote_image cluster2 mirror image-primary6 2024-09-12T14:08:21.303 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T14:08:21.303 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:08:21.303 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T14:08:21.303 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local force= 2024-09-12T14:08:21.303 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 mirror image promote mirror/image-primary6 2024-09-12T14:08:21.822 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Image promoted to primary 2024-09-12T14:08:21.836 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \n\b\d ]] 2024-09-12T14:08:21.837 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \k\r\b\d ]] 2024-09-12T14:08:21.837 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo rbd --cluster cluster2 device map -t krbd mirror/image-primary6 2024-09-12T14:08:21.987 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ dev=/dev/rbd3 2024-09-12T14:08:21.989 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo md5sum /dev/rbd3 2024-09-12T14:08:21.989 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ awk '{print $1}' 2024-09-12T14:08:23.179 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary2 up+unknown '' '' 2024-09-12T14:08:23.179 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:08:23.179 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:08:23.179 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T14:08:23.179 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:08:23.179 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:08:23.179 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:08:23.179 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:08:23.180 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:08:23.180 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary2 2024-09-12T14:08:23.196 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary7 up+unknown '' '' 2024-09-12T14:08:23.197 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary4 up+unknown '' '' 2024-09-12T14:08:23.197 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:08:23.197 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:08:23.197 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:08:23.197 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T14:08:23.197 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:08:23.197 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:08:23.197 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T14:08:23.197 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:08:23.197 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:08:23.197 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:08:23.197 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:08:23.197 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:08:23.198 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:08:23.198 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:08:23.198 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:08:23.198 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary4 2024-09-12T14:08:23.198 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:08:23.198 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary7 2024-09-12T14:08:23.290 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary2: 2024-09-12T14:08:23.290 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: ccaafd72-518c-4588-86d7-96c322bcf1bd 2024-09-12T14:08:23.290 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+unknown 2024-09-12T14:08:23.290 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: remote image is not primary 2024-09-12T14:08:23.290 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T14:08:23.290 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 14:08:17' 2024-09-12T14:08:23.290 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T14:08:23.294 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+unknown 2024-09-12T14:08:23.297 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T14:08:23.298 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: remote image is not primary 2024-09-12T14:08:23.298 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T14:08:23.299 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T14:08:23.300 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T14:08:23.302 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T14:08:23.303 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T14:08:23.304 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary7: 2024-09-12T14:08:23.304 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 5795f6f3-43c3-4e14-89d9-f4798e240cc6 2024-09-12T14:08:23.304 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+unknown 2024-09-12T14:08:23.304 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: remote image is not primary 2024-09-12T14:08:23.304 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T14:08:23.304 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 14:08:17' 2024-09-12T14:08:23.305 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T14:08:23.308 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+unknown 2024-09-12T14:08:23.308 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T14:08:23.309 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T14:08:23.309 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T14:08:23.310 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T14:08:23.310 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary4: 2024-09-12T14:08:23.311 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: a6cc92cf-b177-4aea-95c5-b61892fd9062 2024-09-12T14:08:23.311 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+unknown 2024-09-12T14:08:23.311 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: remote image is not primary 2024-09-12T14:08:23.311 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T14:08:23.311 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 14:08:17' 2024-09-12T14:08:23.311 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T14:08:23.313 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: remote image is not primary 2024-09-12T14:08:23.313 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+unknown 2024-09-12T14:08:23.315 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T14:08:23.315 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T14:08:23.315 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T14:08:23.316 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 14:08:17' 2024-09-12T14:08:23.316 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary2 up+unknown '2024-09-12 14:08:17' 2024-09-12T14:08:23.316 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T14:08:23.316 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:08:23.317 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:08:23.317 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T14:08:23.317 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:08:23.317 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 14:08:17' 2024-09-12T14:08:23.317 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:08:23.317 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:08:23.317 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-09-12T14:08:23.319 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: remote image is not primary 2024-09-12T14:08:23.319 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T14:08:23.320 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T14:08:23.320 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T14:08:23.321 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T14:08:23.321 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T14:08:23.324 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T14:08:23.324 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T14:08:23.325 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T14:08:23.325 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T14:08:23.326 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T14:08:23.329 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T14:08:23.330 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T14:08:23.333 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T14:08:23.333 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 14:08:17' 2024-09-12T14:08:23.333 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary7 up+unknown '2024-09-12 14:08:17' 2024-09-12T14:08:23.333 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:08:23.333 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:08:23.333 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T14:08:23.333 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:08:23.333 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 14:08:17' 2024-09-12T14:08:23.333 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:08:23.334 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:08:23.334 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-09-12T14:08:23.338 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 14:08:17' 2024-09-12T14:08:23.338 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary4 up+unknown '2024-09-12 14:08:17' 2024-09-12T14:08:23.338 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:08:23.338 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:08:23.338 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T14:08:23.338 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:08:23.339 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 14:08:17' 2024-09-12T14:08:23.339 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:08:23.339 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:08:23.339 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-09-12T14:08:23.506 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041934232mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary19bf3eb7e-eb1b-464c-b684-743ee2313f71up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:08:17image-primary10b0e88823-56d3-40a7-88c0-8d4c22f008eeup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:08:17image-primary2ccaafd72-518c-4588-86d7-96c322bcf1bdup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:08:17image-primary3d356f103-adde-4c7a-855e-96419b23abefup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:08:17image-primary4a6cc92cf-b177-4aea-95c5-b61892fd9062up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:08:17image-primary5399c5b9a-89f0-4680-96c5-f2cac59115cfup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:08:17image-primary6f8d03c2d-535f-4038-a91f-c159711e8e6cup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:08:17image-primary75795f6f3-43c3-4e14-89d9-f4798e240cc6up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:08:17image-primary8fa74ee32-e225-4343-82da-f93063849113up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:08:17image-primary95991a19a-9e33-46ee-965b-85f990022e98up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:08:17' 2024-09-12T14:08:23.506 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T14:08:23.507 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary7'\'']/last_update' 2024-09-12T14:08:23.512 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 14:08:17' 2024-09-12T14:08:23.513 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary7'\'']/state' 2024-09-12T14:08:23.518 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041934232mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary19bf3eb7e-eb1b-464c-b684-743ee2313f71up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:08:17image-primary10b0e88823-56d3-40a7-88c0-8d4c22f008eeup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:08:17image-primary2ccaafd72-518c-4588-86d7-96c322bcf1bdup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:08:17image-primary3d356f103-adde-4c7a-855e-96419b23abefup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:08:17image-primary4a6cc92cf-b177-4aea-95c5-b61892fd9062up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:08:17image-primary5399c5b9a-89f0-4680-96c5-f2cac59115cfup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:08:17image-primary6f8d03c2d-535f-4038-a91f-c159711e8e6cup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:08:17image-primary75795f6f3-43c3-4e14-89d9-f4798e240cc6up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:08:17image-primary8fa74ee32-e225-4343-82da-f93063849113up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:08:17image-primary95991a19a-9e33-46ee-965b-85f990022e98up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:08:17' 2024-09-12T14:08:23.518 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T14:08:23.518 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+unknown 2024-09-12T14:08:23.519 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary2'\'']/last_update' 2024-09-12T14:08:23.519 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T14:08:23.520 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+unknown 2024-09-12T14:08:23.523 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T14:08:23.524 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:08:23.524 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:08:23.524 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster2 mirror image-primary7 up+unknown 2024-09-12T14:08:23.524 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T14:08:23.524 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:08:23.524 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T14:08:23.525 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:08:23.525 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:08:23.525 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:08:23.525 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:08:23.525 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T14:08:23.525 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 14:08:17' 2024-09-12T14:08:23.527 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary2'\'']/state' 2024-09-12T14:08:23.531 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041934232mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary19bf3eb7e-eb1b-464c-b684-743ee2313f71up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:08:17image-primary10b0e88823-56d3-40a7-88c0-8d4c22f008eeup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:08:17image-primary2ccaafd72-518c-4588-86d7-96c322bcf1bdup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:08:17image-primary3d356f103-adde-4c7a-855e-96419b23abefup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:08:17image-primary4a6cc92cf-b177-4aea-95c5-b61892fd9062up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:08:17image-primary5399c5b9a-89f0-4680-96c5-f2cac59115cfup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:08:17image-primary6f8d03c2d-535f-4038-a91f-c159711e8e6cup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:08:17image-primary75795f6f3-43c3-4e14-89d9-f4798e240cc6up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:08:17image-primary8fa74ee32-e225-4343-82da-f93063849113up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:08:17image-primary95991a19a-9e33-46ee-965b-85f990022e98up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:08:17' 2024-09-12T14:08:23.531 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T14:08:23.532 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary4'\'']/last_update' 2024-09-12T14:08:23.533 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+unknown 2024-09-12T14:08:23.534 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T14:08:23.535 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+unknown 2024-09-12T14:08:23.536 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 14:08:17' 2024-09-12T14:08:23.537 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary4'\'']/state' 2024-09-12T14:08:23.538 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T14:08:23.539 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:08:23.540 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:08:23.540 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster2 mirror image-primary2 up+unknown 2024-09-12T14:08:23.540 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T14:08:23.540 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:08:23.540 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T14:08:23.540 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:08:23.540 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:08:23.540 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:08:23.540 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:08:23.540 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T14:08:23.541 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+unknown 2024-09-12T14:08:23.542 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T14:08:23.543 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+unknown 2024-09-12T14:08:23.545 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T14:08:23.546 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:08:23.546 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:08:23.546 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster2 mirror image-primary4 up+unknown 2024-09-12T14:08:23.546 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T14:08:23.546 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:08:23.546 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T14:08:23.546 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:08:23.546 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:08:23.546 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:08:23.546 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:08:23.547 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T14:08:24.035 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary8 up+unknown '' '' 2024-09-12T14:08:24.035 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:08:24.035 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:08:24.036 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T14:08:24.036 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:08:24.036 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:08:24.036 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:08:24.036 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:08:24.036 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:08:24.036 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary8 2024-09-12T14:08:24.040 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary10 up+unknown '' '' 2024-09-12T14:08:24.040 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:08:24.040 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:08:24.040 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T14:08:24.040 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:08:24.040 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:08:24.040 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:08:24.040 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:08:24.041 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:08:24.041 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary10 2024-09-12T14:08:24.143 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary8: 2024-09-12T14:08:24.143 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: fa74ee32-e225-4343-82da-f93063849113 2024-09-12T14:08:24.143 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+unknown 2024-09-12T14:08:24.143 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: remote image is not primary 2024-09-12T14:08:24.143 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T14:08:24.143 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 14:08:17' 2024-09-12T14:08:24.143 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T14:08:24.147 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+unknown 2024-09-12T14:08:24.148 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T14:08:24.152 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary10: 2024-09-12T14:08:24.152 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: b0e88823-56d3-40a7-88c0-8d4c22f008ee 2024-09-12T14:08:24.152 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+unknown 2024-09-12T14:08:24.152 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: remote image is not primary 2024-09-12T14:08:24.152 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T14:08:24.153 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 14:08:17' 2024-09-12T14:08:24.153 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T14:08:24.153 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: remote image is not primary 2024-09-12T14:08:24.153 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T14:08:24.154 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T14:08:24.154 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T14:08:24.155 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+unknown 2024-09-12T14:08:24.156 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T14:08:24.157 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T14:08:24.158 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T14:08:24.160 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: remote image is not primary 2024-09-12T14:08:24.161 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T14:08:24.162 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T14:08:24.162 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T14:08:24.163 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T14:08:24.163 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T14:08:24.164 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T14:08:24.167 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T14:08:24.168 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T14:08:24.171 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 14:08:17' 2024-09-12T14:08:24.171 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary8 up+unknown '2024-09-12 14:08:17' 2024-09-12T14:08:24.171 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:08:24.171 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:08:24.171 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T14:08:24.171 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:08:24.171 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 14:08:17' 2024-09-12T14:08:24.171 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:08:24.172 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:08:24.172 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-09-12T14:08:24.172 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T14:08:24.173 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T14:08:24.174 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T14:08:24.180 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 14:08:17' 2024-09-12T14:08:24.181 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary10 up+unknown '2024-09-12 14:08:17' 2024-09-12T14:08:24.181 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:08:24.181 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:08:24.181 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T14:08:24.181 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:08:24.181 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 14:08:17' 2024-09-12T14:08:24.181 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:08:24.182 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:08:24.182 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-09-12T14:08:24.309 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041934232mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary19bf3eb7e-eb1b-464c-b684-743ee2313f71up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:08:17image-primary10b0e88823-56d3-40a7-88c0-8d4c22f008eeup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:08:17image-primary2ccaafd72-518c-4588-86d7-96c322bcf1bdup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:08:17image-primary3d356f103-adde-4c7a-855e-96419b23abefup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:08:17image-primary4a6cc92cf-b177-4aea-95c5-b61892fd9062up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:08:17image-primary5399c5b9a-89f0-4680-96c5-f2cac59115cfup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:08:17image-primary6f8d03c2d-535f-4038-a91f-c159711e8e6cup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:08:17image-primary75795f6f3-43c3-4e14-89d9-f4798e240cc6up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:08:17image-primary8fa74ee32-e225-4343-82da-f93063849113up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:08:17image-primary95991a19a-9e33-46ee-965b-85f990022e98up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:08:17' 2024-09-12T14:08:24.310 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T14:08:24.311 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary8'\'']/last_update' 2024-09-12T14:08:24.318 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 14:08:17' 2024-09-12T14:08:24.319 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary8'\'']/state' 2024-09-12T14:08:24.324 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+unknown 2024-09-12T14:08:24.325 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T14:08:24.326 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+unknown 2024-09-12T14:08:24.330 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T14:08:24.331 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:08:24.331 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:08:24.331 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster2 mirror image-primary8 up+unknown 2024-09-12T14:08:24.331 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T14:08:24.331 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:08:24.331 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T14:08:24.331 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:08:24.331 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:08:24.331 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:08:24.331 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:08:24.331 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T14:08:24.345 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041934232mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary19bf3eb7e-eb1b-464c-b684-743ee2313f71up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:08:17image-primary10b0e88823-56d3-40a7-88c0-8d4c22f008eeup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:08:17image-primary2ccaafd72-518c-4588-86d7-96c322bcf1bdup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:08:17image-primary3d356f103-adde-4c7a-855e-96419b23abefup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:08:17image-primary4a6cc92cf-b177-4aea-95c5-b61892fd9062up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:08:17image-primary5399c5b9a-89f0-4680-96c5-f2cac59115cfup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:08:17image-primary6f8d03c2d-535f-4038-a91f-c159711e8e6cup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:08:17image-primary75795f6f3-43c3-4e14-89d9-f4798e240cc6up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:08:17image-primary8fa74ee32-e225-4343-82da-f93063849113up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:08:17image-primary95991a19a-9e33-46ee-965b-85f990022e98up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:08:17' 2024-09-12T14:08:24.346 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T14:08:24.347 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary10'\'']/last_update' 2024-09-12T14:08:24.354 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 14:08:17' 2024-09-12T14:08:24.355 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary10'\'']/state' 2024-09-12T14:08:24.362 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+unknown 2024-09-12T14:08:24.363 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T14:08:24.364 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+unknown 2024-09-12T14:08:24.367 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T14:08:24.368 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:08:24.368 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:08:24.368 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster2 mirror image-primary10 up+unknown 2024-09-12T14:08:24.368 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T14:08:24.369 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:08:24.369 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T14:08:24.369 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:08:24.369 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:08:24.369 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:08:24.369 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:08:24.369 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T14:08:24.528 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster2 mirror image-primary7 up+unknown '' '' 2024-09-12T14:08:24.528 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T14:08:24.529 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:08:24.529 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T14:08:24.529 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:08:24.529 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:08:24.529 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:08:24.529 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:08:24.529 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:08:24.529 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster2 mirror image status mirror/image-primary7 2024-09-12T14:08:24.543 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster2 mirror image-primary2 up+unknown '' '' 2024-09-12T14:08:24.543 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T14:08:24.543 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:08:24.543 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T14:08:24.543 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:08:24.543 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:08:24.544 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:08:24.544 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:08:24.544 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:08:24.545 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster2 mirror image status mirror/image-primary2 2024-09-12T14:08:24.548 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster2 mirror image-primary4 up+unknown '' '' 2024-09-12T14:08:24.548 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T14:08:24.549 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:08:24.549 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T14:08:24.549 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:08:24.549 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:08:24.549 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:08:24.549 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:08:24.550 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:08:24.550 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster2 mirror image status mirror/image-primary4 2024-09-12T14:08:24.644 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary7: 2024-09-12T14:08:24.644 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 5795f6f3-43c3-4e14-89d9-f4798e240cc6 2024-09-12T14:08:24.644 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+unknown 2024-09-12T14:08:24.644 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: remote image is not primary 2024-09-12T14:08:24.644 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T14:08:24.644 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 14:08:17' 2024-09-12T14:08:24.644 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T14:08:24.647 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+unknown 2024-09-12T14:08:24.648 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T14:08:24.652 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: remote image is not primary 2024-09-12T14:08:24.653 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T14:08:24.654 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T14:08:24.654 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T14:08:24.659 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T14:08:24.659 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T14:08:24.662 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary2: 2024-09-12T14:08:24.662 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: ccaafd72-518c-4588-86d7-96c322bcf1bd 2024-09-12T14:08:24.662 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+unknown 2024-09-12T14:08:24.662 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: remote image is not primary 2024-09-12T14:08:24.663 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T14:08:24.663 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 14:08:17' 2024-09-12T14:08:24.663 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T14:08:24.664 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T14:08:24.665 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T14:08:24.665 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+unknown 2024-09-12T14:08:24.666 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T14:08:24.666 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T14:08:24.670 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: remote image is not primary 2024-09-12T14:08:24.670 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary4: 2024-09-12T14:08:24.670 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: a6cc92cf-b177-4aea-95c5-b61892fd9062 2024-09-12T14:08:24.670 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+unknown 2024-09-12T14:08:24.670 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: remote image is not primary 2024-09-12T14:08:24.671 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T14:08:24.671 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 14:08:17' 2024-09-12T14:08:24.671 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T14:08:24.671 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T14:08:24.672 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T14:08:24.672 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T14:08:24.673 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 14:08:17' 2024-09-12T14:08:24.673 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster2 mirror image-primary7 up+unknown '2024-09-12 14:08:17' 2024-09-12T14:08:24.673 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T14:08:24.673 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:08:24.673 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T14:08:24.673 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:08:24.673 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 14:08:17' 2024-09-12T14:08:24.673 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:08:24.674 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:08:24.674 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2024-09-12T14:08:24.675 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+unknown 2024-09-12T14:08:24.675 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T14:08:24.675 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T14:08:24.676 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T14:08:24.679 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: remote image is not primary 2024-09-12T14:08:24.679 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T14:08:24.680 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T14:08:24.680 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T14:08:24.681 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T14:08:24.681 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T14:08:24.682 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T14:08:24.683 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T14:08:24.684 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T14:08:24.687 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 14:08:17' 2024-09-12T14:08:24.687 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster2 mirror image-primary2 up+unknown '2024-09-12 14:08:17' 2024-09-12T14:08:24.687 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T14:08:24.687 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:08:24.687 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T14:08:24.688 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:08:24.688 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 14:08:17' 2024-09-12T14:08:24.688 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:08:24.688 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:08:24.688 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2024-09-12T14:08:24.689 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T14:08:24.689 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T14:08:24.691 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T14:08:24.698 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 14:08:17' 2024-09-12T14:08:24.698 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster2 mirror image-primary4 up+unknown '2024-09-12 14:08:17' 2024-09-12T14:08:24.698 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T14:08:24.698 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:08:24.698 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T14:08:24.698 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:08:24.698 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 14:08:17' 2024-09-12T14:08:24.698 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:08:24.699 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:08:24.699 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2024-09-12T14:08:24.927 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041924231mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary19bf3eb7e-eb1b-464c-b684-743ee2313f71up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:08:17image-primary10b0e88823-56d3-40a7-88c0-8d4c22f008eeup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:08:17image-primary2ccaafd72-518c-4588-86d7-96c322bcf1bdup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:08:17image-primary3d356f103-adde-4c7a-855e-96419b23abefup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:08:17image-primary4a6cc92cf-b177-4aea-95c5-b61892fd9062up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:08:17image-primary5399c5b9a-89f0-4680-96c5-f2cac59115cfup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:08:17image-primary6f8d03c2d-535f-4038-a91f-c159711e8e6cup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:08:17image-primary75795f6f3-43c3-4e14-89d9-f4798e240cc6up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:08:17image-primary8fa74ee32-e225-4343-82da-f93063849113up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:08:17image-primary95991a19a-9e33-46ee-965b-85f990022e98up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:08:17' 2024-09-12T14:08:24.927 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T14:08:24.929 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary2'\'']/last_update' 2024-09-12T14:08:24.938 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 14:08:17' 2024-09-12T14:08:24.939 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary2'\'']/state' 2024-09-12T14:08:24.943 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041924231mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary19bf3eb7e-eb1b-464c-b684-743ee2313f71up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:08:17image-primary10b0e88823-56d3-40a7-88c0-8d4c22f008eeup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:08:17image-primary2ccaafd72-518c-4588-86d7-96c322bcf1bdup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:08:17image-primary3d356f103-adde-4c7a-855e-96419b23abefup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:08:17image-primary4a6cc92cf-b177-4aea-95c5-b61892fd9062up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:08:17image-primary5399c5b9a-89f0-4680-96c5-f2cac59115cfup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:08:17image-primary6f8d03c2d-535f-4038-a91f-c159711e8e6cup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:08:17image-primary75795f6f3-43c3-4e14-89d9-f4798e240cc6up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:08:17image-primary8fa74ee32-e225-4343-82da-f93063849113up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:08:17image-primary95991a19a-9e33-46ee-965b-85f990022e98up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:08:17' 2024-09-12T14:08:24.944 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T14:08:24.945 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary7'\'']/last_update' 2024-09-12T14:08:24.948 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+unknown 2024-09-12T14:08:24.950 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T14:08:24.953 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+unknown 2024-09-12T14:08:24.953 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 14:08:17' 2024-09-12T14:08:24.955 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary7'\'']/state' 2024-09-12T14:08:24.955 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T14:08:24.957 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:08:24.957 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:08:24.957 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ promote_image cluster2 mirror image-primary2 2024-09-12T14:08:24.957 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T14:08:24.957 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:08:24.957 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T14:08:24.957 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local force= 2024-09-12T14:08:24.957 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 mirror image promote mirror/image-primary2 2024-09-12T14:08:24.965 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+unknown 2024-09-12T14:08:24.970 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T14:08:24.971 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+unknown 2024-09-12T14:08:24.971 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041924231mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary19bf3eb7e-eb1b-464c-b684-743ee2313f71up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:08:17image-primary10b0e88823-56d3-40a7-88c0-8d4c22f008eeup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:08:17image-primary2ccaafd72-518c-4588-86d7-96c322bcf1bdup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:08:17image-primary3d356f103-adde-4c7a-855e-96419b23abefup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:08:17image-primary4a6cc92cf-b177-4aea-95c5-b61892fd9062up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:08:17image-primary5399c5b9a-89f0-4680-96c5-f2cac59115cfup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:08:17image-primary6f8d03c2d-535f-4038-a91f-c159711e8e6cup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:08:17image-primary75795f6f3-43c3-4e14-89d9-f4798e240cc6up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:08:17image-primary8fa74ee32-e225-4343-82da-f93063849113up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:08:17image-primary95991a19a-9e33-46ee-965b-85f990022e98up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:08:17' 2024-09-12T14:08:24.971 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T14:08:24.972 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T14:08:24.972 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary4'\'']/last_update' 2024-09-12T14:08:24.972 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:08:24.973 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:08:24.973 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ promote_image cluster2 mirror image-primary7 2024-09-12T14:08:24.973 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T14:08:24.973 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:08:24.973 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T14:08:24.973 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local force= 2024-09-12T14:08:24.973 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 mirror image promote mirror/image-primary7 2024-09-12T14:08:24.974 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 14:08:17' 2024-09-12T14:08:24.976 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary4'\'']/state' 2024-09-12T14:08:24.982 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+unknown 2024-09-12T14:08:24.983 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T14:08:24.984 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+unknown 2024-09-12T14:08:24.988 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T14:08:24.989 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:08:24.989 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:08:24.989 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ promote_image cluster2 mirror image-primary4 2024-09-12T14:08:24.989 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T14:08:24.989 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:08:24.989 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T14:08:24.989 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local force= 2024-09-12T14:08:24.990 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 mirror image promote mirror/image-primary4 2024-09-12T14:08:25.072 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary9 up+unknown '' '' 2024-09-12T14:08:25.072 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:08:25.072 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:08:25.072 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T14:08:25.072 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:08:25.072 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:08:25.072 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:08:25.072 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:08:25.073 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:08:25.073 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary9 2024-09-12T14:08:25.164 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary9: 2024-09-12T14:08:25.164 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 5991a19a-9e33-46ee-965b-85f990022e98 2024-09-12T14:08:25.164 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+unknown 2024-09-12T14:08:25.165 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: remote image is not primary 2024-09-12T14:08:25.165 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T14:08:25.165 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 14:08:17' 2024-09-12T14:08:25.165 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T14:08:25.167 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+unknown 2024-09-12T14:08:25.167 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T14:08:25.172 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: remote image is not primary 2024-09-12T14:08:25.172 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T14:08:25.172 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T14:08:25.172 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T14:08:25.174 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T14:08:25.174 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T14:08:25.177 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T14:08:25.178 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T14:08:25.178 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T14:08:25.183 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 14:08:17' 2024-09-12T14:08:25.183 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary9 up+unknown '2024-09-12 14:08:17' 2024-09-12T14:08:25.183 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:08:25.183 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:08:25.183 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T14:08:25.183 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:08:25.183 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 14:08:17' 2024-09-12T14:08:25.183 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:08:25.183 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:08:25.184 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-09-12T14:08:25.330 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041934232mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary19bf3eb7e-eb1b-464c-b684-743ee2313f71up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:08:17image-primary10b0e88823-56d3-40a7-88c0-8d4c22f008eeup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:08:17image-primary2ccaafd72-518c-4588-86d7-96c322bcf1bdup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:08:17image-primary3d356f103-adde-4c7a-855e-96419b23abefup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:08:17image-primary4a6cc92cf-b177-4aea-95c5-b61892fd9062up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:08:17image-primary5399c5b9a-89f0-4680-96c5-f2cac59115cfup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:08:17image-primary6f8d03c2d-535f-4038-a91f-c159711e8e6cup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:08:17image-primary75795f6f3-43c3-4e14-89d9-f4798e240cc6up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:08:17image-primary8fa74ee32-e225-4343-82da-f93063849113up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:08:17image-primary95991a19a-9e33-46ee-965b-85f990022e98up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:08:17' 2024-09-12T14:08:25.331 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T14:08:25.331 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary9'\'']/last_update' 2024-09-12T14:08:25.335 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster2 mirror image-primary8 up+unknown '' '' 2024-09-12T14:08:25.335 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T14:08:25.335 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:08:25.335 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T14:08:25.335 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:08:25.336 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:08:25.336 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:08:25.336 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:08:25.336 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:08:25.336 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster2 mirror image status mirror/image-primary8 2024-09-12T14:08:25.339 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 14:08:17' 2024-09-12T14:08:25.340 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary9'\'']/state' 2024-09-12T14:08:25.348 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+unknown 2024-09-12T14:08:25.348 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T14:08:25.349 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+unknown 2024-09-12T14:08:25.352 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T14:08:25.353 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:08:25.354 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:08:25.354 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster2 mirror image-primary9 up+unknown 2024-09-12T14:08:25.354 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T14:08:25.354 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:08:25.354 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T14:08:25.354 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:08:25.354 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:08:25.354 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:08:25.354 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:08:25.354 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T14:08:25.372 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster2 mirror image-primary10 up+unknown '' '' 2024-09-12T14:08:25.372 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T14:08:25.372 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:08:25.373 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T14:08:25.373 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:08:25.373 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:08:25.373 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:08:25.373 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:08:25.374 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:08:25.374 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster2 mirror image status mirror/image-primary10 2024-09-12T14:08:25.451 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary8: 2024-09-12T14:08:25.451 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: fa74ee32-e225-4343-82da-f93063849113 2024-09-12T14:08:25.451 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+unknown 2024-09-12T14:08:25.451 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: remote image is not primary 2024-09-12T14:08:25.451 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T14:08:25.452 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 14:08:17' 2024-09-12T14:08:25.452 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T14:08:25.455 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+unknown 2024-09-12T14:08:25.456 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T14:08:25.459 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: remote image is not primary 2024-09-12T14:08:25.460 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T14:08:25.461 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T14:08:25.462 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T14:08:25.466 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T14:08:25.466 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T14:08:25.471 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T14:08:25.471 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T14:08:25.473 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T14:08:25.478 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 14:08:17' 2024-09-12T14:08:25.479 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster2 mirror image-primary8 up+unknown '2024-09-12 14:08:17' 2024-09-12T14:08:25.479 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T14:08:25.479 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:08:25.479 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T14:08:25.479 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:08:25.479 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 14:08:17' 2024-09-12T14:08:25.479 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:08:25.479 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:08:25.479 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2024-09-12T14:08:25.487 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary10: 2024-09-12T14:08:25.487 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: b0e88823-56d3-40a7-88c0-8d4c22f008ee 2024-09-12T14:08:25.487 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+unknown 2024-09-12T14:08:25.487 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: remote image is not primary 2024-09-12T14:08:25.487 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T14:08:25.487 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 14:08:17' 2024-09-12T14:08:25.487 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T14:08:25.491 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+unknown 2024-09-12T14:08:25.492 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T14:08:25.497 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: remote image is not primary 2024-09-12T14:08:25.497 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T14:08:25.498 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T14:08:25.498 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T14:08:25.501 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T14:08:25.502 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T14:08:25.507 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T14:08:25.507 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T14:08:25.508 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T14:08:25.515 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 14:08:17' 2024-09-12T14:08:25.515 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster2 mirror image-primary10 up+unknown '2024-09-12 14:08:17' 2024-09-12T14:08:25.515 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T14:08:25.515 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:08:25.515 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T14:08:25.515 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:08:25.515 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 14:08:17' 2024-09-12T14:08:25.515 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:08:25.516 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:08:25.516 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2024-09-12T14:08:25.641 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041924231mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary19bf3eb7e-eb1b-464c-b684-743ee2313f71up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:08:17image-primary10b0e88823-56d3-40a7-88c0-8d4c22f008eeup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:08:17image-primary2ccaafd72-518c-4588-86d7-96c322bcf1bdup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:08:17image-primary3d356f103-adde-4c7a-855e-96419b23abefup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:08:17image-primary4a6cc92cf-b177-4aea-95c5-b61892fd9062up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:08:17image-primary5399c5b9a-89f0-4680-96c5-f2cac59115cfup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:08:17image-primary6f8d03c2d-535f-4038-a91f-c159711e8e6cup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:08:17image-primary75795f6f3-43c3-4e14-89d9-f4798e240cc6up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:08:17image-primary8fa74ee32-e225-4343-82da-f93063849113up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:08:17image-primary95991a19a-9e33-46ee-965b-85f990022e98up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:08:17' 2024-09-12T14:08:25.641 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T14:08:25.642 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary8'\'']/last_update' 2024-09-12T14:08:25.656 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 14:08:17' 2024-09-12T14:08:25.659 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary8'\'']/state' 2024-09-12T14:08:25.670 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+unknown 2024-09-12T14:08:25.671 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T14:08:25.672 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+unknown 2024-09-12T14:08:25.676 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T14:08:25.678 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:08:25.679 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:08:25.679 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ promote_image cluster2 mirror image-primary8 2024-09-12T14:08:25.679 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T14:08:25.680 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:08:25.680 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T14:08:25.680 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local force= 2024-09-12T14:08:25.680 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 mirror image promote mirror/image-primary8 2024-09-12T14:08:25.731 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041924231mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary19bf3eb7e-eb1b-464c-b684-743ee2313f71up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:08:17image-primary10b0e88823-56d3-40a7-88c0-8d4c22f008eeup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:08:17image-primary2ccaafd72-518c-4588-86d7-96c322bcf1bdup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:08:17image-primary3d356f103-adde-4c7a-855e-96419b23abefup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:08:17image-primary4a6cc92cf-b177-4aea-95c5-b61892fd9062up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:08:17image-primary5399c5b9a-89f0-4680-96c5-f2cac59115cfup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:08:17image-primary6f8d03c2d-535f-4038-a91f-c159711e8e6cup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:08:17image-primary75795f6f3-43c3-4e14-89d9-f4798e240cc6up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:08:17image-primary8fa74ee32-e225-4343-82da-f93063849113up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:08:17image-primary95991a19a-9e33-46ee-965b-85f990022e98up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:08:17' 2024-09-12T14:08:25.731 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T14:08:25.731 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary10'\'']/last_update' 2024-09-12T14:08:25.738 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 14:08:17' 2024-09-12T14:08:25.740 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary10'\'']/state' 2024-09-12T14:08:25.745 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+unknown 2024-09-12T14:08:25.748 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T14:08:25.748 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+unknown 2024-09-12T14:08:25.759 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T14:08:25.760 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:08:25.760 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:08:25.761 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ promote_image cluster2 mirror image-primary10 2024-09-12T14:08:25.761 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T14:08:25.761 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:08:25.761 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T14:08:25.761 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local force= 2024-09-12T14:08:25.761 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 mirror image promote mirror/image-primary10 2024-09-12T14:08:25.906 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Image promoted to primary 2024-09-12T14:08:25.909 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Image promoted to primary 2024-09-12T14:08:25.911 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Image promoted to primary 2024-09-12T14:08:25.957 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \n\b\d ]] 2024-09-12T14:08:25.957 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \k\r\b\d ]] 2024-09-12T14:08:25.958 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \n\b\d ]] 2024-09-12T14:08:25.958 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \k\r\b\d ]] 2024-09-12T14:08:25.958 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo rbd --cluster cluster2 device map -t krbd mirror/image-primary7 2024-09-12T14:08:25.959 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo rbd --cluster cluster2 device map -t krbd mirror/image-primary4 2024-09-12T14:08:25.966 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \n\b\d ]] 2024-09-12T14:08:25.966 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \k\r\b\d ]] 2024-09-12T14:08:25.969 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo rbd --cluster cluster2 device map -t krbd mirror/image-primary2 2024-09-12T14:08:26.216 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ dev=/dev/rbd4 2024-09-12T14:08:26.219 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ awk '{print $1}' 2024-09-12T14:08:26.220 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo md5sum /dev/rbd4 2024-09-12T14:08:26.252 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ dev=/dev/rbd5 2024-09-12T14:08:26.254 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ dev=/dev/rbd6 2024-09-12T14:08:26.255 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo md5sum /dev/rbd5 2024-09-12T14:08:26.255 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ awk '{print $1}' 2024-09-12T14:08:26.257 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ awk '{print $1}' 2024-09-12T14:08:26.258 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo md5sum /dev/rbd6 2024-09-12T14:08:26.358 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster2 mirror image-primary9 up+unknown '' '' 2024-09-12T14:08:26.358 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T14:08:26.358 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:08:26.358 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T14:08:26.358 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:08:26.358 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:08:26.359 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:08:26.359 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:08:26.359 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:08:26.360 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster2 mirror image status mirror/image-primary9 2024-09-12T14:08:26.502 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary9: 2024-09-12T14:08:26.502 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 5991a19a-9e33-46ee-965b-85f990022e98 2024-09-12T14:08:26.502 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+unknown 2024-09-12T14:08:26.503 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: remote image is not primary 2024-09-12T14:08:26.503 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T14:08:26.503 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 14:08:17' 2024-09-12T14:08:26.503 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T14:08:26.507 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+unknown 2024-09-12T14:08:26.508 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T14:08:26.513 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: remote image is not primary 2024-09-12T14:08:26.513 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T14:08:26.515 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T14:08:26.515 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T14:08:26.520 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T14:08:26.521 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T14:08:26.525 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T14:08:26.526 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T14:08:26.528 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T14:08:26.535 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 14:08:17' 2024-09-12T14:08:26.535 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster2 mirror image-primary9 up+unknown '2024-09-12 14:08:17' 2024-09-12T14:08:26.535 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T14:08:26.535 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:08:26.535 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T14:08:26.535 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:08:26.535 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 14:08:17' 2024-09-12T14:08:26.535 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:08:26.537 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:08:26.537 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2024-09-12T14:08:26.779 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041924231mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary19bf3eb7e-eb1b-464c-b684-743ee2313f71up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:08:17image-primary10b0e88823-56d3-40a7-88c0-8d4c22f008eeup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:08:17image-primary2ccaafd72-518c-4588-86d7-96c322bcf1bdup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:08:17image-primary3d356f103-adde-4c7a-855e-96419b23abefup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:08:17image-primary4a6cc92cf-b177-4aea-95c5-b61892fd9062up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:08:17image-primary5399c5b9a-89f0-4680-96c5-f2cac59115cfup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:08:17image-primary6f8d03c2d-535f-4038-a91f-c159711e8e6cup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:08:17image-primary75795f6f3-43c3-4e14-89d9-f4798e240cc6up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:08:17image-primary8fa74ee32-e225-4343-82da-f93063849113up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:08:17image-primary95991a19a-9e33-46ee-965b-85f990022e98up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:08:17' 2024-09-12T14:08:26.779 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T14:08:26.779 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary9'\'']/last_update' 2024-09-12T14:08:26.787 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 14:08:17' 2024-09-12T14:08:26.789 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary9'\'']/state' 2024-09-12T14:08:26.797 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+unknown 2024-09-12T14:08:26.799 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+unknown 2024-09-12T14:08:26.799 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T14:08:26.804 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T14:08:26.805 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:08:26.805 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:08:26.805 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ promote_image cluster2 mirror image-primary9 2024-09-12T14:08:26.805 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T14:08:26.805 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:08:26.806 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T14:08:26.806 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local force= 2024-09-12T14:08:26.806 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 mirror image promote mirror/image-primary9 2024-09-12T14:08:26.891 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Image promoted to primary 2024-09-12T14:08:26.891 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Image promoted to primary 2024-09-12T14:08:26.927 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \n\b\d ]] 2024-09-12T14:08:26.928 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \k\r\b\d ]] 2024-09-12T14:08:26.928 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo rbd --cluster cluster2 device map -t krbd mirror/image-primary10 2024-09-12T14:08:26.933 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \n\b\d ]] 2024-09-12T14:08:26.933 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \k\r\b\d ]] 2024-09-12T14:08:26.935 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo rbd --cluster cluster2 device map -t krbd mirror/image-primary8 2024-09-12T14:08:27.409 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ dev=/dev/rbd8 2024-09-12T14:08:27.413 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ awk '{print $1}' 2024-09-12T14:08:27.413 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo md5sum /dev/rbd8 2024-09-12T14:08:27.417 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ dev=/dev/rbd7 2024-09-12T14:08:27.420 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo md5sum /dev/rbd7 2024-09-12T14:08:27.421 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ awk '{print $1}' 2024-09-12T14:08:27.895 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Image promoted to primary 2024-09-12T14:08:27.949 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \n\b\d ]] 2024-09-12T14:08:27.949 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \k\r\b\d ]] 2024-09-12T14:08:27.954 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo rbd --cluster cluster2 device map -t krbd mirror/image-primary9 2024-09-12T14:08:28.382 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ dev=/dev/rbd9 2024-09-12T14:08:28.387 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo md5sum /dev/rbd9 2024-09-12T14:08:28.389 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ awk '{print $1}' 2024-09-12T14:09:07.947 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ promote_md5=f23f150e00f53f7158113196b6b4bd34 2024-09-12T14:09:07.947 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo rbd --cluster cluster2 device unmap -t krbd /dev/rbd1 2024-09-12T14:09:08.216 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ f23f150e00f53f7158113196b6b4bd34 != \f\2\3\f\1\5\0\e\0\0\f\5\3\f\7\1\5\8\1\1\3\1\9\6\b\6\b\4\b\d\3\4 ]] 2024-09-12T14:09:08.219 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${COMPARE_PIDS[@]} 2024-09-12T14:09:08.219 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 137666 2024-09-12T14:09:11.359 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ promote_md5=527686362ecdfb4b7f3c247431caa979 2024-09-12T14:09:11.360 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo rbd --cluster cluster2 device unmap -t krbd /dev/rbd0 2024-09-12T14:09:11.635 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ 527686362ecdfb4b7f3c247431caa979 != \5\2\7\6\8\6\3\6\2\e\c\d\f\b\4\b\7\f\3\c\2\4\7\4\3\1\c\a\a\9\7\9 ]] 2024-09-12T14:09:13.186 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ promote_md5=a58afde9a16731c66f673f9349cf3686 2024-09-12T14:09:13.186 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo rbd --cluster cluster2 device unmap -t krbd /dev/rbd2 2024-09-12T14:09:13.387 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ a58afde9a16731c66f673f9349cf3686 != \a\5\8\a\f\d\e\9\a\1\6\7\3\1\c\6\6\f\6\7\3\f\9\3\4\9\c\f\3\6\8\6 ]] 2024-09-12T14:09:16.064 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ promote_md5=0393d04de9574824e5fa7aefcef31dd8 2024-09-12T14:09:16.064 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo rbd --cluster cluster2 device unmap -t krbd /dev/rbd3 2024-09-12T14:09:16.220 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ 0393d04de9574824e5fa7aefcef31dd8 != \0\3\9\3\d\0\4\d\e\9\5\7\4\8\2\4\e\5\f\a\7\a\e\f\c\e\f\3\1\d\d\8 ]] 2024-09-12T14:09:17.399 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ promote_md5=89cebaf0f069635630e707839fd61396 2024-09-12T14:09:17.399 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo rbd --cluster cluster2 device unmap -t krbd /dev/rbd4 2024-09-12T14:09:17.604 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ 89cebaf0f069635630e707839fd61396 != \8\9\c\e\b\a\f\0\f\0\6\9\6\3\5\6\3\0\e\7\0\7\8\3\9\f\d\6\1\3\9\6 ]] 2024-09-12T14:09:17.970 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ promote_md5=42539596e7b0352ed1e2ab5083d992fe 2024-09-12T14:09:17.970 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo rbd --cluster cluster2 device unmap -t krbd /dev/rbd7 2024-09-12T14:09:18.174 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ 42539596e7b0352ed1e2ab5083d992fe != \4\2\5\3\9\5\9\6\e\7\b\0\3\5\2\e\d\1\e\2\a\b\5\0\8\3\d\9\9\2\f\e ]] 2024-09-12T14:09:18.325 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ promote_md5=726359b6503be8b926d2ecbe06764c40 2024-09-12T14:09:18.325 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo rbd --cluster cluster2 device unmap -t krbd /dev/rbd5 2024-09-12T14:09:18.454 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ 726359b6503be8b926d2ecbe06764c40 != \7\2\6\3\5\9\b\6\5\0\3\b\e\8\b\9\2\6\d\2\e\c\b\e\0\6\7\6\4\c\4\0 ]] 2024-09-12T14:09:18.455 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${COMPARE_PIDS[@]} 2024-09-12T14:09:18.455 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 137668 2024-09-12T14:09:18.455 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${COMPARE_PIDS[@]} 2024-09-12T14:09:18.455 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 137670 2024-09-12T14:09:18.466 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ promote_md5=3318816507f6cb2bf4e2cb61d48dd326 2024-09-12T14:09:18.466 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo rbd --cluster cluster2 device unmap -t krbd /dev/rbd6 2024-09-12T14:09:18.587 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ 3318816507f6cb2bf4e2cb61d48dd326 != \3\3\1\8\8\1\6\5\0\7\f\6\c\b\2\b\f\4\e\2\c\b\6\1\d\4\8\d\d\3\2\6 ]] 2024-09-12T14:09:18.588 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${COMPARE_PIDS[@]} 2024-09-12T14:09:18.588 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 137672 2024-09-12T14:09:18.588 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${COMPARE_PIDS[@]} 2024-09-12T14:09:18.588 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 137674 2024-09-12T14:09:18.588 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${COMPARE_PIDS[@]} 2024-09-12T14:09:18.588 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 137676 2024-09-12T14:09:18.588 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${COMPARE_PIDS[@]} 2024-09-12T14:09:18.588 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 137678 2024-09-12T14:09:18.588 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${COMPARE_PIDS[@]} 2024-09-12T14:09:18.588 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 137679 2024-09-12T14:09:19.029 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ promote_md5=761401645037fbc384db7229420cff4e 2024-09-12T14:09:19.030 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo rbd --cluster cluster2 device unmap -t krbd /dev/rbd8 2024-09-12T14:09:19.172 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ 761401645037fbc384db7229420cff4e != \7\6\1\4\0\1\6\4\5\0\3\7\f\b\c\3\8\4\d\b\7\2\2\9\4\2\0\c\f\f\4\e ]] 2024-09-12T14:09:19.967 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ promote_md5=08b3821aa5466400f387d2934beb2fef 2024-09-12T14:09:19.967 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo rbd --cluster cluster2 device unmap -t krbd /dev/rbd9 2024-09-12T14:09:20.061 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ 08b3821aa5466400f387d2934beb2fef != \0\8\b\3\8\2\1\a\a\5\4\6\6\4\0\0\f\3\8\7\d\2\9\3\4\b\e\b\2\f\e\f ]] 2024-09-12T14:09:20.061 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${COMPARE_PIDS[@]} 2024-09-12T14:09:20.061 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 137683 2024-09-12T14:09:20.061 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ (( RET != 0 )) 2024-09-12T14:09:20.061 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T14:09:20.062 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ IMG=image-primary1 2024-09-12T14:09:20.062 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_replaying_status_in_pool_dir cluster1 mirror image-primary1 2024-09-12T14:09:20.062 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:09:20.062 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:09:20.062 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T14:09:20.062 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' snapshot = journal ']' 2024-09-12T14:09:20.062 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary1 up+replaying 2024-09-12T14:09:20.063 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:09:20.063 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:09:20.063 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T14:09:20.063 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T14:09:20.063 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:09:20.063 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:09:20.063 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:09:20.064 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T14:09:21.062 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary1 up+replaying '' '' 2024-09-12T14:09:21.062 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:09:21.062 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:09:21.062 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T14:09:21.062 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T14:09:21.062 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:09:21.062 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:09:21.062 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:09:21.063 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:09:21.063 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary1 2024-09-12T14:09:21.144 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary1: 2024-09-12T14:09:21.145 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 9bf3eb7e-eb1b-464c-b684-743ee2313f71 2024-09-12T14:09:21.145 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+replaying 2024-09-12T14:09:21.145 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: replaying, {"bytes_per_second":3332300.8,"bytes_per_snapshot":66646016.0,"last_snapshot_bytes":66646016,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726150099,"remote_snapshot_timestamp":1726150099,"replay_state":"idle"} 2024-09-12T14:09:21.145 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T14:09:21.145 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 14:09:17' 2024-09-12T14:09:21.145 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+replaying' 2024-09-12T14:09:21.147 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+replaying 2024-09-12T14:09:21.147 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T14:09:21.150 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: replaying, {"bytes_per_second":3332300.8,"bytes_per_snapshot":66646016.0,"last_snapshot_bytes":66646016,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726150099,"remote_snapshot_timestamp":1726150099,"replay_state":"idle"} 2024-09-12T14:09:21.150 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T14:09:21.151 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+replaying 2024-09-12T14:09:21.151 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T14:09:21.153 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+replaying 2024-09-12T14:09:21.154 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T14:09:21.156 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T14:09:21.156 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T14:09:21.157 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T14:09:21.161 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 14:09:17' 2024-09-12T14:09:21.161 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary1 up+replaying '2024-09-12 14:09:17' 2024-09-12T14:09:21.161 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:09:21.161 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:09:21.161 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T14:09:21.161 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T14:09:21.161 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 14:09:17' 2024-09-12T14:09:21.162 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:09:21.162 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:09:21.162 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-09-12T14:09:21.273 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041934232mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary19bf3eb7e-eb1b-464c-b684-743ee2313f71up+replayingreplaying, {"bytes_per_second":3332300.8,"bytes_per_snapshot":66646016.0,"last_snapshot_bytes":66646016,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726150099,"remote_snapshot_timestamp":1726150099,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 14:09:17image-primary10b0e88823-56d3-40a7-88c0-8d4c22f008eeup+replayingreplaying, {"bytes_per_second":3139379.2,"bytes_per_snapshot":62787584.0,"last_snapshot_bytes":62787584,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726150106,"remote_snapshot_timestamp":1726150106,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 14:09:17image-primary2ccaafd72-518c-4588-86d7-96c322bcf1bdup+replayingreplaying, {"bytes_per_second":3448627.2,"bytes_per_snapshot":68972544.0,"last_snapshot_bytes":68972544,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726150105,"remote_snapshot_timestamp":1726150105,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 14:09:17image-primary3d356f103-adde-4c7a-855e-96419b23abefup+replayingreplaying, {"bytes_per_second":3221504.0,"bytes_per_snapshot":64430080.0,"last_snapshot_bytes":64430080,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726150100,"remote_snapshot_timestamp":1726150100,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 14:09:17image-primary4a6cc92cf-b177-4aea-95c5-b61892fd9062up+replayingreplaying, {"bytes_per_second":3248332.8,"bytes_per_snapshot":64966656.0,"last_snapshot_bytes":64966656,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726150105,"remote_snapshot_timestamp":1726150105,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 14:09:17image-primary5399c5b9a-89f0-4680-96c5-f2cac59115cfup+replayingreplaying, {"bytes_per_second":3616153.6,"bytes_per_snapshot":72323072.0,"last_snapshot_bytes":72323072,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726150099,"remote_snapshot_timestamp":1726150099,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 14:09:17image-primary6f8d03c2d-535f-4038-a91f-c159711e8e6cup+replayingreplaying, {"bytes_per_second":3246489.6,"bytes_per_snapshot":64929792.0,"last_snapshot_bytes":64929792,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726150101,"remote_snapshot_timestamp":1726150101,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 14:09:17image-primary75795f6f3-43c3-4e14-89d9-f4798e240cc6up+replayingreplaying, {"bytes_per_second":3157401.6,"bytes_per_snapshot":63148032.0,"last_snapshot_bytes":63148032,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726150105,"remote_snapshot_timestamp":1726150105,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 14:09:17image-primary8fa74ee32-e225-4343-82da-f93063849113up+replayingreplaying, {"bytes_per_second":3141632.0,"bytes_per_snapshot":62832640.0,"last_snapshot_bytes":62832640,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726150106,"remote_snapshot_timestamp":1726150106,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 14:09:17image-primary95991a19a-9e33-46ee-965b-85f990022e98up+replayingreplaying, {"bytes_per_second":3153920.0,"bytes_per_snapshot":63078400.0,"last_snapshot_bytes":63078400,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726150107,"remote_snapshot_timestamp":1726150107,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 14:09:17' 2024-09-12T14:09:21.273 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T14:09:21.274 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary1'\'']/last_update' 2024-09-12T14:09:21.279 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 14:09:17' 2024-09-12T14:09:21.280 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary1'\'']/state' 2024-09-12T14:09:21.284 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+replaying 2024-09-12T14:09:21.284 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+replaying 2024-09-12T14:09:21.285 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+replaying 2024-09-12T14:09:21.287 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+replaying 2024-09-12T14:09:21.287 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:09:21.287 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:09:21.287 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ remove_image cluster2 mirror image-primary1 2024-09-12T14:09:21.288 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T14:09:21.288 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:09:21.288 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T14:09:21.288 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 snap purge mirror/image-primary1 2024-09-12T14:09:21.349 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 rm mirror/image-primary1 2024-09-12T14:09:21.496 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T14:09:21.496+0000 7f67e5200640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x56055bafb520 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-09-12T14:09:21.496 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T14:09:21.496+0000 7f67eca00640 -1 rbd::mirror::ImageReplayer: 0x5605561ad680 [3/9bf3eb7e-eb1b-464c-b684-743ee2313f71] operator(): start failed: (85) Interrupted system call should be restarted 2024-09-12T14:09:22.982 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-09-12T14:09:22.988 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_image_removal cluster1 mirror image-primary1 2024-09-12T14:09:22.988 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:09:22.988 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:09:22.988 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T14:09:22.988 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:09:22.989 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 ls mirror 2024-09-12T14:09:22.989 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep -wq image-primary1 2024-09-12T14:09:23.040 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:09:23.040 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rm -rf test-primary1 2024-09-12T14:09:23.135 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T14:09:23.135 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ IMG=image-primary2 2024-09-12T14:09:23.135 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_replaying_status_in_pool_dir cluster1 mirror image-primary2 2024-09-12T14:09:23.135 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:09:23.135 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:09:23.135 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T14:09:23.135 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' snapshot = journal ']' 2024-09-12T14:09:23.136 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary2 up+replaying 2024-09-12T14:09:23.136 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:09:23.136 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:09:23.136 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T14:09:23.136 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T14:09:23.136 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:09:23.136 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:09:23.136 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:09:23.136 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T14:09:24.137 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary2 up+replaying '' '' 2024-09-12T14:09:24.137 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:09:24.137 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:09:24.137 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T14:09:24.137 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T14:09:24.138 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:09:24.138 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:09:24.138 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:09:24.138 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:09:24.138 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary2 2024-09-12T14:09:24.222 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary2: 2024-09-12T14:09:24.223 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: ccaafd72-518c-4588-86d7-96c322bcf1bd 2024-09-12T14:09:24.223 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+replaying 2024-09-12T14:09:24.223 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: replaying, {"bytes_per_second":3448627.2,"bytes_per_snapshot":68972544.0,"last_snapshot_bytes":68972544,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726150105,"remote_snapshot_timestamp":1726150105,"replay_state":"idle"} 2024-09-12T14:09:24.223 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T14:09:24.223 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 14:09:17' 2024-09-12T14:09:24.223 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+replaying' 2024-09-12T14:09:24.225 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+replaying 2024-09-12T14:09:24.225 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T14:09:24.228 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: replaying, {"bytes_per_second":3448627.2,"bytes_per_snapshot":68972544.0,"last_snapshot_bytes":68972544,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726150105,"remote_snapshot_timestamp":1726150105,"replay_state":"idle"} 2024-09-12T14:09:24.228 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T14:09:24.229 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+replaying 2024-09-12T14:09:24.229 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T14:09:24.231 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+replaying 2024-09-12T14:09:24.231 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T14:09:24.234 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T14:09:24.234 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T14:09:24.235 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T14:09:24.239 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 14:09:17' 2024-09-12T14:09:24.239 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary2 up+replaying '2024-09-12 14:09:17' 2024-09-12T14:09:24.239 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:09:24.240 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:09:24.240 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T14:09:24.240 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T14:09:24.240 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 14:09:17' 2024-09-12T14:09:24.240 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:09:24.240 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:09:24.240 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-09-12T14:09:24.349 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING941934232mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary10b0e88823-56d3-40a7-88c0-8d4c22f008eeup+replayingreplaying, {"bytes_per_second":3139379.2,"bytes_per_snapshot":62787584.0,"last_snapshot_bytes":62787584,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726150106,"remote_snapshot_timestamp":1726150106,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 14:09:17image-primary2ccaafd72-518c-4588-86d7-96c322bcf1bdup+replayingreplaying, {"bytes_per_second":3448627.2,"bytes_per_snapshot":68972544.0,"last_snapshot_bytes":68972544,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726150105,"remote_snapshot_timestamp":1726150105,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 14:09:17image-primary3d356f103-adde-4c7a-855e-96419b23abefup+replayingreplaying, {"bytes_per_second":3221504.0,"bytes_per_snapshot":64430080.0,"last_snapshot_bytes":64430080,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726150100,"remote_snapshot_timestamp":1726150100,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 14:09:17image-primary4a6cc92cf-b177-4aea-95c5-b61892fd9062up+replayingreplaying, {"bytes_per_second":3248332.8,"bytes_per_snapshot":64966656.0,"last_snapshot_bytes":64966656,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726150105,"remote_snapshot_timestamp":1726150105,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 14:09:17image-primary5399c5b9a-89f0-4680-96c5-f2cac59115cfup+replayingreplaying, {"bytes_per_second":3616153.6,"bytes_per_snapshot":72323072.0,"last_snapshot_bytes":72323072,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726150099,"remote_snapshot_timestamp":1726150099,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 14:09:17image-primary6f8d03c2d-535f-4038-a91f-c159711e8e6cup+replayingreplaying, {"bytes_per_second":3246489.6,"bytes_per_snapshot":64929792.0,"last_snapshot_bytes":64929792,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726150101,"remote_snapshot_timestamp":1726150101,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 14:09:17image-primary75795f6f3-43c3-4e14-89d9-f4798e240cc6up+replayingreplaying, {"bytes_per_second":3157401.6,"bytes_per_snapshot":63148032.0,"last_snapshot_bytes":63148032,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726150105,"remote_snapshot_timestamp":1726150105,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 14:09:17image-primary8fa74ee32-e225-4343-82da-f93063849113up+replayingreplaying, {"bytes_per_second":3141632.0,"bytes_per_snapshot":62832640.0,"last_snapshot_bytes":62832640,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726150106,"remote_snapshot_timestamp":1726150106,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 14:09:17image-primary95991a19a-9e33-46ee-965b-85f990022e98up+replayingreplaying, {"bytes_per_second":3153920.0,"bytes_per_snapshot":63078400.0,"last_snapshot_bytes":63078400,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726150107,"remote_snapshot_timestamp":1726150107,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 14:09:17' 2024-09-12T14:09:24.349 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T14:09:24.349 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary2'\'']/last_update' 2024-09-12T14:09:24.353 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 14:09:17' 2024-09-12T14:09:24.354 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary2'\'']/state' 2024-09-12T14:09:24.358 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+replaying 2024-09-12T14:09:24.358 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+replaying 2024-09-12T14:09:24.359 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+replaying 2024-09-12T14:09:24.361 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+replaying 2024-09-12T14:09:24.361 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:09:24.362 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:09:24.362 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ remove_image cluster2 mirror image-primary2 2024-09-12T14:09:24.362 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T14:09:24.362 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:09:24.362 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T14:09:24.362 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 snap purge mirror/image-primary2 2024-09-12T14:09:24.422 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 rm mirror/image-primary2 2024-09-12T14:09:24.568 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T14:09:24.568+0000 7f67e5200640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x56055b9721a0 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-09-12T14:09:24.568 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T14:09:24.568+0000 7f67eca00640 -1 rbd::mirror::ImageReplayer: 0x5605561acf00 [3/ccaafd72-518c-4588-86d7-96c322bcf1bd] operator(): start failed: (85) Interrupted system call should be restarted 2024-09-12T14:09:25.990 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-09-12T14:09:25.997 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_image_removal cluster1 mirror image-primary2 2024-09-12T14:09:25.997 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:09:25.997 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:09:25.997 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T14:09:25.997 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:09:25.998 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 ls mirror 2024-09-12T14:09:25.998 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep -wq image-primary2 2024-09-12T14:09:26.054 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:09:26.054 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rm -rf test-primary2 2024-09-12T14:09:26.057 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T14:09:26.057 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ IMG=image-primary3 2024-09-12T14:09:26.057 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_replaying_status_in_pool_dir cluster1 mirror image-primary3 2024-09-12T14:09:26.057 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:09:26.057 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:09:26.057 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T14:09:26.058 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' snapshot = journal ']' 2024-09-12T14:09:26.058 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary3 up+replaying 2024-09-12T14:09:26.058 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:09:26.058 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:09:26.058 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T14:09:26.058 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T14:09:26.058 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:09:26.058 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:09:26.058 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:09:26.058 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T14:09:27.060 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary3 up+replaying '' '' 2024-09-12T14:09:27.060 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:09:27.060 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:09:27.060 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T14:09:27.060 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T14:09:27.060 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:09:27.060 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:09:27.060 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:09:27.061 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:09:27.061 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary3 2024-09-12T14:09:27.144 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary3: 2024-09-12T14:09:27.144 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: d356f103-adde-4c7a-855e-96419b23abef 2024-09-12T14:09:27.144 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+replaying 2024-09-12T14:09:27.144 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: replaying, {"bytes_per_second":3221504.0,"bytes_per_snapshot":64430080.0,"last_snapshot_bytes":64430080,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726150100,"remote_snapshot_timestamp":1726150100,"replay_state":"idle"} 2024-09-12T14:09:27.144 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T14:09:27.144 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 14:09:17' 2024-09-12T14:09:27.144 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+replaying' 2024-09-12T14:09:27.146 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+replaying 2024-09-12T14:09:27.146 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T14:09:27.149 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: replaying, {"bytes_per_second":3221504.0,"bytes_per_snapshot":64430080.0,"last_snapshot_bytes":64430080,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726150100,"remote_snapshot_timestamp":1726150100,"replay_state":"idle"} 2024-09-12T14:09:27.149 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T14:09:27.150 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+replaying 2024-09-12T14:09:27.150 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T14:09:27.152 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+replaying 2024-09-12T14:09:27.153 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T14:09:27.155 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T14:09:27.156 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T14:09:27.156 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T14:09:27.160 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 14:09:17' 2024-09-12T14:09:27.160 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary3 up+replaying '2024-09-12 14:09:17' 2024-09-12T14:09:27.161 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:09:27.161 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:09:27.161 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T14:09:27.161 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T14:09:27.161 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 14:09:17' 2024-09-12T14:09:27.161 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:09:27.161 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:09:27.161 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-09-12T14:09:27.264 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING841934232mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary10b0e88823-56d3-40a7-88c0-8d4c22f008eeup+replayingreplaying, {"bytes_per_second":3139379.2,"bytes_per_snapshot":62787584.0,"last_snapshot_bytes":62787584,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726150106,"remote_snapshot_timestamp":1726150106,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 14:09:17image-primary3d356f103-adde-4c7a-855e-96419b23abefup+replayingreplaying, {"bytes_per_second":3221504.0,"bytes_per_snapshot":64430080.0,"last_snapshot_bytes":64430080,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726150100,"remote_snapshot_timestamp":1726150100,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 14:09:17image-primary4a6cc92cf-b177-4aea-95c5-b61892fd9062up+replayingreplaying, {"bytes_per_second":3248332.8,"bytes_per_snapshot":64966656.0,"last_snapshot_bytes":64966656,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726150105,"remote_snapshot_timestamp":1726150105,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 14:09:17image-primary5399c5b9a-89f0-4680-96c5-f2cac59115cfup+replayingreplaying, {"bytes_per_second":3616153.6,"bytes_per_snapshot":72323072.0,"last_snapshot_bytes":72323072,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726150099,"remote_snapshot_timestamp":1726150099,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 14:09:17image-primary6f8d03c2d-535f-4038-a91f-c159711e8e6cup+replayingreplaying, {"bytes_per_second":3246489.6,"bytes_per_snapshot":64929792.0,"last_snapshot_bytes":64929792,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726150101,"remote_snapshot_timestamp":1726150101,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 14:09:17image-primary75795f6f3-43c3-4e14-89d9-f4798e240cc6up+replayingreplaying, {"bytes_per_second":3157401.6,"bytes_per_snapshot":63148032.0,"last_snapshot_bytes":63148032,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726150105,"remote_snapshot_timestamp":1726150105,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 14:09:17image-primary8fa74ee32-e225-4343-82da-f93063849113up+replayingreplaying, {"bytes_per_second":3141632.0,"bytes_per_snapshot":62832640.0,"last_snapshot_bytes":62832640,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726150106,"remote_snapshot_timestamp":1726150106,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 14:09:17image-primary95991a19a-9e33-46ee-965b-85f990022e98up+replayingreplaying, {"bytes_per_second":3153920.0,"bytes_per_snapshot":63078400.0,"last_snapshot_bytes":63078400,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726150107,"remote_snapshot_timestamp":1726150107,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 14:09:17' 2024-09-12T14:09:27.265 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T14:09:27.265 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary3'\'']/last_update' 2024-09-12T14:09:27.269 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 14:09:17' 2024-09-12T14:09:27.270 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary3'\'']/state' 2024-09-12T14:09:27.273 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+replaying 2024-09-12T14:09:27.274 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+replaying 2024-09-12T14:09:27.274 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+replaying 2024-09-12T14:09:27.276 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+replaying 2024-09-12T14:09:27.277 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:09:27.277 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:09:27.277 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ remove_image cluster2 mirror image-primary3 2024-09-12T14:09:27.278 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T14:09:27.278 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:09:27.278 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T14:09:27.278 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 snap purge mirror/image-primary3 2024-09-12T14:09:27.338 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 rm mirror/image-primary3 2024-09-12T14:09:27.480 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T14:09:27.480+0000 7f67e4800640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x56055b865860 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-09-12T14:09:27.481 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T14:09:27.480+0000 7f67eca00640 -1 rbd::mirror::ImageReplayer: 0x5605573cbb80 [3/d356f103-adde-4c7a-855e-96419b23abef] operator(): start failed: (85) Interrupted system call should be restarted 2024-09-12T14:09:29.001 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-09-12T14:09:29.008 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_image_removal cluster1 mirror image-primary3 2024-09-12T14:09:29.008 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:09:29.008 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:09:29.008 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T14:09:29.008 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:09:29.008 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 ls mirror 2024-09-12T14:09:29.008 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep -wq image-primary3 2024-09-12T14:09:29.064 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:09:29.064 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rm -rf test-primary3 2024-09-12T14:09:29.067 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T14:09:29.067 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ IMG=image-primary4 2024-09-12T14:09:29.067 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_replaying_status_in_pool_dir cluster1 mirror image-primary4 2024-09-12T14:09:29.068 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:09:29.068 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:09:29.068 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T14:09:29.068 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' snapshot = journal ']' 2024-09-12T14:09:29.068 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary4 up+replaying 2024-09-12T14:09:29.068 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:09:29.068 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:09:29.068 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T14:09:29.068 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T14:09:29.068 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:09:29.068 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:09:29.068 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:09:29.068 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T14:09:30.069 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary4 up+replaying '' '' 2024-09-12T14:09:30.070 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:09:30.070 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:09:30.070 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T14:09:30.070 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T14:09:30.070 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:09:30.070 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:09:30.070 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:09:30.071 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:09:30.071 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary4 2024-09-12T14:09:30.183 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary4: 2024-09-12T14:09:30.183 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: a6cc92cf-b177-4aea-95c5-b61892fd9062 2024-09-12T14:09:30.183 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+replaying 2024-09-12T14:09:30.184 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: replaying, {"bytes_per_second":3248332.8,"bytes_per_snapshot":64966656.0,"last_snapshot_bytes":64966656,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726150105,"remote_snapshot_timestamp":1726150105,"replay_state":"idle"} 2024-09-12T14:09:30.184 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T14:09:30.184 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 14:09:17' 2024-09-12T14:09:30.184 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+replaying' 2024-09-12T14:09:30.188 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+replaying 2024-09-12T14:09:30.189 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T14:09:30.189 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: replaying, {"bytes_per_second":3248332.8,"bytes_per_snapshot":64966656.0,"last_snapshot_bytes":64966656,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726150105,"remote_snapshot_timestamp":1726150105,"replay_state":"idle"} 2024-09-12T14:09:30.189 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T14:09:30.189 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+replaying 2024-09-12T14:09:30.190 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T14:09:30.192 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+replaying 2024-09-12T14:09:30.192 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T14:09:30.194 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T14:09:30.195 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T14:09:30.196 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T14:09:30.200 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 14:09:17' 2024-09-12T14:09:30.200 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary4 up+replaying '2024-09-12 14:09:17' 2024-09-12T14:09:30.200 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:09:30.200 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:09:30.200 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T14:09:30.200 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T14:09:30.200 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 14:09:17' 2024-09-12T14:09:30.200 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:09:30.201 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:09:30.201 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-09-12T14:09:30.306 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING741934232mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary10b0e88823-56d3-40a7-88c0-8d4c22f008eeup+replayingreplaying, {"bytes_per_second":3139379.2,"bytes_per_snapshot":62787584.0,"last_snapshot_bytes":62787584,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726150106,"remote_snapshot_timestamp":1726150106,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 14:09:17image-primary4a6cc92cf-b177-4aea-95c5-b61892fd9062up+replayingreplaying, {"bytes_per_second":3248332.8,"bytes_per_snapshot":64966656.0,"last_snapshot_bytes":64966656,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726150105,"remote_snapshot_timestamp":1726150105,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 14:09:17image-primary5399c5b9a-89f0-4680-96c5-f2cac59115cfup+replayingreplaying, {"bytes_per_second":3616153.6,"bytes_per_snapshot":72323072.0,"last_snapshot_bytes":72323072,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726150099,"remote_snapshot_timestamp":1726150099,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 14:09:17image-primary6f8d03c2d-535f-4038-a91f-c159711e8e6cup+replayingreplaying, {"bytes_per_second":3246489.6,"bytes_per_snapshot":64929792.0,"last_snapshot_bytes":64929792,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726150101,"remote_snapshot_timestamp":1726150101,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 14:09:17image-primary75795f6f3-43c3-4e14-89d9-f4798e240cc6up+replayingreplaying, {"bytes_per_second":3157401.6,"bytes_per_snapshot":63148032.0,"last_snapshot_bytes":63148032,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726150105,"remote_snapshot_timestamp":1726150105,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 14:09:17image-primary8fa74ee32-e225-4343-82da-f93063849113up+replayingreplaying, {"bytes_per_second":3141632.0,"bytes_per_snapshot":62832640.0,"last_snapshot_bytes":62832640,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726150106,"remote_snapshot_timestamp":1726150106,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 14:09:17image-primary95991a19a-9e33-46ee-965b-85f990022e98up+replayingreplaying, {"bytes_per_second":3153920.0,"bytes_per_snapshot":63078400.0,"last_snapshot_bytes":63078400,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726150107,"remote_snapshot_timestamp":1726150107,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 14:09:17' 2024-09-12T14:09:30.306 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T14:09:30.306 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary4'\'']/last_update' 2024-09-12T14:09:30.311 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 14:09:17' 2024-09-12T14:09:30.311 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary4'\'']/state' 2024-09-12T14:09:30.315 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+replaying 2024-09-12T14:09:30.315 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+replaying 2024-09-12T14:09:30.316 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+replaying 2024-09-12T14:09:30.318 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+replaying 2024-09-12T14:09:30.319 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:09:30.319 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:09:30.319 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ remove_image cluster2 mirror image-primary4 2024-09-12T14:09:30.319 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T14:09:30.319 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:09:30.319 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T14:09:30.319 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 snap purge mirror/image-primary4 2024-09-12T14:09:30.380 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 rm mirror/image-primary4 2024-09-12T14:09:30.550 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T14:09:30.550+0000 7f67e4800640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x560556e1a340 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-09-12T14:09:30.551 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T14:09:30.550+0000 7f67eca00640 -1 rbd::mirror::ImageReplayer: 0x56055c199180 [3/a6cc92cf-b177-4aea-95c5-b61892fd9062] operator(): start failed: (85) Interrupted system call should be restarted 2024-09-12T14:09:32.621 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-09-12T14:09:32.627 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_image_removal cluster1 mirror image-primary4 2024-09-12T14:09:32.627 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:09:32.627 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:09:32.628 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T14:09:32.628 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:09:32.628 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 ls mirror 2024-09-12T14:09:32.628 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep -wq image-primary4 2024-09-12T14:09:32.677 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:09:32.677 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rm -rf test-primary4 2024-09-12T14:09:32.680 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T14:09:32.680 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ IMG=image-primary5 2024-09-12T14:09:32.680 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_replaying_status_in_pool_dir cluster1 mirror image-primary5 2024-09-12T14:09:32.680 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:09:32.680 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:09:32.680 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T14:09:32.680 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' snapshot = journal ']' 2024-09-12T14:09:32.680 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary5 up+replaying 2024-09-12T14:09:32.680 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:09:32.680 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:09:32.681 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T14:09:32.681 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T14:09:32.681 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:09:32.681 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:09:32.681 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:09:32.681 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T14:09:33.682 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary5 up+replaying '' '' 2024-09-12T14:09:33.682 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:09:33.683 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:09:33.683 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T14:09:33.683 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T14:09:33.683 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:09:33.683 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:09:33.683 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:09:33.683 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:09:33.684 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary5 2024-09-12T14:09:33.763 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary5: 2024-09-12T14:09:33.763 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 399c5b9a-89f0-4680-96c5-f2cac59115cf 2024-09-12T14:09:33.763 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+replaying 2024-09-12T14:09:33.763 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: replaying, {"bytes_per_second":3616153.6,"bytes_per_snapshot":72323072.0,"last_snapshot_bytes":72323072,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726150099,"remote_snapshot_timestamp":1726150099,"replay_state":"idle"} 2024-09-12T14:09:33.763 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T14:09:33.763 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 14:09:17' 2024-09-12T14:09:33.763 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+replaying' 2024-09-12T14:09:33.765 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+replaying 2024-09-12T14:09:33.765 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T14:09:33.768 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: replaying, {"bytes_per_second":3616153.6,"bytes_per_snapshot":72323072.0,"last_snapshot_bytes":72323072,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726150099,"remote_snapshot_timestamp":1726150099,"replay_state":"idle"} 2024-09-12T14:09:33.768 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T14:09:33.769 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+replaying 2024-09-12T14:09:33.769 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T14:09:33.771 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+replaying 2024-09-12T14:09:33.771 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T14:09:33.774 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T14:09:33.774 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T14:09:33.775 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T14:09:33.779 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 14:09:17' 2024-09-12T14:09:33.779 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary5 up+replaying '2024-09-12 14:09:17' 2024-09-12T14:09:33.779 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:09:33.779 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:09:33.779 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T14:09:33.779 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T14:09:33.779 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 14:09:17' 2024-09-12T14:09:33.779 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:09:33.780 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:09:33.780 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-09-12T14:09:33.880 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING641934232mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary10b0e88823-56d3-40a7-88c0-8d4c22f008eeup+replayingreplaying, {"bytes_per_second":3139379.2,"bytes_per_snapshot":62787584.0,"last_snapshot_bytes":62787584,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726150106,"remote_snapshot_timestamp":1726150106,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 14:09:17image-primary5399c5b9a-89f0-4680-96c5-f2cac59115cfup+replayingreplaying, {"bytes_per_second":3616153.6,"bytes_per_snapshot":72323072.0,"last_snapshot_bytes":72323072,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726150099,"remote_snapshot_timestamp":1726150099,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 14:09:17image-primary6f8d03c2d-535f-4038-a91f-c159711e8e6cup+replayingreplaying, {"bytes_per_second":3246489.6,"bytes_per_snapshot":64929792.0,"last_snapshot_bytes":64929792,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726150101,"remote_snapshot_timestamp":1726150101,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 14:09:17image-primary75795f6f3-43c3-4e14-89d9-f4798e240cc6up+replayingreplaying, {"bytes_per_second":3157401.6,"bytes_per_snapshot":63148032.0,"last_snapshot_bytes":63148032,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726150105,"remote_snapshot_timestamp":1726150105,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 14:09:17image-primary8fa74ee32-e225-4343-82da-f93063849113up+replayingreplaying, {"bytes_per_second":3141632.0,"bytes_per_snapshot":62832640.0,"last_snapshot_bytes":62832640,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726150106,"remote_snapshot_timestamp":1726150106,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 14:09:17image-primary95991a19a-9e33-46ee-965b-85f990022e98up+replayingreplaying, {"bytes_per_second":3153920.0,"bytes_per_snapshot":63078400.0,"last_snapshot_bytes":63078400,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726150107,"remote_snapshot_timestamp":1726150107,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 14:09:17' 2024-09-12T14:09:33.880 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T14:09:33.881 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary5'\'']/last_update' 2024-09-12T14:09:33.884 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 14:09:17' 2024-09-12T14:09:33.885 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary5'\'']/state' 2024-09-12T14:09:33.889 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+replaying 2024-09-12T14:09:33.889 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+replaying 2024-09-12T14:09:33.890 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+replaying 2024-09-12T14:09:33.892 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+replaying 2024-09-12T14:09:33.892 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:09:33.892 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:09:33.892 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ remove_image cluster2 mirror image-primary5 2024-09-12T14:09:33.893 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T14:09:33.893 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:09:33.893 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T14:09:33.893 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 snap purge mirror/image-primary5 2024-09-12T14:09:33.958 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 rm mirror/image-primary5 2024-09-12T14:09:34.105 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T14:09:34.104+0000 7f67e5200640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x5605557d5860 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-09-12T14:09:34.105 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T14:09:34.104+0000 7f67ed400640 -1 rbd::mirror::ImageReplayer: 0x56055c199680 [3/399c5b9a-89f0-4680-96c5-f2cac59115cf] operator(): start failed: (85) Interrupted system call should be restarted 2024-09-12T14:09:35.628 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-09-12T14:09:35.634 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_image_removal cluster1 mirror image-primary5 2024-09-12T14:09:35.634 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:09:35.634 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:09:35.634 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T14:09:35.634 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:09:35.635 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 ls mirror 2024-09-12T14:09:35.635 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep -wq image-primary5 2024-09-12T14:09:35.684 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:09:35.685 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rm -rf test-primary5 2024-09-12T14:09:35.687 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T14:09:35.687 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ IMG=image-primary6 2024-09-12T14:09:35.687 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_replaying_status_in_pool_dir cluster1 mirror image-primary6 2024-09-12T14:09:35.687 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:09:35.687 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:09:35.687 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T14:09:35.688 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' snapshot = journal ']' 2024-09-12T14:09:35.688 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary6 up+replaying 2024-09-12T14:09:35.688 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:09:35.688 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:09:35.688 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T14:09:35.688 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T14:09:35.688 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:09:35.688 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:09:35.688 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:09:35.688 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T14:09:36.690 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary6 up+replaying '' '' 2024-09-12T14:09:36.690 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:09:36.690 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:09:36.690 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T14:09:36.690 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T14:09:36.690 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:09:36.690 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:09:36.690 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:09:36.691 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:09:36.691 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary6 2024-09-12T14:09:36.801 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary6: 2024-09-12T14:09:36.802 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: f8d03c2d-535f-4038-a91f-c159711e8e6c 2024-09-12T14:09:36.802 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+replaying 2024-09-12T14:09:36.802 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: replaying, {"bytes_per_second":3246489.6,"bytes_per_snapshot":64929792.0,"last_snapshot_bytes":64929792,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726150101,"remote_snapshot_timestamp":1726150101,"replay_state":"idle"} 2024-09-12T14:09:36.802 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T14:09:36.802 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 14:09:17' 2024-09-12T14:09:36.802 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+replaying' 2024-09-12T14:09:36.804 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+replaying 2024-09-12T14:09:36.804 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T14:09:36.807 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: replaying, {"bytes_per_second":3246489.6,"bytes_per_snapshot":64929792.0,"last_snapshot_bytes":64929792,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726150101,"remote_snapshot_timestamp":1726150101,"replay_state":"idle"} 2024-09-12T14:09:36.807 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T14:09:36.808 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+replaying 2024-09-12T14:09:36.808 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T14:09:36.810 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+replaying 2024-09-12T14:09:36.811 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T14:09:36.813 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T14:09:36.814 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T14:09:36.814 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T14:09:36.818 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 14:09:17' 2024-09-12T14:09:36.818 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary6 up+replaying '2024-09-12 14:09:17' 2024-09-12T14:09:36.818 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:09:36.819 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:09:36.819 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T14:09:36.819 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T14:09:36.819 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 14:09:17' 2024-09-12T14:09:36.819 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:09:36.819 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:09:36.819 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-09-12T14:09:36.960 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING541934232mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary10b0e88823-56d3-40a7-88c0-8d4c22f008eeup+replayingreplaying, {"bytes_per_second":3139379.2,"bytes_per_snapshot":62787584.0,"last_snapshot_bytes":62787584,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726150106,"remote_snapshot_timestamp":1726150106,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 14:09:17image-primary6f8d03c2d-535f-4038-a91f-c159711e8e6cup+replayingreplaying, {"bytes_per_second":3246489.6,"bytes_per_snapshot":64929792.0,"last_snapshot_bytes":64929792,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726150101,"remote_snapshot_timestamp":1726150101,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 14:09:17image-primary75795f6f3-43c3-4e14-89d9-f4798e240cc6up+replayingreplaying, {"bytes_per_second":3157401.6,"bytes_per_snapshot":63148032.0,"last_snapshot_bytes":63148032,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726150105,"remote_snapshot_timestamp":1726150105,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 14:09:17image-primary8fa74ee32-e225-4343-82da-f93063849113up+replayingreplaying, {"bytes_per_second":3141632.0,"bytes_per_snapshot":62832640.0,"last_snapshot_bytes":62832640,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726150106,"remote_snapshot_timestamp":1726150106,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 14:09:17image-primary95991a19a-9e33-46ee-965b-85f990022e98up+replayingreplaying, {"bytes_per_second":3153920.0,"bytes_per_snapshot":63078400.0,"last_snapshot_bytes":63078400,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726150107,"remote_snapshot_timestamp":1726150107,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 14:09:17' 2024-09-12T14:09:36.960 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T14:09:36.960 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary6'\'']/last_update' 2024-09-12T14:09:36.964 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 14:09:17' 2024-09-12T14:09:36.965 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary6'\'']/state' 2024-09-12T14:09:36.969 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+replaying 2024-09-12T14:09:36.969 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+replaying 2024-09-12T14:09:36.969 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+replaying 2024-09-12T14:09:36.971 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+replaying 2024-09-12T14:09:36.973 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:09:36.973 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:09:36.973 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ remove_image cluster2 mirror image-primary6 2024-09-12T14:09:36.973 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T14:09:36.973 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:09:36.973 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T14:09:36.973 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 snap purge mirror/image-primary6 2024-09-12T14:09:37.035 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 rm mirror/image-primary6 2024-09-12T14:09:37.265 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T14:09:37.265+0000 7f67e5200640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x5605571a1520 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-09-12T14:09:37.265 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T14:09:37.265+0000 7f67ed400640 -1 rbd::mirror::ImageReplayer: 0x56055c198500 [3/f8d03c2d-535f-4038-a91f-c159711e8e6c] operator(): start failed: (85) Interrupted system call should be restarted 2024-09-12T14:09:38.971 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-09-12T14:09:38.977 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_image_removal cluster1 mirror image-primary6 2024-09-12T14:09:38.977 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:09:38.977 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:09:38.977 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T14:09:38.977 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:09:38.978 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 ls mirror 2024-09-12T14:09:38.978 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep -wq image-primary6 2024-09-12T14:09:39.028 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:09:39.028 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rm -rf test-primary6 2024-09-12T14:09:39.030 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T14:09:39.031 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ IMG=image-primary7 2024-09-12T14:09:39.031 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_replaying_status_in_pool_dir cluster1 mirror image-primary7 2024-09-12T14:09:39.031 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:09:39.031 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:09:39.031 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T14:09:39.031 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' snapshot = journal ']' 2024-09-12T14:09:39.031 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary7 up+replaying 2024-09-12T14:09:39.031 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:09:39.031 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:09:39.031 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T14:09:39.031 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T14:09:39.031 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:09:39.031 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:09:39.032 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:09:39.032 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T14:09:40.033 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary7 up+replaying '' '' 2024-09-12T14:09:40.033 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:09:40.033 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:09:40.033 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T14:09:40.033 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T14:09:40.033 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:09:40.033 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:09:40.033 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:09:40.034 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:09:40.034 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary7 2024-09-12T14:09:40.117 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary7: 2024-09-12T14:09:40.118 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 5795f6f3-43c3-4e14-89d9-f4798e240cc6 2024-09-12T14:09:40.118 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+replaying 2024-09-12T14:09:40.118 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: replaying, {"bytes_per_second":3157401.6,"bytes_per_snapshot":63148032.0,"last_snapshot_bytes":63148032,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726150105,"remote_snapshot_timestamp":1726150105,"replay_state":"idle"} 2024-09-12T14:09:40.118 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T14:09:40.118 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 14:09:17' 2024-09-12T14:09:40.118 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+replaying' 2024-09-12T14:09:40.120 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+replaying 2024-09-12T14:09:40.120 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T14:09:40.123 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: replaying, {"bytes_per_second":3157401.6,"bytes_per_snapshot":63148032.0,"last_snapshot_bytes":63148032,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726150105,"remote_snapshot_timestamp":1726150105,"replay_state":"idle"} 2024-09-12T14:09:40.123 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T14:09:40.124 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+replaying 2024-09-12T14:09:40.124 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T14:09:40.126 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+replaying 2024-09-12T14:09:40.126 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T14:09:40.129 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T14:09:40.130 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T14:09:40.131 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T14:09:40.134 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 14:09:17' 2024-09-12T14:09:40.134 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary7 up+replaying '2024-09-12 14:09:17' 2024-09-12T14:09:40.134 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:09:40.134 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:09:40.134 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T14:09:40.135 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T14:09:40.135 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 14:09:17' 2024-09-12T14:09:40.135 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:09:40.135 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:09:40.135 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-09-12T14:09:40.228 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING441934232mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary10b0e88823-56d3-40a7-88c0-8d4c22f008eeup+replayingreplaying, {"bytes_per_second":3139379.2,"bytes_per_snapshot":62787584.0,"last_snapshot_bytes":62787584,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726150106,"remote_snapshot_timestamp":1726150106,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 14:09:17image-primary75795f6f3-43c3-4e14-89d9-f4798e240cc6up+replayingreplaying, {"bytes_per_second":3157401.6,"bytes_per_snapshot":63148032.0,"last_snapshot_bytes":63148032,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726150105,"remote_snapshot_timestamp":1726150105,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 14:09:17image-primary8fa74ee32-e225-4343-82da-f93063849113up+replayingreplaying, {"bytes_per_second":3141632.0,"bytes_per_snapshot":62832640.0,"last_snapshot_bytes":62832640,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726150106,"remote_snapshot_timestamp":1726150106,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 14:09:17image-primary95991a19a-9e33-46ee-965b-85f990022e98up+replayingreplaying, {"bytes_per_second":3153920.0,"bytes_per_snapshot":63078400.0,"last_snapshot_bytes":63078400,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726150107,"remote_snapshot_timestamp":1726150107,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 14:09:17' 2024-09-12T14:09:40.229 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T14:09:40.229 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary7'\'']/last_update' 2024-09-12T14:09:40.233 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 14:09:17' 2024-09-12T14:09:40.234 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary7'\'']/state' 2024-09-12T14:09:40.237 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+replaying 2024-09-12T14:09:40.238 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+replaying 2024-09-12T14:09:40.238 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+replaying 2024-09-12T14:09:40.240 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+replaying 2024-09-12T14:09:40.241 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:09:40.241 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:09:40.241 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ remove_image cluster2 mirror image-primary7 2024-09-12T14:09:40.241 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T14:09:40.241 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:09:40.241 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T14:09:40.241 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 snap purge mirror/image-primary7 2024-09-12T14:09:40.302 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 rm mirror/image-primary7 2024-09-12T14:09:40.470 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T14:09:40.470+0000 7f67e4800640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x56055bbfc340 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-09-12T14:09:40.470 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T14:09:40.470+0000 7f67eca00640 -1 rbd::mirror::ImageReplayer: 0x5605561ac000 [3/5795f6f3-43c3-4e14-89d9-f4798e240cc6] operator(): start failed: (85) Interrupted system call should be restarted 2024-09-12T14:09:41.641 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-09-12T14:09:41.648 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_image_removal cluster1 mirror image-primary7 2024-09-12T14:09:41.648 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:09:41.648 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:09:41.648 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T14:09:41.648 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:09:41.648 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 ls mirror 2024-09-12T14:09:41.649 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep -wq image-primary7 2024-09-12T14:09:41.698 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:09:41.698 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rm -rf test-primary7 2024-09-12T14:09:41.701 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T14:09:41.701 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ IMG=image-primary8 2024-09-12T14:09:41.701 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_replaying_status_in_pool_dir cluster1 mirror image-primary8 2024-09-12T14:09:41.701 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:09:41.701 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:09:41.701 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T14:09:41.701 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' snapshot = journal ']' 2024-09-12T14:09:41.701 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary8 up+replaying 2024-09-12T14:09:41.702 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:09:41.702 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:09:41.702 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T14:09:41.702 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T14:09:41.702 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:09:41.702 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:09:41.702 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:09:41.702 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T14:09:42.703 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary8 up+replaying '' '' 2024-09-12T14:09:42.705 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:09:42.705 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:09:42.705 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T14:09:42.705 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T14:09:42.705 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:09:42.705 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:09:42.705 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:09:42.706 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:09:42.706 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary8 2024-09-12T14:09:42.782 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary8: 2024-09-12T14:09:42.782 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: fa74ee32-e225-4343-82da-f93063849113 2024-09-12T14:09:42.782 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+replaying 2024-09-12T14:09:42.782 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: replaying, {"bytes_per_second":3141632.0,"bytes_per_snapshot":62832640.0,"last_snapshot_bytes":62832640,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726150106,"remote_snapshot_timestamp":1726150106,"replay_state":"idle"} 2024-09-12T14:09:42.782 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T14:09:42.782 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 14:09:17' 2024-09-12T14:09:42.782 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+replaying' 2024-09-12T14:09:42.784 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+replaying 2024-09-12T14:09:42.785 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T14:09:42.787 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: replaying, {"bytes_per_second":3141632.0,"bytes_per_snapshot":62832640.0,"last_snapshot_bytes":62832640,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726150106,"remote_snapshot_timestamp":1726150106,"replay_state":"idle"} 2024-09-12T14:09:42.788 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T14:09:42.789 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+replaying 2024-09-12T14:09:42.789 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T14:09:42.790 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+replaying 2024-09-12T14:09:42.791 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T14:09:42.794 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T14:09:42.794 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T14:09:42.795 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T14:09:42.798 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 14:09:17' 2024-09-12T14:09:42.798 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary8 up+replaying '2024-09-12 14:09:17' 2024-09-12T14:09:42.799 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:09:42.799 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:09:42.799 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T14:09:42.799 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T14:09:42.799 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 14:09:17' 2024-09-12T14:09:42.799 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:09:42.799 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:09:42.799 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-09-12T14:09:42.889 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING341934232mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary10b0e88823-56d3-40a7-88c0-8d4c22f008eeup+replayingreplaying, {"bytes_per_second":3139379.2,"bytes_per_snapshot":62787584.0,"last_snapshot_bytes":62787584,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726150106,"remote_snapshot_timestamp":1726150106,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 14:09:17image-primary8fa74ee32-e225-4343-82da-f93063849113up+replayingreplaying, {"bytes_per_second":3141632.0,"bytes_per_snapshot":62832640.0,"last_snapshot_bytes":62832640,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726150106,"remote_snapshot_timestamp":1726150106,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 14:09:17image-primary95991a19a-9e33-46ee-965b-85f990022e98up+replayingreplaying, {"bytes_per_second":3153920.0,"bytes_per_snapshot":63078400.0,"last_snapshot_bytes":63078400,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726150107,"remote_snapshot_timestamp":1726150107,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 14:09:17' 2024-09-12T14:09:42.889 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T14:09:42.890 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary8'\'']/last_update' 2024-09-12T14:09:42.894 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 14:09:17' 2024-09-12T14:09:42.895 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary8'\'']/state' 2024-09-12T14:09:42.898 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+replaying 2024-09-12T14:09:42.899 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+replaying 2024-09-12T14:09:42.899 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+replaying 2024-09-12T14:09:42.901 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+replaying 2024-09-12T14:09:42.902 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:09:42.902 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:09:42.902 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ remove_image cluster2 mirror image-primary8 2024-09-12T14:09:42.902 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T14:09:42.902 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:09:42.902 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T14:09:42.902 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 snap purge mirror/image-primary8 2024-09-12T14:09:42.963 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 rm mirror/image-primary8 2024-09-12T14:09:43.113 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T14:09:43.113+0000 7f67e5200640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x56055b7b5040 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-09-12T14:09:43.113 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T14:09:43.113+0000 7f67ed400640 -1 rbd::mirror::ImageReplayer: 0x5605561ac280 [3/fa74ee32-e225-4343-82da-f93063849113] operator(): start failed: (85) Interrupted system call should be restarted 2024-09-12T14:09:44.657 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-09-12T14:09:44.664 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_image_removal cluster1 mirror image-primary8 2024-09-12T14:09:44.664 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:09:44.664 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:09:44.664 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T14:09:44.664 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:09:44.665 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 ls mirror 2024-09-12T14:09:44.665 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep -wq image-primary8 2024-09-12T14:09:44.714 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:09:44.714 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rm -rf test-primary8 2024-09-12T14:09:44.717 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T14:09:44.717 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ IMG=image-primary9 2024-09-12T14:09:44.717 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_replaying_status_in_pool_dir cluster1 mirror image-primary9 2024-09-12T14:09:44.717 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:09:44.717 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:09:44.717 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T14:09:44.717 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' snapshot = journal ']' 2024-09-12T14:09:44.717 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary9 up+replaying 2024-09-12T14:09:44.717 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:09:44.717 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:09:44.717 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T14:09:44.717 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T14:09:44.718 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:09:44.718 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:09:44.718 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:09:44.718 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T14:09:45.719 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary9 up+replaying '' '' 2024-09-12T14:09:45.719 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:09:45.719 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:09:45.719 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T14:09:45.719 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T14:09:45.720 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:09:45.720 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:09:45.720 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:09:45.720 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:09:45.720 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary9 2024-09-12T14:09:45.823 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary9: 2024-09-12T14:09:45.823 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 5991a19a-9e33-46ee-965b-85f990022e98 2024-09-12T14:09:45.823 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+replaying 2024-09-12T14:09:45.823 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: replaying, {"bytes_per_second":3153920.0,"bytes_per_snapshot":63078400.0,"last_snapshot_bytes":63078400,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726150107,"remote_snapshot_timestamp":1726150107,"replay_state":"idle"} 2024-09-12T14:09:45.824 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T14:09:45.824 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 14:09:17' 2024-09-12T14:09:45.824 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+replaying' 2024-09-12T14:09:45.826 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+replaying 2024-09-12T14:09:45.826 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T14:09:45.829 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: replaying, {"bytes_per_second":3153920.0,"bytes_per_snapshot":63078400.0,"last_snapshot_bytes":63078400,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726150107,"remote_snapshot_timestamp":1726150107,"replay_state":"idle"} 2024-09-12T14:09:45.829 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T14:09:45.830 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+replaying 2024-09-12T14:09:45.830 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T14:09:45.832 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+replaying 2024-09-12T14:09:45.832 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T14:09:45.835 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T14:09:45.835 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T14:09:45.836 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T14:09:45.840 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 14:09:17' 2024-09-12T14:09:45.840 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary9 up+replaying '2024-09-12 14:09:17' 2024-09-12T14:09:45.840 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:09:45.840 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:09:45.840 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T14:09:45.840 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T14:09:45.840 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 14:09:17' 2024-09-12T14:09:45.840 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:09:45.841 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:09:45.841 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-09-12T14:09:45.960 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING241934232mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary10b0e88823-56d3-40a7-88c0-8d4c22f008eeup+replayingreplaying, {"bytes_per_second":3139379.2,"bytes_per_snapshot":62787584.0,"last_snapshot_bytes":62787584,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726150106,"remote_snapshot_timestamp":1726150106,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 14:09:17image-primary95991a19a-9e33-46ee-965b-85f990022e98up+replayingreplaying, {"bytes_per_second":3153920.0,"bytes_per_snapshot":63078400.0,"last_snapshot_bytes":63078400,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726150107,"remote_snapshot_timestamp":1726150107,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 14:09:17' 2024-09-12T14:09:45.960 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T14:09:45.960 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary9'\'']/last_update' 2024-09-12T14:09:45.964 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 14:09:17' 2024-09-12T14:09:45.965 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary9'\'']/state' 2024-09-12T14:09:45.969 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+replaying 2024-09-12T14:09:45.969 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+replaying 2024-09-12T14:09:45.970 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+replaying 2024-09-12T14:09:45.972 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+replaying 2024-09-12T14:09:45.973 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:09:45.973 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:09:45.973 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ remove_image cluster2 mirror image-primary9 2024-09-12T14:09:45.973 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T14:09:45.973 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:09:45.973 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T14:09:45.973 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 snap purge mirror/image-primary9 2024-09-12T14:09:46.035 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 rm mirror/image-primary9 2024-09-12T14:09:46.259 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T14:09:46.260+0000 7f67e4800640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x56055bce9a00 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-09-12T14:09:46.260 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T14:09:46.260+0000 7f67eca00640 -1 rbd::mirror::ImageReplayer: 0x56055c198780 [3/5991a19a-9e33-46ee-965b-85f990022e98] operator(): start failed: (85) Interrupted system call should be restarted 2024-09-12T14:09:47.639 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-09-12T14:09:47.644 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_image_removal cluster1 mirror image-primary9 2024-09-12T14:09:47.645 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:09:47.645 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:09:47.645 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T14:09:47.645 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:09:47.645 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 ls mirror 2024-09-12T14:09:47.646 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep -wq image-primary9 2024-09-12T14:09:47.702 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:09:47.702 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rm -rf test-primary9 2024-09-12T14:09:47.705 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T14:09:47.705 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ IMG=image-primary10 2024-09-12T14:09:47.705 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_replaying_status_in_pool_dir cluster1 mirror image-primary10 2024-09-12T14:09:47.705 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:09:47.705 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:09:47.705 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T14:09:47.705 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' snapshot = journal ']' 2024-09-12T14:09:47.705 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary10 up+replaying 2024-09-12T14:09:47.706 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:09:47.706 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:09:47.706 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T14:09:47.706 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T14:09:47.706 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:09:47.706 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:09:47.706 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:09:47.706 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T14:09:48.707 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary10 up+replaying '' '' 2024-09-12T14:09:48.708 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:09:48.708 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:09:48.708 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T14:09:48.708 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T14:09:48.708 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:09:48.708 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:09:48.708 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:09:48.708 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:09:48.708 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary10 2024-09-12T14:09:48.785 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary10: 2024-09-12T14:09:48.786 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: b0e88823-56d3-40a7-88c0-8d4c22f008ee 2024-09-12T14:09:48.786 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+replaying 2024-09-12T14:09:48.786 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: replaying, {"bytes_per_second":0.0,"bytes_per_snapshot":62787584.0,"last_snapshot_bytes":62787584,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726150106,"remote_snapshot_timestamp":1726150106,"replay_state":"idle"} 2024-09-12T14:09:48.786 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T14:09:48.786 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 14:09:47' 2024-09-12T14:09:48.786 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+replaying' 2024-09-12T14:09:48.788 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+replaying 2024-09-12T14:09:48.789 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T14:09:48.791 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: replaying, {"bytes_per_second":0.0,"bytes_per_snapshot":62787584.0,"last_snapshot_bytes":62787584,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726150106,"remote_snapshot_timestamp":1726150106,"replay_state":"idle"} 2024-09-12T14:09:48.792 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T14:09:48.793 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+replaying 2024-09-12T14:09:48.793 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T14:09:48.795 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+replaying 2024-09-12T14:09:48.795 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T14:09:48.798 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T14:09:48.799 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T14:09:48.800 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T14:09:48.804 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 14:09:47' 2024-09-12T14:09:48.804 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary10 up+replaying '2024-09-12 14:09:47' 2024-09-12T14:09:48.804 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:09:48.805 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:09:48.805 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T14:09:48.805 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T14:09:48.805 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 14:09:47' 2024-09-12T14:09:48.805 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:09:48.806 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:09:48.806 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-09-12T14:09:48.891 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING141934232mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary10b0e88823-56d3-40a7-88c0-8d4c22f008eeup+replayingreplaying, {"bytes_per_second":0.0,"bytes_per_snapshot":62787584.0,"last_snapshot_bytes":62787584,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726150106,"remote_snapshot_timestamp":1726150106,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 14:09:47' 2024-09-12T14:09:48.891 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T14:09:48.891 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary10'\'']/last_update' 2024-09-12T14:09:48.896 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 14:09:47' 2024-09-12T14:09:48.897 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary10'\'']/state' 2024-09-12T14:09:48.901 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+replaying 2024-09-12T14:09:48.901 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+replaying 2024-09-12T14:09:48.902 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+replaying 2024-09-12T14:09:48.904 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+replaying 2024-09-12T14:09:48.905 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:09:48.905 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:09:48.905 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ remove_image cluster2 mirror image-primary10 2024-09-12T14:09:48.905 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T14:09:48.906 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:09:48.906 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T14:09:48.906 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 snap purge mirror/image-primary10 2024-09-12T14:09:48.971 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 rm mirror/image-primary10 2024-09-12T14:09:49.112 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T14:09:49.113+0000 7f67e5200640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x5605560f89c0 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-09-12T14:09:49.112 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T14:09:49.113+0000 7f67ed400640 -1 rbd::mirror::ImageReplayer: 0x560556cef180 [3/b0e88823-56d3-40a7-88c0-8d4c22f008ee] operator(): start failed: (85) Interrupted system call should be restarted 2024-09-12T14:09:50.707 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-09-12T14:09:50.713 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_image_removal cluster1 mirror image-primary10 2024-09-12T14:09:50.713 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:09:50.713 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:09:50.713 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T14:09:50.713 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:09:50.714 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 ls mirror 2024-09-12T14:09:50.714 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep -wq image-primary10 2024-09-12T14:09:50.764 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:09:50.764 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rm -rf test-primary10 2024-09-12T14:09:50.767 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..10} 2024-09-12T14:09:50.767 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEVS=() 2024-09-12T14:09:50.767 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ SNAP_PIDS=() 2024-09-12T14:09:50.767 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ COMPARE_PIDS=() 2024-09-12T14:09:50.767 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ WORKLOAD_PIDS=() 2024-09-12T14:09:50.767 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ RET=0 2024-09-12T14:09:50.767 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T14:09:50.767 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ IMG=image-primary1 2024-09-12T14:09:50.768 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ MNTPT=test-primary1 2024-09-12T14:09:50.768 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ create_image_and_enable_mirror cluster1 mirror image-primary1 snapshot 10G 2024-09-12T14:09:50.768 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:09:50.768 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T14:09:50.768 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:09:50.768 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T14:09:50.768 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T14:09:50.768 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T14:09:50.768 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mode=snapshot 2024-09-12T14:09:50.768 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n snapshot ']' 2024-09-12T14:09:50.768 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T14:09:50.768 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ create_image cluster1 mirror image-primary1 10G 2024-09-12T14:09:50.768 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:09:50.769 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T14:09:50.769 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:09:50.769 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T14:09:50.769 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T14:09:50.769 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T14:09:50.769 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local size=128 2024-09-12T14:09:50.769 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n 10G ']' 2024-09-12T14:09:50.770 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ size=10G 2024-09-12T14:09:50.770 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T14:09:50.770 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 create --size 10G --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image-primary1 2024-09-12T14:09:50.841 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ is_pool_mirror_mode_image mirror 2024-09-12T14:09:50.841 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:09:50.841 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' image = image ']' 2024-09-12T14:09:50.841 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:09:50.841 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ enable_mirror cluster1 mirror image-primary1 snapshot 2024-09-12T14:09:50.841 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:09:50.842 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:09:50.842 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T14:09:50.842 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mode=snapshot 2024-09-12T14:09:50.842 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 mirror image enable mirror/image-primary1 snapshot 2024-09-12T14:09:51.552 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Mirroring enabled 2024-09-12T14:09:51.568 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 info mirror/image-primary1 2024-09-12T14:09:51.623 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:rbd image 'image-primary1': 2024-09-12T14:09:51.623 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: size 10 GiB in 2560 objects 2024-09-12T14:09:51.623 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: order 22 (4 MiB objects) 2024-09-12T14:09:51.623 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: snapshot_count: 1 2024-09-12T14:09:51.624 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: id: 3d0388bf8a27 2024-09-12T14:09:51.624 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: block_name_prefix: rbd_data.3d0388bf8a27 2024-09-12T14:09:51.624 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: format: 2 2024-09-12T14:09:51.624 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: features: layering, exclusive-lock, object-map, fast-diff 2024-09-12T14:09:51.624 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: op_features: 2024-09-12T14:09:51.624 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: flags: 2024-09-12T14:09:51.624 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: create_timestamp: Thu Sep 12 14:09:50 2024 2024-09-12T14:09:51.624 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: access_timestamp: Thu Sep 12 14:09:50 2024 2024-09-12T14:09:51.624 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: modify_timestamp: Thu Sep 12 14:09:50 2024 2024-09-12T14:09:51.624 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring state: enabled 2024-09-12T14:09:51.624 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring mode: snapshot 2024-09-12T14:09:51.624 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring global id: 9b69126b-93e7-41c5-b8d9-d3784d684fd8 2024-09-12T14:09:51.624 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring primary: true 2024-09-12T14:09:51.629 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \n\b\d ]] 2024-09-12T14:09:51.629 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \k\r\b\d ]] 2024-09-12T14:09:51.630 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo rbd --cluster cluster1 device map -t krbd mirror/image-primary1 2024-09-12T14:09:51.766 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEV=/dev/rbd0 2024-09-12T14:09:51.766 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEVS+=($DEV) 2024-09-12T14:09:51.766 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo mkfs.ext4 /dev/rbd0 2024-09-12T14:09:51.814 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:mke2fs 1.46.5 (30-Dec-2021) 2024-09-12T14:09:51.828 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Discarding device blocks: 0/2621440 done 2024-09-12T14:09:51.828 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Creating filesystem with 2621440 4k blocks and 655360 inodes 2024-09-12T14:09:51.828 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Filesystem UUID: 4efa6082-d0c6-43ca-ae0e-2c76794e2dfe 2024-09-12T14:09:51.828 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Superblock backups stored on blocks: 2024-09-12T14:09:51.828 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 2024-09-12T14:09:51.828 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 2024-09-12T14:09:51.829 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Allocating group tables: 0/80 done 2024-09-12T14:09:51.829 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Writing inode tables: 0/80 done 2024-09-12T14:09:51.833 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Creating journal (16384 blocks): done 2024-09-12T14:09:51.896 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Writing superblocks and filesystem accounting information: 0/80 done 2024-09-12T14:09:51.896 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 2024-09-12T14:09:51.900 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mkdir test-primary1 2024-09-12T14:09:51.902 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo mount /dev/rbd0 test-primary1 2024-09-12T14:09:51.959 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ whoami 2024-09-12T14:09:51.967 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo chown ubuntu test-primary1 2024-09-12T14:09:52.001 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ SNAP_PIDS+=($!) 2024-09-12T14:09:52.001 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ take_mirror_snapshots cluster1 mirror image-primary1 2024-09-12T14:09:52.001 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:09:52.001 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:09:52.001 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T14:09:52.001 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:09:52.001 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T14:09:52.001 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:09:52.001 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:09:52.001 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T14:09:52.001 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T14:09:52.001 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ WORKLOAD_PIDS+=($!) 2024-09-12T14:09:52.001 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T14:09:52.002 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ IMG=image-primary2 2024-09-12T14:09:52.002 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ MNTPT=test-primary2 2024-09-12T14:09:52.002 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ slow_untar_workload test-primary1 2024-09-12T14:09:52.002 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ create_image_and_enable_mirror cluster1 mirror image-primary2 snapshot 10G 2024-09-12T14:09:52.002 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:09:52.002 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T14:09:52.002 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:09:52.002 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T14:09:52.002 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T14:09:52.002 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T14:09:52.002 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mode=snapshot 2024-09-12T14:09:52.002 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n snapshot ']' 2024-09-12T14:09:52.002 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T14:09:52.003 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ create_image cluster1 mirror image-primary2 10G 2024-09-12T14:09:52.003 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mountpt=test-primary1 2024-09-12T14:09:52.003 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ cp linux-5.4.tar.gz test-primary1 2024-09-12T14:09:52.003 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:09:52.003 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T14:09:52.004 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:09:52.004 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T14:09:52.004 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T14:09:52.004 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T14:09:52.004 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local size=128 2024-09-12T14:09:52.004 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n 10G ']' 2024-09-12T14:09:52.004 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ size=10G 2024-09-12T14:09:52.004 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T14:09:52.004 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 create --size 10G --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image-primary2 2024-09-12T14:09:52.071 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ is_pool_mirror_mode_image mirror 2024-09-12T14:09:52.072 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:09:52.072 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' image = image ']' 2024-09-12T14:09:52.072 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:09:52.072 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ enable_mirror cluster1 mirror image-primary2 snapshot 2024-09-12T14:09:52.072 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:09:52.072 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:09:52.072 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T14:09:52.072 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mode=snapshot 2024-09-12T14:09:52.072 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 mirror image enable mirror/image-primary2 snapshot 2024-09-12T14:09:52.286 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local ret=0 2024-09-12T14:09:52.287 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ timeout 5m bash -c 'zcat test-primary1/linux-5.4.tar.gz | pv -L 256K | tar xf - -C test-primary1' 2024-09-12T14:09:52.555 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Mirroring enabled 2024-09-12T14:09:52.562 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2974 2024-09-12T14:09:52.572 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:09:52.572 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 info mirror/image-primary2 2024-09-12T14:09:52.628 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:rbd image 'image-primary2': 2024-09-12T14:09:52.629 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: size 10 GiB in 2560 objects 2024-09-12T14:09:52.629 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: order 22 (4 MiB objects) 2024-09-12T14:09:52.629 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: snapshot_count: 1 2024-09-12T14:09:52.629 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: id: 3d13b8d10159 2024-09-12T14:09:52.629 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: block_name_prefix: rbd_data.3d13b8d10159 2024-09-12T14:09:52.629 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: format: 2 2024-09-12T14:09:52.629 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: features: layering, exclusive-lock, object-map, fast-diff 2024-09-12T14:09:52.629 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: op_features: 2024-09-12T14:09:52.629 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: flags: 2024-09-12T14:09:52.629 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: create_timestamp: Thu Sep 12 14:09:52 2024 2024-09-12T14:09:52.629 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: access_timestamp: Thu Sep 12 14:09:52 2024 2024-09-12T14:09:52.629 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: modify_timestamp: Thu Sep 12 14:09:52 2024 2024-09-12T14:09:52.630 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring state: enabled 2024-09-12T14:09:52.630 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring mode: snapshot 2024-09-12T14:09:52.630 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring global id: 7d8bfe1f-b066-4c3e-95a4-c9c60426a59c 2024-09-12T14:09:52.630 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring primary: true 2024-09-12T14:09:52.634 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \n\b\d ]] 2024-09-12T14:09:52.634 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \k\r\b\d ]] 2024-09-12T14:09:52.634 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo rbd --cluster cluster1 device map -t krbd mirror/image-primary2 2024-09-12T14:09:52.755 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEV=/dev/rbd1 2024-09-12T14:09:52.755 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEVS+=($DEV) 2024-09-12T14:09:52.756 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo mkfs.ext4 /dev/rbd1 2024-09-12T14:09:52.776 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:mke2fs 1.46.5 (30-Dec-2021) 2024-09-12T14:09:52.793 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Discarding device blocks: 0/2621440 done 2024-09-12T14:09:52.793 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Creating filesystem with 2621440 4k blocks and 655360 inodes 2024-09-12T14:09:52.793 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Filesystem UUID: 44fd6eac-a420-48f2-9550-e13e03ac09f8 2024-09-12T14:09:52.793 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Superblock backups stored on blocks: 2024-09-12T14:09:52.793 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 2024-09-12T14:09:52.793 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 2024-09-12T14:09:52.793 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Allocating group tables: 0/80 done 2024-09-12T14:09:52.793 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Writing inode tables: 0/80 done 2024-09-12T14:09:52.798 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Creating journal (16384 blocks): done 2024-09-12T14:09:52.858 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Writing superblocks and filesystem accounting information: 0/80 done 2024-09-12T14:09:52.858 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 2024-09-12T14:09:52.861 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mkdir test-primary2 2024-09-12T14:09:52.863 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo mount /dev/rbd1 test-primary2 2024-09-12T14:09:52.920 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ whoami 2024-09-12T14:09:52.921 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo chown ubuntu test-primary2 2024-09-12T14:09:52.946 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ SNAP_PIDS+=($!) 2024-09-12T14:09:52.946 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ take_mirror_snapshots cluster1 mirror image-primary2 2024-09-12T14:09:52.946 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:09:52.946 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:09:52.946 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T14:09:52.946 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:09:52.946 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T14:09:52.946 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:09:52.946 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ WORKLOAD_PIDS+=($!) 2024-09-12T14:09:52.946 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:09:52.946 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T14:09:52.946 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T14:09:52.946 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T14:09:52.947 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ IMG=image-primary3 2024-09-12T14:09:52.947 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ MNTPT=test-primary3 2024-09-12T14:09:52.947 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ slow_untar_workload test-primary2 2024-09-12T14:09:52.947 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ create_image_and_enable_mirror cluster1 mirror image-primary3 snapshot 10G 2024-09-12T14:09:52.948 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:09:52.948 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T14:09:52.948 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:09:52.948 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T14:09:52.948 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T14:09:52.948 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T14:09:52.948 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mode=snapshot 2024-09-12T14:09:52.948 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n snapshot ']' 2024-09-12T14:09:52.948 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T14:09:52.949 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ create_image cluster1 mirror image-primary3 10G 2024-09-12T14:09:52.949 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mountpt=test-primary2 2024-09-12T14:09:52.949 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:09:52.949 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T14:09:52.949 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:09:52.949 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T14:09:52.949 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T14:09:52.949 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T14:09:52.949 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local size=128 2024-09-12T14:09:52.949 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ cp linux-5.4.tar.gz test-primary2 2024-09-12T14:09:52.949 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n 10G ']' 2024-09-12T14:09:52.949 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ size=10G 2024-09-12T14:09:52.949 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T14:09:52.949 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 create --size 10G --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image-primary3 2024-09-12T14:09:53.019 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ is_pool_mirror_mode_image mirror 2024-09-12T14:09:53.020 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:09:53.020 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' image = image ']' 2024-09-12T14:09:53.020 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:09:53.020 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ enable_mirror cluster1 mirror image-primary3 snapshot 2024-09-12T14:09:53.020 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:09:53.020 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:09:53.020 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T14:09:53.020 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mode=snapshot 2024-09-12T14:09:53.020 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 mirror image enable mirror/image-primary3 snapshot 2024-09-12T14:09:53.236 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local ret=0 2024-09-12T14:09:53.237 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ timeout 5m bash -c 'zcat test-primary2/linux-5.4.tar.gz | pv -L 256K | tar xf - -C test-primary2' 2024-09-12T14:09:53.567 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2976 2024-09-12T14:09:53.569 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Mirroring enabled 2024-09-12T14:09:53.580 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:09:53.584 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 info mirror/image-primary3 2024-09-12T14:09:53.648 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:rbd image 'image-primary3': 2024-09-12T14:09:53.648 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: size 10 GiB in 2560 objects 2024-09-12T14:09:53.648 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: order 22 (4 MiB objects) 2024-09-12T14:09:53.648 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: snapshot_count: 1 2024-09-12T14:09:53.648 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: id: 3d22a99bcfa8 2024-09-12T14:09:53.648 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: block_name_prefix: rbd_data.3d22a99bcfa8 2024-09-12T14:09:53.648 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: format: 2 2024-09-12T14:09:53.648 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: features: layering, exclusive-lock, object-map, fast-diff 2024-09-12T14:09:53.648 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: op_features: 2024-09-12T14:09:53.648 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: flags: 2024-09-12T14:09:53.648 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: create_timestamp: Thu Sep 12 14:09:53 2024 2024-09-12T14:09:53.648 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: access_timestamp: Thu Sep 12 14:09:53 2024 2024-09-12T14:09:53.648 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: modify_timestamp: Thu Sep 12 14:09:53 2024 2024-09-12T14:09:53.649 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring state: enabled 2024-09-12T14:09:53.649 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring mode: snapshot 2024-09-12T14:09:53.649 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring global id: 4f62e81b-1fb5-4b07-89a7-8a820b4d657d 2024-09-12T14:09:53.649 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring primary: true 2024-09-12T14:09:53.653 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \n\b\d ]] 2024-09-12T14:09:53.653 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \k\r\b\d ]] 2024-09-12T14:09:53.654 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo rbd --cluster cluster1 device map -t krbd mirror/image-primary3 2024-09-12T14:09:53.774 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEV=/dev/rbd2 2024-09-12T14:09:53.774 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEVS+=($DEV) 2024-09-12T14:09:53.774 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo mkfs.ext4 /dev/rbd2 2024-09-12T14:09:53.794 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:mke2fs 1.46.5 (30-Dec-2021) 2024-09-12T14:09:53.810 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Discarding device blocks: 0/2621440 done 2024-09-12T14:09:53.810 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Creating filesystem with 2621440 4k blocks and 655360 inodes 2024-09-12T14:09:53.810 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Filesystem UUID: 0235e241-4973-4fbf-a8ad-3097b1a8b73c 2024-09-12T14:09:53.810 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Superblock backups stored on blocks: 2024-09-12T14:09:53.810 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 2024-09-12T14:09:53.810 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 2024-09-12T14:09:53.810 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Allocating group tables: 0/80 done 2024-09-12T14:09:53.810 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Writing inode tables: 0/80 done 2024-09-12T14:09:53.817 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Creating journal (16384 blocks): done 2024-09-12T14:09:53.881 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Writing superblocks and filesystem accounting information: 0/80 done 2024-09-12T14:09:53.881 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 2024-09-12T14:09:53.883 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mkdir test-primary3 2024-09-12T14:09:53.886 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo mount /dev/rbd2 test-primary3 2024-09-12T14:09:53.946 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ whoami 2024-09-12T14:09:53.948 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo chown ubuntu test-primary3 2024-09-12T14:09:53.971 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ SNAP_PIDS+=($!) 2024-09-12T14:09:53.972 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ take_mirror_snapshots cluster1 mirror image-primary3 2024-09-12T14:09:53.972 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:09:53.972 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:09:53.972 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T14:09:53.972 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:09:53.972 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T14:09:53.972 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:09:53.972 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:09:53.972 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T14:09:53.972 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T14:09:53.972 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ WORKLOAD_PIDS+=($!) 2024-09-12T14:09:53.972 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T14:09:53.973 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ IMG=image-primary4 2024-09-12T14:09:53.974 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ MNTPT=test-primary4 2024-09-12T14:09:53.974 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ create_image_and_enable_mirror cluster1 mirror image-primary4 snapshot 10G 2024-09-12T14:09:53.974 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ slow_untar_workload test-primary3 2024-09-12T14:09:53.974 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:09:53.974 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T14:09:53.974 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:09:53.974 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T14:09:53.975 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T14:09:53.975 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T14:09:53.975 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mode=snapshot 2024-09-12T14:09:53.975 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n snapshot ']' 2024-09-12T14:09:53.975 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T14:09:53.975 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ create_image cluster1 mirror image-primary4 10G 2024-09-12T14:09:53.975 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:09:53.975 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T14:09:53.975 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:09:53.976 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T14:09:53.976 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T14:09:53.976 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T14:09:53.976 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local size=128 2024-09-12T14:09:53.976 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n 10G ']' 2024-09-12T14:09:53.976 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ size=10G 2024-09-12T14:09:53.976 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T14:09:53.976 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 create --size 10G --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image-primary4 2024-09-12T14:09:53.976 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mountpt=test-primary3 2024-09-12T14:09:53.976 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ cp linux-5.4.tar.gz test-primary3 2024-09-12T14:09:54.048 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ is_pool_mirror_mode_image mirror 2024-09-12T14:09:54.048 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:09:54.048 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' image = image ']' 2024-09-12T14:09:54.048 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:09:54.048 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ enable_mirror cluster1 mirror image-primary4 snapshot 2024-09-12T14:09:54.048 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:09:54.048 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:09:54.049 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T14:09:54.049 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mode=snapshot 2024-09-12T14:09:54.049 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 mirror image enable mirror/image-primary4 snapshot 2024-09-12T14:09:54.262 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local ret=0 2024-09-12T14:09:54.262 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ timeout 5m bash -c 'zcat test-primary3/linux-5.4.tar.gz | pv -L 256K | tar xf - -C test-primary3' 2024-09-12T14:09:54.572 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Mirroring enabled 2024-09-12T14:09:54.589 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2978 2024-09-12T14:09:54.601 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:09:54.614 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 info mirror/image-primary4 2024-09-12T14:09:54.687 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:rbd image 'image-primary4': 2024-09-12T14:09:54.687 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: size 10 GiB in 2560 objects 2024-09-12T14:09:54.687 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: order 22 (4 MiB objects) 2024-09-12T14:09:54.687 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: snapshot_count: 1 2024-09-12T14:09:54.687 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: id: 3d3067bac05a 2024-09-12T14:09:54.687 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: block_name_prefix: rbd_data.3d3067bac05a 2024-09-12T14:09:54.687 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: format: 2 2024-09-12T14:09:54.687 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: features: layering, exclusive-lock, object-map, fast-diff 2024-09-12T14:09:54.687 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: op_features: 2024-09-12T14:09:54.687 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: flags: 2024-09-12T14:09:54.687 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: create_timestamp: Thu Sep 12 14:09:54 2024 2024-09-12T14:09:54.688 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: access_timestamp: Thu Sep 12 14:09:54 2024 2024-09-12T14:09:54.688 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: modify_timestamp: Thu Sep 12 14:09:54 2024 2024-09-12T14:09:54.688 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring state: enabled 2024-09-12T14:09:54.688 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring mode: snapshot 2024-09-12T14:09:54.688 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring global id: f1c9b79b-6866-4e42-bd0f-25d91fd7fdf5 2024-09-12T14:09:54.688 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring primary: true 2024-09-12T14:09:54.692 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \n\b\d ]] 2024-09-12T14:09:54.693 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \k\r\b\d ]] 2024-09-12T14:09:54.693 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo rbd --cluster cluster1 device map -t krbd mirror/image-primary4 2024-09-12T14:09:54.871 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEV=/dev/rbd3 2024-09-12T14:09:54.872 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEVS+=($DEV) 2024-09-12T14:09:54.872 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo mkfs.ext4 /dev/rbd3 2024-09-12T14:09:54.902 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:mke2fs 1.46.5 (30-Dec-2021) 2024-09-12T14:09:54.916 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Discarding device blocks: 0/2621440 done 2024-09-12T14:09:54.916 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Creating filesystem with 2621440 4k blocks and 655360 inodes 2024-09-12T14:09:54.916 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Filesystem UUID: 3d1b69bb-a192-42fd-a44a-c9ea2f9aa9ed 2024-09-12T14:09:54.917 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Superblock backups stored on blocks: 2024-09-12T14:09:54.917 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 2024-09-12T14:09:54.917 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 2024-09-12T14:09:54.917 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Allocating group tables: 0/80 done 2024-09-12T14:09:54.917 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Writing inode tables: 0/80 done 2024-09-12T14:09:54.922 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Creating journal (16384 blocks): done 2024-09-12T14:09:55.022 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Writing superblocks and filesystem accounting information: 0/80 done 2024-09-12T14:09:55.022 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 2024-09-12T14:09:55.024 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mkdir test-primary4 2024-09-12T14:09:55.027 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo mount /dev/rbd3 test-primary4 2024-09-12T14:09:55.113 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ whoami 2024-09-12T14:09:55.115 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo chown ubuntu test-primary4 2024-09-12T14:09:55.141 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ SNAP_PIDS+=($!) 2024-09-12T14:09:55.141 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ take_mirror_snapshots cluster1 mirror image-primary4 2024-09-12T14:09:55.141 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:09:55.141 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:09:55.141 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T14:09:55.141 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ WORKLOAD_PIDS+=($!) 2024-09-12T14:09:55.141 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:09:55.141 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T14:09:55.141 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T14:09:55.141 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:09:55.141 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:09:55.142 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ IMG=image-primary5 2024-09-12T14:09:55.142 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T14:09:55.142 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ MNTPT=test-primary5 2024-09-12T14:09:55.142 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ create_image_and_enable_mirror cluster1 mirror image-primary5 snapshot 10G 2024-09-12T14:09:55.142 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T14:09:55.142 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ slow_untar_workload test-primary4 2024-09-12T14:09:55.142 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:09:55.142 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T14:09:55.142 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:09:55.142 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T14:09:55.142 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T14:09:55.142 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T14:09:55.142 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mode=snapshot 2024-09-12T14:09:55.143 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n snapshot ']' 2024-09-12T14:09:55.143 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T14:09:55.143 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ create_image cluster1 mirror image-primary5 10G 2024-09-12T14:09:55.143 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mountpt=test-primary4 2024-09-12T14:09:55.144 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:09:55.144 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T14:09:55.144 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ cp linux-5.4.tar.gz test-primary4 2024-09-12T14:09:55.144 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:09:55.144 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T14:09:55.144 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T14:09:55.144 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T14:09:55.144 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local size=128 2024-09-12T14:09:55.144 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n 10G ']' 2024-09-12T14:09:55.144 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ size=10G 2024-09-12T14:09:55.144 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T14:09:55.144 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 create --size 10G --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image-primary5 2024-09-12T14:09:55.236 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ is_pool_mirror_mode_image mirror 2024-09-12T14:09:55.236 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:09:55.236 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' image = image ']' 2024-09-12T14:09:55.236 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:09:55.236 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ enable_mirror cluster1 mirror image-primary5 snapshot 2024-09-12T14:09:55.236 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:09:55.236 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:09:55.237 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T14:09:55.237 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mode=snapshot 2024-09-12T14:09:55.237 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 mirror image enable mirror/image-primary5 snapshot 2024-09-12T14:09:55.441 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local ret=0 2024-09-12T14:09:55.441 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ timeout 5m bash -c 'zcat test-primary4/linux-5.4.tar.gz | pv -L 256K | tar xf - -C test-primary4' 2024-09-12T14:09:55.576 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:09:55.577 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T14:09:55.577 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:09:55.577 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:09:55.577 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T14:09:55.578 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T14:09:55.599 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2980 2024-09-12T14:09:55.609 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Mirroring enabled 2024-09-12T14:09:55.617 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:09:55.638 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 info mirror/image-primary5 2024-09-12T14:09:55.723 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:rbd image 'image-primary5': 2024-09-12T14:09:55.723 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: size 10 GiB in 2560 objects 2024-09-12T14:09:55.723 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: order 22 (4 MiB objects) 2024-09-12T14:09:55.723 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: snapshot_count: 1 2024-09-12T14:09:55.723 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: id: 3d3d73cdeaf0 2024-09-12T14:09:55.724 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: block_name_prefix: rbd_data.3d3d73cdeaf0 2024-09-12T14:09:55.724 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: format: 2 2024-09-12T14:09:55.724 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: features: layering, exclusive-lock, object-map, fast-diff 2024-09-12T14:09:55.724 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: op_features: 2024-09-12T14:09:55.724 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: flags: 2024-09-12T14:09:55.724 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: create_timestamp: Thu Sep 12 14:09:55 2024 2024-09-12T14:09:55.724 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: access_timestamp: Thu Sep 12 14:09:55 2024 2024-09-12T14:09:55.724 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: modify_timestamp: Thu Sep 12 14:09:55 2024 2024-09-12T14:09:55.724 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring state: enabled 2024-09-12T14:09:55.724 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring mode: snapshot 2024-09-12T14:09:55.724 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring global id: d03e84bc-1230-4374-9749-7cc5f3588093 2024-09-12T14:09:55.724 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring primary: true 2024-09-12T14:09:55.728 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \n\b\d ]] 2024-09-12T14:09:55.728 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \k\r\b\d ]] 2024-09-12T14:09:55.729 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo rbd --cluster cluster1 device map -t krbd mirror/image-primary5 2024-09-12T14:09:55.896 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEV=/dev/rbd4 2024-09-12T14:09:55.897 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEVS+=($DEV) 2024-09-12T14:09:55.897 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo mkfs.ext4 /dev/rbd4 2024-09-12T14:09:55.917 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:mke2fs 1.46.5 (30-Dec-2021) 2024-09-12T14:09:55.929 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Discarding device blocks: 0/2621440 done 2024-09-12T14:09:55.930 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Creating filesystem with 2621440 4k blocks and 655360 inodes 2024-09-12T14:09:55.930 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Filesystem UUID: 971577b3-cc6d-4789-a8b1-25c6ba065bee 2024-09-12T14:09:55.930 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Superblock backups stored on blocks: 2024-09-12T14:09:55.930 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 2024-09-12T14:09:55.930 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 2024-09-12T14:09:55.930 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Allocating group tables: 0/80 done 2024-09-12T14:09:55.930 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Writing inode tables: 0/80 done 2024-09-12T14:09:55.934 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Creating journal (16384 blocks): done 2024-09-12T14:09:56.046 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Writing superblocks and filesystem accounting information: 0/80 done 2024-09-12T14:09:56.046 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 2024-09-12T14:09:56.049 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mkdir test-primary5 2024-09-12T14:09:56.051 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo mount /dev/rbd4 test-primary5 2024-09-12T14:09:56.134 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ whoami 2024-09-12T14:09:56.136 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo chown ubuntu test-primary5 2024-09-12T14:09:56.161 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ SNAP_PIDS+=($!) 2024-09-12T14:09:56.161 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ take_mirror_snapshots cluster1 mirror image-primary5 2024-09-12T14:09:56.161 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:09:56.161 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:09:56.161 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T14:09:56.162 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:09:56.162 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T14:09:56.162 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ WORKLOAD_PIDS+=($!) 2024-09-12T14:09:56.162 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:09:56.162 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:09:56.162 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T14:09:56.162 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T14:09:56.162 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T14:09:56.162 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ IMG=image-primary6 2024-09-12T14:09:56.162 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ MNTPT=test-primary6 2024-09-12T14:09:56.162 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ slow_untar_workload test-primary5 2024-09-12T14:09:56.162 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ create_image_and_enable_mirror cluster1 mirror image-primary6 snapshot 10G 2024-09-12T14:09:56.162 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:09:56.163 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T14:09:56.163 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:09:56.163 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T14:09:56.163 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T14:09:56.163 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T14:09:56.163 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mode=snapshot 2024-09-12T14:09:56.163 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n snapshot ']' 2024-09-12T14:09:56.163 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mountpt=test-primary5 2024-09-12T14:09:56.163 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T14:09:56.163 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ create_image cluster1 mirror image-primary6 10G 2024-09-12T14:09:56.163 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ cp linux-5.4.tar.gz test-primary5 2024-09-12T14:09:56.163 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:09:56.163 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T14:09:56.163 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:09:56.164 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T14:09:56.164 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T14:09:56.164 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T14:09:56.164 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local size=128 2024-09-12T14:09:56.164 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n 10G ']' 2024-09-12T14:09:56.164 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ size=10G 2024-09-12T14:09:56.164 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T14:09:56.164 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 create --size 10G --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image-primary6 2024-09-12T14:09:56.262 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ is_pool_mirror_mode_image mirror 2024-09-12T14:09:56.262 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:09:56.262 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' image = image ']' 2024-09-12T14:09:56.262 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:09:56.262 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ enable_mirror cluster1 mirror image-primary6 snapshot 2024-09-12T14:09:56.262 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:09:56.262 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:09:56.262 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T14:09:56.263 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mode=snapshot 2024-09-12T14:09:56.263 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 mirror image enable mirror/image-primary6 snapshot 2024-09-12T14:09:56.467 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local ret=0 2024-09-12T14:09:56.468 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ timeout 5m bash -c 'zcat test-primary5/linux-5.4.tar.gz | pv -L 256K | tar xf - -C test-primary5' 2024-09-12T14:09:56.584 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:09:56.584 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T14:09:56.584 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:09:56.584 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:09:56.584 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T14:09:56.584 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T14:09:56.610 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Mirroring enabled 2024-09-12T14:09:56.623 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2982 2024-09-12T14:09:56.635 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:09:56.636 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2983 2024-09-12T14:09:56.651 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 info mirror/image-primary6 2024-09-12T14:09:56.654 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:09:56.729 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:rbd image 'image-primary6': 2024-09-12T14:09:56.729 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: size 10 GiB in 2560 objects 2024-09-12T14:09:56.729 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: order 22 (4 MiB objects) 2024-09-12T14:09:56.729 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: snapshot_count: 1 2024-09-12T14:09:56.729 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: id: 3d4eda92b24c 2024-09-12T14:09:56.730 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: block_name_prefix: rbd_data.3d4eda92b24c 2024-09-12T14:09:56.730 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: format: 2 2024-09-12T14:09:56.730 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: features: layering, exclusive-lock, object-map, fast-diff 2024-09-12T14:09:56.730 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: op_features: 2024-09-12T14:09:56.730 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: flags: 2024-09-12T14:09:56.730 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: create_timestamp: Thu Sep 12 14:09:56 2024 2024-09-12T14:09:56.730 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: access_timestamp: Thu Sep 12 14:09:56 2024 2024-09-12T14:09:56.730 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: modify_timestamp: Thu Sep 12 14:09:56 2024 2024-09-12T14:09:56.730 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring state: enabled 2024-09-12T14:09:56.730 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring mode: snapshot 2024-09-12T14:09:56.730 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring global id: b2ea2747-3b65-4670-9e03-38ab325e0695 2024-09-12T14:09:56.730 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring primary: true 2024-09-12T14:09:56.734 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \n\b\d ]] 2024-09-12T14:09:56.734 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \k\r\b\d ]] 2024-09-12T14:09:56.735 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo rbd --cluster cluster1 device map -t krbd mirror/image-primary6 2024-09-12T14:09:56.894 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEV=/dev/rbd5 2024-09-12T14:09:56.895 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEVS+=($DEV) 2024-09-12T14:09:56.895 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo mkfs.ext4 /dev/rbd5 2024-09-12T14:09:56.915 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:mke2fs 1.46.5 (30-Dec-2021) 2024-09-12T14:09:56.931 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Discarding device blocks: 0/2621440 done 2024-09-12T14:09:56.931 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Creating filesystem with 2621440 4k blocks and 655360 inodes 2024-09-12T14:09:56.931 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Filesystem UUID: 8ea65bce-f2ed-430a-a11b-76e58cfdb36d 2024-09-12T14:09:56.931 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Superblock backups stored on blocks: 2024-09-12T14:09:56.931 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 2024-09-12T14:09:56.931 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 2024-09-12T14:09:56.931 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Allocating group tables: 0/80 done 2024-09-12T14:09:56.931 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Writing inode tables: 0/80 done 2024-09-12T14:09:56.937 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Creating journal (16384 blocks): done 2024-09-12T14:09:57.022 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Writing superblocks and filesystem accounting information: 0/80 done 2024-09-12T14:09:57.022 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 2024-09-12T14:09:57.025 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mkdir test-primary6 2024-09-12T14:09:57.028 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo mount /dev/rbd5 test-primary6 2024-09-12T14:09:57.153 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ whoami 2024-09-12T14:09:57.155 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo chown ubuntu test-primary6 2024-09-12T14:09:57.180 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ SNAP_PIDS+=($!) 2024-09-12T14:09:57.180 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ take_mirror_snapshots cluster1 mirror image-primary6 2024-09-12T14:09:57.180 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:09:57.180 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:09:57.180 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T14:09:57.181 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:09:57.181 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T14:09:57.181 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:09:57.181 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:09:57.181 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T14:09:57.181 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T14:09:57.181 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ WORKLOAD_PIDS+=($!) 2024-09-12T14:09:57.181 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T14:09:57.181 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ IMG=image-primary7 2024-09-12T14:09:57.181 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ MNTPT=test-primary7 2024-09-12T14:09:57.181 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ create_image_and_enable_mirror cluster1 mirror image-primary7 snapshot 10G 2024-09-12T14:09:57.181 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ slow_untar_workload test-primary6 2024-09-12T14:09:57.181 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:09:57.181 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T14:09:57.182 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:09:57.182 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T14:09:57.182 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T14:09:57.182 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T14:09:57.182 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mode=snapshot 2024-09-12T14:09:57.182 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n snapshot ']' 2024-09-12T14:09:57.182 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T14:09:57.182 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ create_image cluster1 mirror image-primary7 10G 2024-09-12T14:09:57.182 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:09:57.182 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T14:09:57.182 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:09:57.182 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T14:09:57.183 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T14:09:57.183 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T14:09:57.183 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local size=128 2024-09-12T14:09:57.183 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n 10G ']' 2024-09-12T14:09:57.183 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ size=10G 2024-09-12T14:09:57.183 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T14:09:57.183 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mountpt=test-primary6 2024-09-12T14:09:57.183 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 create --size 10G --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image-primary7 2024-09-12T14:09:57.183 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ cp linux-5.4.tar.gz test-primary6 2024-09-12T14:09:57.252 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ is_pool_mirror_mode_image mirror 2024-09-12T14:09:57.252 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:09:57.252 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' image = image ']' 2024-09-12T14:09:57.252 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:09:57.252 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ enable_mirror cluster1 mirror image-primary7 snapshot 2024-09-12T14:09:57.252 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:09:57.252 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:09:57.252 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T14:09:57.252 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mode=snapshot 2024-09-12T14:09:57.253 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 mirror image enable mirror/image-primary7 snapshot 2024-09-12T14:09:57.475 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local ret=0 2024-09-12T14:09:57.476 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ timeout 5m bash -c 'zcat test-primary6/linux-5.4.tar.gz | pv -L 256K | tar xf - -C test-primary6' 2024-09-12T14:09:57.563 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Mirroring enabled 2024-09-12T14:09:57.570 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2985 2024-09-12T14:09:57.573 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2986 2024-09-12T14:09:57.580 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 info mirror/image-primary7 2024-09-12T14:09:57.582 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:09:57.583 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:09:57.605 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:09:57.605 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T14:09:57.605 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:09:57.605 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:09:57.605 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T14:09:57.605 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T14:09:57.642 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:rbd image 'image-primary7': 2024-09-12T14:09:57.642 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: size 10 GiB in 2560 objects 2024-09-12T14:09:57.643 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: order 22 (4 MiB objects) 2024-09-12T14:09:57.643 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: snapshot_count: 1 2024-09-12T14:09:57.643 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: id: 3d5f991a2868 2024-09-12T14:09:57.643 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: block_name_prefix: rbd_data.3d5f991a2868 2024-09-12T14:09:57.643 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: format: 2 2024-09-12T14:09:57.643 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: features: layering, exclusive-lock, object-map, fast-diff 2024-09-12T14:09:57.643 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: op_features: 2024-09-12T14:09:57.643 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: flags: 2024-09-12T14:09:57.643 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: create_timestamp: Thu Sep 12 14:09:57 2024 2024-09-12T14:09:57.643 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: access_timestamp: Thu Sep 12 14:09:57 2024 2024-09-12T14:09:57.643 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: modify_timestamp: Thu Sep 12 14:09:57 2024 2024-09-12T14:09:57.643 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring state: enabled 2024-09-12T14:09:57.643 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring mode: snapshot 2024-09-12T14:09:57.644 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring global id: 639c39b5-6a6c-4696-8c6f-e70477f65d39 2024-09-12T14:09:57.644 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring primary: true 2024-09-12T14:09:57.648 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \n\b\d ]] 2024-09-12T14:09:57.648 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \k\r\b\d ]] 2024-09-12T14:09:57.649 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo rbd --cluster cluster1 device map -t krbd mirror/image-primary7 2024-09-12T14:09:57.770 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEV=/dev/rbd6 2024-09-12T14:09:57.770 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEVS+=($DEV) 2024-09-12T14:09:57.771 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo mkfs.ext4 /dev/rbd6 2024-09-12T14:09:57.791 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:mke2fs 1.46.5 (30-Dec-2021) 2024-09-12T14:09:57.804 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Discarding device blocks: 0/2621440 done 2024-09-12T14:09:57.804 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Creating filesystem with 2621440 4k blocks and 655360 inodes 2024-09-12T14:09:57.804 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Filesystem UUID: 46be1582-57da-47ef-95da-8b619eb4811d 2024-09-12T14:09:57.804 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Superblock backups stored on blocks: 2024-09-12T14:09:57.805 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 2024-09-12T14:09:57.805 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 2024-09-12T14:09:57.805 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Allocating group tables: 0/80 done 2024-09-12T14:09:57.805 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Writing inode tables: 0/80 done 2024-09-12T14:09:57.809 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Creating journal (16384 blocks): done 2024-09-12T14:09:57.866 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Writing superblocks and filesystem accounting information: 0/80 done 2024-09-12T14:09:57.866 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 2024-09-12T14:09:57.869 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mkdir test-primary7 2024-09-12T14:09:57.871 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo mount /dev/rbd6 test-primary7 2024-09-12T14:09:57.930 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ whoami 2024-09-12T14:09:57.932 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo chown ubuntu test-primary7 2024-09-12T14:09:57.958 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ SNAP_PIDS+=($!) 2024-09-12T14:09:57.958 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ take_mirror_snapshots cluster1 mirror image-primary7 2024-09-12T14:09:57.958 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:09:57.958 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:09:57.958 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T14:09:57.958 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ WORKLOAD_PIDS+=($!) 2024-09-12T14:09:57.958 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T14:09:57.958 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:09:57.958 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T14:09:57.958 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ IMG=image-primary8 2024-09-12T14:09:57.958 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:09:57.958 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ MNTPT=test-primary8 2024-09-12T14:09:57.959 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:09:57.959 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T14:09:57.959 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ create_image_and_enable_mirror cluster1 mirror image-primary8 snapshot 10G 2024-09-12T14:09:57.959 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T14:09:57.959 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ slow_untar_workload test-primary7 2024-09-12T14:09:57.959 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:09:57.959 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T14:09:57.959 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:09:57.959 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T14:09:57.959 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T14:09:57.959 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T14:09:57.959 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mode=snapshot 2024-09-12T14:09:57.959 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n snapshot ']' 2024-09-12T14:09:57.959 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T14:09:57.960 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ create_image cluster1 mirror image-primary8 10G 2024-09-12T14:09:57.960 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:09:57.960 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T14:09:57.960 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:09:57.960 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T14:09:57.960 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T14:09:57.960 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T14:09:57.960 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local size=128 2024-09-12T14:09:57.960 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n 10G ']' 2024-09-12T14:09:57.960 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mountpt=test-primary7 2024-09-12T14:09:57.960 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ size=10G 2024-09-12T14:09:57.960 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T14:09:57.960 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 create --size 10G --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image-primary8 2024-09-12T14:09:57.960 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ cp linux-5.4.tar.gz test-primary7 2024-09-12T14:09:58.103 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ is_pool_mirror_mode_image mirror 2024-09-12T14:09:58.103 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:09:58.104 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' image = image ']' 2024-09-12T14:09:58.104 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:09:58.104 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ enable_mirror cluster1 mirror image-primary8 snapshot 2024-09-12T14:09:58.104 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:09:58.104 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:09:58.104 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T14:09:58.104 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mode=snapshot 2024-09-12T14:09:58.104 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 mirror image enable mirror/image-primary8 snapshot 2024-09-12T14:09:58.275 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local ret=0 2024-09-12T14:09:58.276 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ timeout 5m bash -c 'zcat test-primary7/linux-5.4.tar.gz | pv -L 256K | tar xf - -C test-primary7' 2024-09-12T14:09:58.556 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2989 2024-09-12T14:09:58.557 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Mirroring enabled 2024-09-12T14:09:58.561 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2988 2024-09-12T14:09:58.579 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:09:58.588 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:09:58.589 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 info mirror/image-primary8 2024-09-12T14:09:58.623 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:09:58.623 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T14:09:58.623 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:09:58.623 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:09:58.623 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T14:09:58.623 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T14:09:58.705 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:rbd image 'image-primary8': 2024-09-12T14:09:58.705 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: size 10 GiB in 2560 objects 2024-09-12T14:09:58.705 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: order 22 (4 MiB objects) 2024-09-12T14:09:58.705 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: snapshot_count: 1 2024-09-12T14:09:58.705 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: id: 3d71daf685c4 2024-09-12T14:09:58.705 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: block_name_prefix: rbd_data.3d71daf685c4 2024-09-12T14:09:58.705 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: format: 2 2024-09-12T14:09:58.705 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: features: layering, exclusive-lock, object-map, fast-diff 2024-09-12T14:09:58.706 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: op_features: 2024-09-12T14:09:58.706 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: flags: 2024-09-12T14:09:58.706 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: create_timestamp: Thu Sep 12 14:09:58 2024 2024-09-12T14:09:58.706 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: access_timestamp: Thu Sep 12 14:09:58 2024 2024-09-12T14:09:58.706 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: modify_timestamp: Thu Sep 12 14:09:58 2024 2024-09-12T14:09:58.706 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring state: enabled 2024-09-12T14:09:58.706 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring mode: snapshot 2024-09-12T14:09:58.706 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring global id: 8c5b00c9-9b25-4271-b4e1-865fb28703a0 2024-09-12T14:09:58.706 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring primary: true 2024-09-12T14:09:58.723 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \n\b\d ]] 2024-09-12T14:09:58.724 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \k\r\b\d ]] 2024-09-12T14:09:58.726 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo rbd --cluster cluster1 device map -t krbd mirror/image-primary8 2024-09-12T14:09:58.891 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEV=/dev/rbd7 2024-09-12T14:09:58.892 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEVS+=($DEV) 2024-09-12T14:09:58.892 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo mkfs.ext4 /dev/rbd7 2024-09-12T14:09:58.912 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:mke2fs 1.46.5 (30-Dec-2021) 2024-09-12T14:09:58.924 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Discarding device blocks: 0/2621440 done 2024-09-12T14:09:58.924 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Creating filesystem with 2621440 4k blocks and 655360 inodes 2024-09-12T14:09:58.925 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Filesystem UUID: 7642bbd3-bca5-457d-991a-847206e923c7 2024-09-12T14:09:58.925 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Superblock backups stored on blocks: 2024-09-12T14:09:58.925 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 2024-09-12T14:09:58.925 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 2024-09-12T14:09:58.925 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Allocating group tables: 0/80 done 2024-09-12T14:09:58.925 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Writing inode tables: 0/80 done 2024-09-12T14:09:58.929 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Creating journal (16384 blocks): done 2024-09-12T14:09:58.986 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Writing superblocks and filesystem accounting information: 0/80 done 2024-09-12T14:09:58.986 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 2024-09-12T14:09:58.989 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mkdir test-primary8 2024-09-12T14:09:58.991 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo mount /dev/rbd7 test-primary8 2024-09-12T14:09:59.049 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ whoami 2024-09-12T14:09:59.051 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo chown ubuntu test-primary8 2024-09-12T14:09:59.080 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ SNAP_PIDS+=($!) 2024-09-12T14:09:59.080 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ take_mirror_snapshots cluster1 mirror image-primary8 2024-09-12T14:09:59.081 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:09:59.081 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:09:59.081 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T14:09:59.081 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:09:59.081 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T14:09:59.081 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:09:59.081 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:09:59.081 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T14:09:59.081 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T14:09:59.081 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ WORKLOAD_PIDS+=($!) 2024-09-12T14:09:59.081 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T14:09:59.081 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ IMG=image-primary9 2024-09-12T14:09:59.081 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ MNTPT=test-primary9 2024-09-12T14:09:59.082 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ create_image_and_enable_mirror cluster1 mirror image-primary9 snapshot 10G 2024-09-12T14:09:59.082 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ slow_untar_workload test-primary8 2024-09-12T14:09:59.082 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:09:59.083 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T14:09:59.083 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:09:59.083 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T14:09:59.083 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T14:09:59.083 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T14:09:59.083 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mode=snapshot 2024-09-12T14:09:59.083 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n snapshot ']' 2024-09-12T14:09:59.083 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T14:09:59.083 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ create_image cluster1 mirror image-primary9 10G 2024-09-12T14:09:59.083 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:09:59.083 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T14:09:59.083 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:09:59.083 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T14:09:59.084 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T14:09:59.084 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T14:09:59.084 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local size=128 2024-09-12T14:09:59.084 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mountpt=test-primary8 2024-09-12T14:09:59.084 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n 10G ']' 2024-09-12T14:09:59.084 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ size=10G 2024-09-12T14:09:59.084 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T14:09:59.084 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ cp linux-5.4.tar.gz test-primary8 2024-09-12T14:09:59.084 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 create --size 10G --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image-primary9 2024-09-12T14:09:59.174 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ is_pool_mirror_mode_image mirror 2024-09-12T14:09:59.174 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:09:59.174 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' image = image ']' 2024-09-12T14:09:59.174 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:09:59.174 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ enable_mirror cluster1 mirror image-primary9 snapshot 2024-09-12T14:09:59.174 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:09:59.174 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:09:59.174 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T14:09:59.174 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mode=snapshot 2024-09-12T14:09:59.174 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 mirror image enable mirror/image-primary9 snapshot 2024-09-12T14:09:59.406 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local ret=0 2024-09-12T14:09:59.406 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ timeout 5m bash -c 'zcat test-primary8/linux-5.4.tar.gz | pv -L 256K | tar xf - -C test-primary8' 2024-09-12T14:09:59.573 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Mirroring enabled 2024-09-12T14:09:59.585 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2992 2024-09-12T14:09:59.586 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2991 2024-09-12T14:09:59.606 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 info mirror/image-primary9 2024-09-12T14:09:59.619 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:09:59.619 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:09:59.639 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:09:59.639 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T14:09:59.639 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:09:59.640 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:09:59.640 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T14:09:59.640 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T14:09:59.660 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:09:59.660 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T14:09:59.660 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:09:59.660 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:09:59.660 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T14:09:59.660 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T14:09:59.727 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:rbd image 'image-primary9': 2024-09-12T14:09:59.727 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: size 10 GiB in 2560 objects 2024-09-12T14:09:59.727 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: order 22 (4 MiB objects) 2024-09-12T14:09:59.727 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: snapshot_count: 1 2024-09-12T14:09:59.727 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: id: 3d829320e77 2024-09-12T14:09:59.727 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: block_name_prefix: rbd_data.3d829320e77 2024-09-12T14:09:59.727 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: format: 2 2024-09-12T14:09:59.727 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: features: layering, exclusive-lock, object-map, fast-diff 2024-09-12T14:09:59.727 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: op_features: 2024-09-12T14:09:59.727 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: flags: 2024-09-12T14:09:59.728 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: create_timestamp: Thu Sep 12 14:09:59 2024 2024-09-12T14:09:59.728 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: access_timestamp: Thu Sep 12 14:09:59 2024 2024-09-12T14:09:59.728 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: modify_timestamp: Thu Sep 12 14:09:59 2024 2024-09-12T14:09:59.728 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring state: enabled 2024-09-12T14:09:59.728 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring mode: snapshot 2024-09-12T14:09:59.728 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring global id: 8c49b822-b2f9-4ec9-9270-a22fd92ddba3 2024-09-12T14:09:59.728 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring primary: true 2024-09-12T14:09:59.749 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \n\b\d ]] 2024-09-12T14:09:59.749 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \k\r\b\d ]] 2024-09-12T14:09:59.750 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo rbd --cluster cluster1 device map -t krbd mirror/image-primary9 2024-09-12T14:09:59.916 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEV=/dev/rbd8 2024-09-12T14:09:59.916 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEVS+=($DEV) 2024-09-12T14:09:59.916 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo mkfs.ext4 /dev/rbd8 2024-09-12T14:09:59.937 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:mke2fs 1.46.5 (30-Dec-2021) 2024-09-12T14:09:59.951 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Discarding device blocks: 0/2621440 done 2024-09-12T14:09:59.951 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Creating filesystem with 2621440 4k blocks and 655360 inodes 2024-09-12T14:09:59.951 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Filesystem UUID: 4f0e2333-ed01-41e4-9897-42fd83c6d4a1 2024-09-12T14:09:59.951 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Superblock backups stored on blocks: 2024-09-12T14:09:59.951 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 2024-09-12T14:09:59.951 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 2024-09-12T14:09:59.951 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Allocating group tables: 0/80 done 2024-09-12T14:09:59.951 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Writing inode tables: 0/80 done 2024-09-12T14:09:59.956 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Creating journal (16384 blocks): done 2024-09-12T14:10:00.013 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Writing superblocks and filesystem accounting information: 0/80 done 2024-09-12T14:10:00.013 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 2024-09-12T14:10:00.016 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mkdir test-primary9 2024-09-12T14:10:00.018 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo mount /dev/rbd8 test-primary9 2024-09-12T14:10:00.074 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ whoami 2024-09-12T14:10:00.076 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo chown ubuntu test-primary9 2024-09-12T14:10:00.103 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ SNAP_PIDS+=($!) 2024-09-12T14:10:00.103 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ take_mirror_snapshots cluster1 mirror image-primary9 2024-09-12T14:10:00.105 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:10:00.106 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:10:00.106 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T14:10:00.106 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:10:00.106 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T14:10:00.106 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ WORKLOAD_PIDS+=($!) 2024-09-12T14:10:00.106 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:10:00.106 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:10:00.106 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T14:10:00.106 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T14:10:00.106 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T14:10:00.106 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ IMG=image-primary10 2024-09-12T14:10:00.106 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ MNTPT=test-primary10 2024-09-12T14:10:00.106 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ slow_untar_workload test-primary9 2024-09-12T14:10:00.106 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ create_image_and_enable_mirror cluster1 mirror image-primary10 snapshot 10G 2024-09-12T14:10:00.107 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:10:00.107 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T14:10:00.107 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:10:00.107 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T14:10:00.107 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T14:10:00.107 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T14:10:00.107 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mode=snapshot 2024-09-12T14:10:00.107 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n snapshot ']' 2024-09-12T14:10:00.107 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T14:10:00.107 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ create_image cluster1 mirror image-primary10 10G 2024-09-12T14:10:00.107 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mountpt=test-primary9 2024-09-12T14:10:00.107 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:10:00.107 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T14:10:00.107 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ cp linux-5.4.tar.gz test-primary9 2024-09-12T14:10:00.107 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:10:00.108 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T14:10:00.108 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T14:10:00.108 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T14:10:00.108 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local size=128 2024-09-12T14:10:00.108 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n 10G ']' 2024-09-12T14:10:00.108 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ size=10G 2024-09-12T14:10:00.109 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ shift 2024-09-12T14:10:00.109 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 create --size 10G --image-feature layering,exclusive-lock,object-map,fast-diff mirror/image-primary10 2024-09-12T14:10:00.197 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ is_pool_mirror_mode_image mirror 2024-09-12T14:10:00.197 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:10:00.197 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' image = image ']' 2024-09-12T14:10:00.197 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:10:00.197 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ enable_mirror cluster1 mirror image-primary10 snapshot 2024-09-12T14:10:00.197 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:10:00.198 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:10:00.198 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T14:10:00.198 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mode=snapshot 2024-09-12T14:10:00.198 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 mirror image enable mirror/image-primary10 snapshot 2024-09-12T14:10:00.436 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local ret=0 2024-09-12T14:10:00.436 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ timeout 5m bash -c 'zcat test-primary9/linux-5.4.tar.gz | pv -L 256K | tar xf - -C test-primary9' 2024-09-12T14:10:00.585 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:10:00.586 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Mirroring enabled 2024-09-12T14:10:00.586 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T14:10:00.586 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:10:00.586 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:10:00.586 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T14:10:00.586 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T14:10:00.587 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2995 2024-09-12T14:10:00.587 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:10:00.587 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T14:10:00.587 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:10:00.587 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:10:00.587 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T14:10:00.588 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T14:10:00.595 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2996 2024-09-12T14:10:00.601 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2994 2024-09-12T14:10:00.625 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:10:00.633 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 info mirror/image-primary10 2024-09-12T14:10:00.640 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:10:00.665 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:10:00.785 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:rbd image 'image-primary10': 2024-09-12T14:10:00.785 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: size 10 GiB in 2560 objects 2024-09-12T14:10:00.785 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: order 22 (4 MiB objects) 2024-09-12T14:10:00.785 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: snapshot_count: 1 2024-09-12T14:10:00.786 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: id: 3d9612f5f091 2024-09-12T14:10:00.786 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: block_name_prefix: rbd_data.3d9612f5f091 2024-09-12T14:10:00.786 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: format: 2 2024-09-12T14:10:00.786 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: features: layering, exclusive-lock, object-map, fast-diff 2024-09-12T14:10:00.786 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: op_features: 2024-09-12T14:10:00.786 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: flags: 2024-09-12T14:10:00.786 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: create_timestamp: Thu Sep 12 14:10:00 2024 2024-09-12T14:10:00.786 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: access_timestamp: Thu Sep 12 14:10:00 2024 2024-09-12T14:10:00.786 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: modify_timestamp: Thu Sep 12 14:10:00 2024 2024-09-12T14:10:00.786 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring state: enabled 2024-09-12T14:10:00.786 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring mode: snapshot 2024-09-12T14:10:00.786 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring global id: d6c75f78-0d2c-4921-8b6d-d39474248deb 2024-09-12T14:10:00.786 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: mirroring primary: true 2024-09-12T14:10:00.797 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \n\b\d ]] 2024-09-12T14:10:00.797 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \k\r\b\d ]] 2024-09-12T14:10:00.798 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo rbd --cluster cluster1 device map -t krbd mirror/image-primary10 2024-09-12T14:10:00.960 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEV=/dev/rbd9 2024-09-12T14:10:00.960 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ DEVS+=($DEV) 2024-09-12T14:10:00.960 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo mkfs.ext4 /dev/rbd9 2024-09-12T14:10:00.981 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:mke2fs 1.46.5 (30-Dec-2021) 2024-09-12T14:10:00.993 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Discarding device blocks: 0/2621440 done 2024-09-12T14:10:00.994 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Creating filesystem with 2621440 4k blocks and 655360 inodes 2024-09-12T14:10:00.994 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Filesystem UUID: 0d779913-99e7-48a3-b35c-f22c61e57911 2024-09-12T14:10:00.994 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Superblock backups stored on blocks: 2024-09-12T14:10:00.994 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 2024-09-12T14:10:00.994 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 2024-09-12T14:10:00.994 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Allocating group tables: 0/80 done 2024-09-12T14:10:00.994 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Writing inode tables: 0/80 done 2024-09-12T14:10:00.999 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Creating journal (16384 blocks): done 2024-09-12T14:10:01.053 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Writing superblocks and filesystem accounting information: 0/80 done 2024-09-12T14:10:01.053 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: 2024-09-12T14:10:01.056 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mkdir test-primary10 2024-09-12T14:10:01.059 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo mount /dev/rbd9 test-primary10 2024-09-12T14:10:01.232 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ whoami 2024-09-12T14:10:01.234 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo chown ubuntu test-primary10 2024-09-12T14:10:01.259 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ SNAP_PIDS+=($!) 2024-09-12T14:10:01.259 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ take_mirror_snapshots cluster1 mirror image-primary10 2024-09-12T14:10:01.259 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:10:01.259 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:10:01.259 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T14:10:01.259 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:10:01.259 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T14:10:01.259 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:10:01.259 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:10:01.259 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T14:10:01.259 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T14:10:01.259 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ WORKLOAD_PIDS+=($!) 2024-09-12T14:10:01.259 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${SNAP_PIDS[@]} 2024-09-12T14:10:01.260 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 142113 2024-09-12T14:10:01.260 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ slow_untar_workload test-primary10 2024-09-12T14:10:01.260 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mountpt=test-primary10 2024-09-12T14:10:01.260 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ cp linux-5.4.tar.gz test-primary10 2024-09-12T14:10:01.586 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:10:01.586 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T14:10:01.586 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:10:01.586 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:10:01.586 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T14:10:01.586 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T14:10:01.592 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:10:01.592 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T14:10:01.592 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:10:01.592 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:10:01.592 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T14:10:01.592 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T14:10:01.612 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3000 2024-09-12T14:10:01.637 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local ret=0 2024-09-12T14:10:01.637 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ timeout 5m bash -c 'zcat test-primary10/linux-5.4.tar.gz | pv -L 256K | tar xf - -C test-primary10' 2024-09-12T14:10:01.676 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:10:01.684 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2999 2024-09-12T14:10:01.684 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 2998 2024-09-12T14:10:01.730 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:10:01.738 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:10:02.604 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3002 2024-09-12T14:10:02.621 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3001 2024-09-12T14:10:02.622 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:10:02.622 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T14:10:02.623 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:10:02.624 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:10:02.624 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T14:10:02.624 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T14:10:02.625 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:10:02.625 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T14:10:02.625 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:10:02.626 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:10:02.626 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T14:10:02.626 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T14:10:02.634 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:10:02.674 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:10:03.616 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3003 2024-09-12T14:10:03.623 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3004 2024-09-12T14:10:03.633 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:10:03.633 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T14:10:03.633 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:10:03.633 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:10:03.633 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T14:10:03.633 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T14:10:03.649 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:10:03.650 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:10:03.650 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T14:10:03.650 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:10:03.650 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:10:03.650 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T14:10:03.651 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T14:10:03.676 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:10:03.676 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T14:10:03.676 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:10:03.676 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:10:03.677 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T14:10:03.677 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T14:10:03.721 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:10:04.634 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3007 2024-09-12T14:10:04.643 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3005 2024-09-12T14:10:04.666 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3006 2024-09-12T14:10:04.683 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:10:04.684 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T14:10:04.685 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:10:04.685 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:10:04.685 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T14:10:04.685 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T14:10:04.698 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:10:04.704 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:10:04.715 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:10:04.737 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:10:04.737 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T14:10:04.737 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:10:04.737 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:10:04.737 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T14:10:04.737 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T14:10:04.747 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:10:04.747 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T14:10:04.747 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:10:04.747 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:10:04.747 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T14:10:04.747 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T14:10:05.639 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:10:05.639 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T14:10:05.639 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:10:05.639 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:10:05.639 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T14:10:05.639 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T14:10:05.668 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3009 2024-09-12T14:10:05.682 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:10:05.682 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T14:10:05.682 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:10:05.682 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:10:05.682 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T14:10:05.683 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T14:10:05.711 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3010 2024-09-12T14:10:05.732 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:10:05.747 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3008 2024-09-12T14:10:05.764 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:10:05.815 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:10:06.653 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:10:06.653 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T14:10:06.653 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:10:06.653 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:10:06.653 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T14:10:06.653 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T14:10:06.727 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3011 2024-09-12T14:10:06.729 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:10:06.730 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T14:10:06.730 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:10:06.730 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:10:06.730 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T14:10:06.730 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T14:10:06.737 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3012 2024-09-12T14:10:06.786 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:10:06.795 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:10:07.658 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3013 2024-09-12T14:10:07.681 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:10:07.684 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3014 2024-09-12T14:10:07.702 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:10:07.702 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T14:10:07.702 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:10:07.702 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:10:07.702 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T14:10:07.702 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T14:10:07.714 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:10:07.714 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T14:10:07.714 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:10:07.714 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:10:07.714 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T14:10:07.714 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T14:10:07.719 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:10:07.734 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:10:07.734 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T14:10:07.734 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:10:07.734 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:10:07.735 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T14:10:07.735 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T14:10:08.555 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3016 2024-09-12T14:10:08.561 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3015 2024-09-12T14:10:08.565 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3017 2024-09-12T14:10:08.595 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:10:08.595 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:10:08.622 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:10:08.740 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:10:08.741 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T14:10:08.741 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:10:08.741 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:10:08.741 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T14:10:08.741 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T14:10:08.775 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:10:08.775 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T14:10:08.775 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:10:08.775 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:10:08.775 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T14:10:08.776 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T14:10:08.820 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:10:08.820 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T14:10:08.820 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:10:08.821 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:10:08.821 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T14:10:08.821 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T14:10:09.551 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3018 2024-09-12T14:10:09.568 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3019 2024-09-12T14:10:09.572 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3020 2024-09-12T14:10:09.579 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:10:09.583 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:10:09.629 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:10:09.795 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:10:09.795 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T14:10:09.795 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:10:09.795 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:10:09.795 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T14:10:09.795 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T14:10:09.799 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:10:09.799 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T14:10:09.799 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:10:09.799 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:10:09.799 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T14:10:09.799 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T14:10:10.586 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3021 2024-09-12T14:10:10.631 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:10:10.674 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3022 2024-09-12T14:10:10.689 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:10:10.689 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T14:10:10.689 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:10:10.689 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:10:10.689 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T14:10:10.689 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T14:10:10.701 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:10:10.725 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:10:10.725 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T14:10:10.725 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:10:10.725 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:10:10.725 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T14:10:10.725 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T14:10:11.570 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3024 2024-09-12T14:10:11.596 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:10:11.598 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3023 2024-09-12T14:10:11.600 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:10:11.600 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T14:10:11.601 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:10:11.601 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:10:11.601 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T14:10:11.601 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T14:10:11.601 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:10:11.601 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T14:10:11.601 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:10:11.601 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:10:11.601 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T14:10:11.601 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T14:10:11.627 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:10:11.627 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T14:10:11.627 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:10:11.627 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:10:11.627 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T14:10:11.627 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T14:10:11.628 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:10:12.582 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:10:12.583 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T14:10:12.583 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:10:12.583 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:10:12.583 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T14:10:12.583 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T14:10:12.585 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3026 2024-09-12T14:10:12.587 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:10:12.587 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T14:10:12.587 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:10:12.587 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:10:12.587 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T14:10:12.587 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T14:10:12.633 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:10:12.633 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T14:10:12.634 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:10:12.634 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:10:12.634 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T14:10:12.634 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T14:10:12.655 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3025 2024-09-12T14:10:12.674 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:10:12.681 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:10:13.580 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3027 2024-09-12T14:10:13.592 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3028 2024-09-12T14:10:13.629 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:10:13.637 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:10:13.637 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T14:10:13.637 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:10:13.637 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:10:13.637 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T14:10:13.637 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T14:10:13.680 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:10:13.708 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:10:13.720 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T14:10:13.721 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:10:13.721 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:10:13.721 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T14:10:13.721 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T14:10:13.721 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3029 2024-09-12T14:10:13.764 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:10:14.601 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:10:14.601 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T14:10:14.601 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:10:14.601 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:10:14.601 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T14:10:14.601 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T14:10:14.632 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:10:14.632 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T14:10:14.632 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:10:14.632 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:10:14.632 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T14:10:14.632 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T14:10:14.864 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3031 2024-09-12T14:10:14.882 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:10:15.589 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3030 2024-09-12T14:10:15.640 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T14:10:15.640+0000 7f67dc600640 -1 rbd::mirror::image_replayer::snapshot::Replayer: 0x56055bad9000 handle_unlink_peer: failed to unlink local peer from remote image: (22) Invalid argument 2024-09-12T14:10:15.647 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:10:15.678 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:10:15.678 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T14:10:15.678 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:10:15.678 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:10:15.678 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T14:10:15.678 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T14:10:15.695 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3034 2024-09-12T14:10:15.703 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:10:15.704 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T14:10:15.704 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:10:15.704 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:10:15.704 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T14:10:15.704 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T14:10:15.741 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:10:16.545 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3032 2024-09-12T14:10:16.588 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:10:16.633 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:10:16.633 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T14:10:16.633 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:10:16.633 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:10:16.633 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T14:10:16.633 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T14:10:16.674 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3035 2024-09-12T14:10:16.684 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:10:16.684 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T14:10:16.685 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:10:16.685 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:10:16.685 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T14:10:16.685 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T14:10:16.761 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:10:16.768 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:10:16.768 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T14:10:16.768 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:10:16.768 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:10:16.768 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T14:10:16.768 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T14:10:17.562 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3033 2024-09-12T14:10:17.612 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:10:17.689 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3038 2024-09-12T14:10:17.715 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:10:17.885 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:10:17.885 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T14:10:17.885 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:10:17.885 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:10:17.886 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T14:10:17.886 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T14:10:18.534 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3036 2024-09-12T14:10:18.592 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:10:18.617 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3037 2024-09-12T14:10:18.642 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:10:18.652 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3039 2024-09-12T14:10:18.652 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:10:18.652 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T14:10:18.652 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:10:18.652 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:10:18.652 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T14:10:18.652 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T14:10:18.697 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:10:18.745 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:10:18.745 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T14:10:18.745 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:10:18.745 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:10:18.745 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T14:10:18.745 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T14:10:19.593 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:10:19.593 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T14:10:19.593 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:10:19.593 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:10:19.593 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T14:10:19.593 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T14:10:19.744 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3041 2024-09-12T14:10:19.764 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:10:19.764 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T14:10:19.765 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:10:19.765 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:10:19.765 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T14:10:19.765 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T14:10:19.767 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:10:20.579 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3040 2024-09-12T14:10:20.615 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:10:20.616 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T14:10:20.616 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:10:20.616 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:10:20.616 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T14:10:20.616 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T14:10:20.647 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:10:20.675 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3043 2024-09-12T14:10:20.687 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:10:20.718 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:10:20.718 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T14:10:20.718 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:10:20.718 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:10:20.718 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T14:10:20.718 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T14:10:21.567 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3042 2024-09-12T14:10:21.595 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:10:21.599 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T14:10:21.599 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:10:21.599 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:10:21.599 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T14:10:21.599 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T14:10:21.608 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3044 2024-09-12T14:10:21.633 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:10:21.647 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:10:21.647 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T14:10:21.647 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:10:21.647 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:10:21.647 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T14:10:21.647 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T14:10:21.673 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:10:21.681 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3045 2024-09-12T14:10:21.706 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:10:21.706 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T14:10:21.706 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:10:21.706 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:10:21.706 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T14:10:21.706 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T14:10:21.754 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:10:22.682 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3046 2024-09-12T14:10:22.710 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:10:22.734 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3047 2024-09-12T14:10:22.774 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:10:22.774 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T14:10:22.774 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:10:22.774 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:10:22.774 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T14:10:22.774 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T14:10:22.789 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:10:23.656 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:10:23.657 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T14:10:23.657 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:10:23.657 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:10:23.657 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T14:10:23.657 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T14:10:23.694 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:10:23.694 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T14:10:23.694 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:10:23.694 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:10:23.694 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T14:10:23.694 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T14:10:24.593 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3049 2024-09-12T14:10:24.641 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:10:24.641 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T14:10:24.641 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:10:24.641 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:10:24.641 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T14:10:24.641 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T14:10:24.654 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:10:24.679 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:10:24.679 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T14:10:24.679 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:10:24.679 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:10:24.679 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T14:10:24.680 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T14:10:24.759 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:10:24.759 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T14:10:24.760 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:10:24.760 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:10:24.760 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T14:10:24.760 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T14:10:24.779 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3051 2024-09-12T14:10:24.830 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:10:25.601 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3048 2024-09-12T14:10:25.671 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3050 2024-09-12T14:10:25.679 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:10:25.715 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:10:25.715 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T14:10:25.715 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:10:25.715 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:10:25.715 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T14:10:25.716 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T14:10:25.716 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3052 2024-09-12T14:10:25.735 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:10:25.764 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:10:25.792 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:10:25.792 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T14:10:25.792 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:10:25.792 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:10:25.792 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T14:10:25.792 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T14:10:25.798 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3053 2024-09-12T14:10:25.869 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:10:26.658 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3054 2024-09-12T14:10:26.686 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3055 2024-09-12T14:10:26.712 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:10:26.712 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:10:27.651 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3057 2024-09-12T14:10:27.658 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:10:27.658 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T14:10:27.658 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:10:27.658 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:10:27.659 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T14:10:27.659 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T14:10:27.679 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3056 2024-09-12T14:10:27.800 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:10:27.805 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:10:27.834 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:10:27.834 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T14:10:27.834 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:10:27.834 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:10:27.834 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T14:10:27.834 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T14:10:28.683 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:10:28.683 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T14:10:28.683 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:10:28.683 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:10:28.684 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T14:10:28.684 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T14:10:28.740 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:10:28.740 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T14:10:28.740 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:10:28.740 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:10:28.740 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T14:10:28.740 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T14:10:28.766 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:10:28.766 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T14:10:28.767 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:10:28.767 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:10:28.767 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T14:10:28.767 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T14:10:28.786 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3058 2024-09-12T14:10:28.795 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:10:28.872 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:10:28.872 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T14:10:28.872 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:10:28.872 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:10:28.872 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T14:10:28.872 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T14:10:29.601 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3060 2024-09-12T14:10:29.618 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:10:29.715 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:10:29.716 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:10:29.716 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T14:10:29.716 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T14:10:29.716 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:10:29.716 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:10:29.716 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T14:10:29.716 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:10:29.716 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:10:29.716 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T14:10:29.716 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T14:10:29.716 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T14:10:30.546 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3059 2024-09-12T14:10:30.593 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:10:30.643 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T14:10:30.643+0000 7f67dc600640 -1 rbd::mirror::image_replayer::snapshot::Replayer: 0x56055b845000 handle_unlink_peer: failed to unlink local peer from remote image: (22) Invalid argument 2024-09-12T14:10:30.659 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T14:10:30.659+0000 7f67dd000640 -1 rbd::mirror::image_replayer::snapshot::Replayer: 0x56055b898c00 handle_unlink_peer: failed to unlink local peer from remote image: (22) Invalid argument 2024-09-12T14:10:30.803 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:10:30.804 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T14:10:30.804 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:10:30.804 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:10:30.804 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T14:10:30.804 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T14:10:30.807 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:10:30.807 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T14:10:30.807 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:10:30.807 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:10:30.807 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T14:10:30.807 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T14:10:31.553 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3062 2024-09-12T14:10:31.557 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3063 2024-09-12T14:10:31.565 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3061 2024-09-12T14:10:31.605 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3064 2024-09-12T14:10:31.613 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3065 2024-09-12T14:10:31.622 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:10:31.622 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:10:31.634 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:10:31.659 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:10:31.666 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:10:31.798 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:10:31.798 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T14:10:31.798 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:10:31.798 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:10:31.798 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T14:10:31.798 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T14:10:32.622 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3067 2024-09-12T14:10:32.622 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:10:32.622 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T14:10:32.622 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:10:32.622 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:10:32.622 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T14:10:32.623 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T14:10:32.643 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3066 2024-09-12T14:10:32.656 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:10:32.669 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:10:33.595 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:10:33.596 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T14:10:33.596 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:10:33.596 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:10:33.596 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T14:10:33.596 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T14:10:33.624 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3068 2024-09-12T14:10:33.644 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:10:34.629 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:10:34.629 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T14:10:34.629 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:10:34.629 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:10:34.629 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T14:10:34.629 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T14:10:34.630 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:10:34.630 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T14:10:34.631 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:10:34.631 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:10:34.631 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T14:10:34.631 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T14:10:34.632 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3069 2024-09-12T14:10:34.639 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:10:34.639 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T14:10:34.639 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:10:34.639 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:10:34.639 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T14:10:34.640 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T14:10:34.655 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3070 2024-09-12T14:10:34.658 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:10:34.663 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:10:34.663 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T14:10:34.663 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:10:34.663 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:10:34.663 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T14:10:34.663 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T14:10:34.669 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:10:34.669 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T14:10:34.669 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:10:34.669 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:10:34.670 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T14:10:34.670 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T14:10:34.713 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:10:35.601 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3071 2024-09-12T14:10:35.604 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3074 2024-09-12T14:10:35.617 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3072 2024-09-12T14:10:35.619 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:10:35.623 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:10:35.639 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:10:35.660 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:10:35.660 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T14:10:35.660 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:10:35.660 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:10:35.660 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T14:10:35.660 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T14:10:35.673 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:10:35.673 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T14:10:35.673 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:10:35.673 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:10:35.674 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T14:10:35.674 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T14:10:36.649 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:10:36.649 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T14:10:36.649 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:10:36.649 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:10:36.649 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T14:10:36.649 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T14:10:37.577 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3073 2024-09-12T14:10:37.583 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3075 2024-09-12T14:10:37.631 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:10:37.638 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:10:37.661 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:10:37.661 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T14:10:37.661 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:10:37.661 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:10:37.661 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T14:10:37.662 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T14:10:37.717 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:10:37.717 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T14:10:37.717 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:10:37.717 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:10:37.717 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T14:10:37.717 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T14:10:38.548 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3076 2024-09-12T14:10:38.557 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3077 2024-09-12T14:10:38.593 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:10:38.598 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:10:38.622 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:10:38.622 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T14:10:38.622 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:10:38.622 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:10:38.623 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T14:10:38.623 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T14:10:38.626 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:10:38.626 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T14:10:38.626 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:10:38.626 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:10:38.626 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T14:10:38.626 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T14:10:38.641 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:10:38.641 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T14:10:38.642 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:10:38.642 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:10:38.642 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T14:10:38.642 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T14:10:39.600 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3078 2024-09-12T14:10:39.663 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:10:39.685 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3081 2024-09-12T14:10:39.704 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3083 2024-09-12T14:10:39.708 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:10:39.715 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:10:40.575 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3079 2024-09-12T14:10:40.587 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3080 2024-09-12T14:10:40.607 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:10:40.634 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:10:40.635 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T14:10:40.635 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:10:40.635 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:10:40.635 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T14:10:40.635 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T14:10:40.640 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:10:40.640 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T14:10:40.640 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:10:40.640 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:10:40.641 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T14:10:40.641 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T14:10:40.642 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:10:41.577 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3082 2024-09-12T14:10:41.596 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:10:41.596 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T14:10:41.596 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:10:41.596 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:10:41.596 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T14:10:41.596 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T14:10:41.601 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:10:41.601 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T14:10:41.601 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:10:41.601 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:10:41.601 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T14:10:41.601 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T14:10:41.632 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:10:41.690 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3085 2024-09-12T14:10:41.705 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3084 2024-09-12T14:10:41.734 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:10:41.752 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:10:42.666 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:10:42.666 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T14:10:42.666 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:10:42.666 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:10:42.667 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T14:10:42.667 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T14:10:42.694 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3086 2024-09-12T14:10:42.702 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3087 2024-09-12T14:10:42.712 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:10:42.712 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T14:10:42.712 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:10:42.713 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:10:42.713 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T14:10:42.713 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T14:10:42.713 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:10:42.717 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:10:42.717 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T14:10:42.717 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:10:42.717 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:10:42.717 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T14:10:42.718 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T14:10:42.723 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:10:43.610 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:10:43.610 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T14:10:43.610 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:10:43.610 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:10:43.611 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T14:10:43.611 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T14:10:43.645 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:10:43.646 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T14:10:43.646 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:10:43.646 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:10:43.646 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T14:10:43.646 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T14:10:43.686 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3090 2024-09-12T14:10:43.692 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3089 2024-09-12T14:10:43.695 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3088 2024-09-12T14:10:43.712 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:10:43.718 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:10:43.743 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:10:44.635 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3092 2024-09-12T14:10:44.636 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:10:44.636 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T14:10:44.636 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:10:44.636 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:10:44.636 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T14:10:44.636 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T14:10:44.638 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3091 2024-09-12T14:10:44.648 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:10:44.650 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:10:44.737 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:10:44.737 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T14:10:44.737 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:10:44.737 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:10:44.737 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T14:10:44.738 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T14:10:44.754 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:10:44.755 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T14:10:44.755 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:10:44.755 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:10:44.755 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T14:10:44.755 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T14:10:45.644 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3093 2024-09-12T14:10:45.653 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:10:45.714 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:10:45.714 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T14:10:45.714 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:10:45.714 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:10:45.714 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T14:10:45.714 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T14:10:45.726 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:10:45.726 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T14:10:45.726 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:10:45.726 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:10:45.726 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T14:10:45.726 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T14:10:46.716 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:10:46.716 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T14:10:46.716 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:10:46.716 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:10:46.716 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T14:10:46.717 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T14:10:46.722 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:10:46.722 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T14:10:46.722 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:10:46.722 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:10:46.722 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T14:10:46.722 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T14:10:46.745 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:10:46.746 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T14:10:46.746 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:10:46.746 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:10:46.746 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T14:10:46.746 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T14:10:47.613 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3095 2024-09-12T14:10:47.624 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3094 2024-09-12T14:10:47.651 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:10:47.651 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T14:10:47.651 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:10:47.652 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:10:47.652 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T14:10:47.652 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T14:10:47.652 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:10:47.652 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T14:10:47.653 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:10:47.653 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:10:47.653 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T14:10:47.653 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T14:10:47.664 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:10:47.669 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:10:47.741 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3100 2024-09-12T14:10:47.745 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3099 2024-09-12T14:10:47.766 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:10:47.770 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:10:48.560 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3096 2024-09-12T14:10:48.567 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3097 2024-09-12T14:10:48.621 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:10:48.643 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:10:48.644 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3098 2024-09-12T14:10:48.657 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:10:48.657 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T14:10:48.657 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:10:48.657 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:10:48.658 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T14:10:48.658 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T14:10:48.667 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:10:50.615 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3102 2024-09-12T14:10:50.615 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3101 2024-09-12T14:10:50.657 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:10:50.667 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:10:50.667 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T14:10:50.667 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:10:50.667 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:10:50.667 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T14:10:50.667 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T14:10:50.672 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:10:50.672 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T14:10:50.672 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:10:50.672 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:10:50.672 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T14:10:50.672 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T14:10:50.675 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:10:50.769 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:10:50.769 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T14:10:50.769 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:10:50.770 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:10:50.770 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T14:10:50.770 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T14:10:50.773 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:10:50.773 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T14:10:50.773 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:10:50.774 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:10:50.774 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T14:10:50.774 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T14:10:51.624 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:10:51.625 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T14:10:51.625 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:10:51.625 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:10:51.625 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T14:10:51.625 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T14:10:51.644 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3103 2024-09-12T14:10:51.647 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:10:51.647 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T14:10:51.647 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:10:51.647 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:10:51.647 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T14:10:51.647 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T14:10:51.670 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:10:51.670 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T14:10:51.671 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:10:51.671 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:10:51.671 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T14:10:51.671 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T14:10:51.716 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:10:51.806 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3105 2024-09-12T14:10:51.817 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3104 2024-09-12T14:10:51.839 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:10:51.859 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:10:52.798 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3109 2024-09-12T14:10:52.808 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3108 2024-09-12T14:10:52.828 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:10:52.857 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:10:53.580 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3107 2024-09-12T14:10:53.586 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3106 2024-09-12T14:10:53.619 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3110 2024-09-12T14:10:53.635 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:10:53.639 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:10:53.647 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:10:53.660 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:10:53.661 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T14:10:53.661 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:10:53.661 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:10:53.661 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T14:10:53.661 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T14:10:53.678 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:10:53.678 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T14:10:53.678 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:10:53.678 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:10:53.678 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T14:10:53.678 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T14:10:54.656 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3111 2024-09-12T14:10:54.660 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3112 2024-09-12T14:10:54.671 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:10:54.677 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:10:54.719 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:10:54.719 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T14:10:54.719 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:10:54.720 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:10:54.720 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T14:10:54.720 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T14:10:54.841 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:10:54.842 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T14:10:54.842 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:10:54.842 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:10:54.842 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T14:10:54.842 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T14:10:54.862 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:10:54.862 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T14:10:54.862 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:10:54.862 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:10:54.862 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T14:10:54.862 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T14:10:55.671 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3113 2024-09-12T14:10:55.680 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:10:55.831 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:10:55.831 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T14:10:55.831 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:10:55.832 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:10:55.832 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T14:10:55.832 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T14:10:55.859 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:10:55.859 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T14:10:55.859 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:10:55.860 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:10:55.860 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T14:10:55.860 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T14:10:56.644 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:10:56.644 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T14:10:56.644 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:10:56.644 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:10:56.644 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T14:10:56.644 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T14:10:56.644 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:10:56.644 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T14:10:56.644 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:10:56.644 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:10:56.645 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T14:10:56.645 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T14:10:56.655 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:10:56.656 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T14:10:56.656 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:10:56.656 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:10:56.656 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T14:10:56.656 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T14:10:56.717 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3115 2024-09-12T14:10:56.743 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:10:57.669 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3114 2024-09-12T14:10:57.675 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:10:57.675 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T14:10:57.675 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:10:57.675 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:10:57.675 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T14:10:57.676 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T14:10:57.680 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:10:57.680 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T14:10:57.680 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:10:57.680 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:10:57.680 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T14:10:57.681 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T14:10:57.738 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:10:57.743 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3117 2024-09-12T14:10:57.749 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3116 2024-09-12T14:10:57.765 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:10:57.766 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:10:58.577 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3119 2024-09-12T14:10:58.628 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:10:58.684 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:10:58.684 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T14:10:58.684 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:10:58.684 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:10:58.684 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T14:10:58.684 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T14:10:59.752 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:10:59.752 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T14:10:59.752 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:10:59.752 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:10:59.753 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T14:10:59.753 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T14:11:00.665 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3122 2024-09-12T14:11:00.678 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3121 2024-09-12T14:11:00.724 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:11:00.729 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:11:00.744 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:11:00.744 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T14:11:00.744 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:11:00.744 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:11:00.744 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T14:11:00.745 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T14:11:00.768 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:11:00.768 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T14:11:00.768 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:11:00.768 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:11:00.768 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T14:11:00.768 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T14:11:00.769 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:11:00.769 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T14:11:00.769 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:11:00.769 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:11:00.769 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T14:11:00.769 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T14:11:01.633 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:11:01.633 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T14:11:01.633 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:11:01.633 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:11:01.633 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T14:11:01.633 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T14:11:01.668 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3118 2024-09-12T14:11:01.675 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3123 2024-09-12T14:11:01.688 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3120 2024-09-12T14:11:01.722 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:11:01.722 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:11:01.741 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:11:01.789 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3125 2024-09-12T14:11:01.800 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:11:02.663 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3124 2024-09-12T14:11:02.719 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:11:03.540 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3127 2024-09-12T14:11:03.545 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3126 2024-09-12T14:11:03.601 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3128 2024-09-12T14:11:03.601 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:11:03.602 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:11:03.633 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:11:03.727 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:11:03.728 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T14:11:03.728 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:11:03.728 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:11:03.728 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T14:11:03.728 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T14:11:03.732 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:11:03.732 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T14:11:03.732 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:11:03.732 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:11:03.732 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T14:11:03.732 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T14:11:04.699 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3130 2024-09-12T14:11:04.702 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3129 2024-09-12T14:11:04.711 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:11:04.721 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:11:04.724 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:11:04.725 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T14:11:04.725 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:11:04.725 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:11:04.725 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T14:11:04.725 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T14:11:04.725 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:11:04.726 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T14:11:04.726 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:11:04.726 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:11:04.726 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T14:11:04.726 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T14:11:04.745 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:11:04.745 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T14:11:04.745 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:11:04.745 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:11:04.745 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T14:11:04.745 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T14:11:04.802 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:11:04.802 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T14:11:04.803 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:11:04.803 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:11:04.803 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T14:11:04.803 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T14:11:05.715 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3132 2024-09-12T14:11:05.720 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3131 2024-09-12T14:11:05.722 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:11:05.722 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T14:11:05.722 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:11:05.722 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:11:05.722 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T14:11:05.722 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T14:11:05.723 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3133 2024-09-12T14:11:05.734 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:11:05.736 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:11:05.739 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:11:06.604 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:11:06.605 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T14:11:06.605 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:11:06.605 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:11:06.605 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T14:11:06.605 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T14:11:06.606 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:11:06.606 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T14:11:06.606 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:11:06.606 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:11:06.606 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T14:11:06.606 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T14:11:06.637 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:11:06.637 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T14:11:06.637 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:11:06.637 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:11:06.637 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T14:11:06.637 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T14:11:06.738 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3135 2024-09-12T14:11:06.748 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:11:07.681 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3134 2024-09-12T14:11:07.702 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:11:07.713 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:11:07.714 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T14:11:07.714 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:11:07.714 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:11:07.714 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T14:11:07.714 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T14:11:07.723 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:11:07.723 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T14:11:07.723 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:11:07.723 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:11:07.723 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T14:11:07.723 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T14:11:07.732 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3137 2024-09-12T14:11:07.738 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3136 2024-09-12T14:11:07.745 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:11:07.749 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:11:08.579 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3138 2024-09-12T14:11:08.598 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:11:08.737 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:11:08.737 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T14:11:08.737 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:11:08.737 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:11:08.737 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T14:11:08.737 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T14:11:08.738 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:11:08.739 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T14:11:08.739 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:11:08.739 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:11:08.739 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T14:11:08.739 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T14:11:08.742 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:11:08.742 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T14:11:08.742 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:11:08.742 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:11:08.742 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T14:11:08.742 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T14:11:09.607 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3140 2024-09-12T14:11:09.611 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3139 2024-09-12T14:11:09.623 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:11:09.625 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:11:09.750 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:11:09.751 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T14:11:09.751 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:11:09.751 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:11:09.751 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T14:11:09.751 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T14:11:10.705 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:11:10.705 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T14:11:10.705 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:11:10.705 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:11:10.705 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T14:11:10.706 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T14:11:10.748 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:11:10.748 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T14:11:10.748 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:11:10.748 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:11:10.748 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T14:11:10.748 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T14:11:10.752 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:11:10.752 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T14:11:10.752 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:11:10.752 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:11:10.752 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T14:11:10.752 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T14:11:11.564 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3141 2024-09-12T14:11:11.576 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3142 2024-09-12T14:11:11.579 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3143 2024-09-12T14:11:11.601 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3144 2024-09-12T14:11:11.602 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:11:11.603 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T14:11:11.603 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:11:11.603 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:11:11.603 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T14:11:11.603 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T14:11:11.612 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:11:11.620 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:11:11.622 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:11:11.641 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:11:11.700 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3145 2024-09-12T14:11:11.713 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:11:12.602 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3147 2024-09-12T14:11:12.611 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3146 2024-09-12T14:11:12.620 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:11:12.628 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:11:12.628 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T14:11:12.628 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:11:12.628 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:11:12.628 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T14:11:12.628 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T14:11:12.628 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:11:12.629 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T14:11:12.629 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:11:12.629 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:11:12.629 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T14:11:12.629 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T14:11:12.630 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:11:13.580 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3148 2024-09-12T14:11:13.607 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:11:14.592 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3150 2024-09-12T14:11:14.609 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:11:14.614 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:11:14.614 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T14:11:14.614 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:11:14.614 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:11:14.614 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T14:11:14.614 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T14:11:14.623 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:11:14.623 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T14:11:14.623 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:11:14.623 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:11:14.623 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T14:11:14.623 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T14:11:14.625 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:11:14.625 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T14:11:14.625 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:11:14.625 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:11:14.625 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T14:11:14.626 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T14:11:14.644 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:11:14.644 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T14:11:14.644 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:11:14.644 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:11:14.644 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T14:11:14.644 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T14:11:14.716 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:11:14.716 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T14:11:14.716 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:11:14.716 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:11:14.716 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T14:11:14.716 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T14:11:15.574 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3149 2024-09-12T14:11:15.604 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:11:15.624 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:11:15.624 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T14:11:15.624 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:11:15.624 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:11:15.624 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T14:11:15.624 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T14:11:15.629 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3151 2024-09-12T14:11:15.636 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:11:15.636 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T14:11:15.636 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:11:15.636 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:11:15.637 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T14:11:15.637 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T14:11:15.650 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3152 2024-09-12T14:11:15.655 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3153 2024-09-12T14:11:15.656 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:11:15.672 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:11:15.686 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:11:16.611 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:11:16.612 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T14:11:16.612 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:11:16.612 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:11:16.612 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T14:11:16.612 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T14:11:16.641 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3155 2024-09-12T14:11:16.644 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3154 2024-09-12T14:11:16.657 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:11:16.667 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:11:17.602 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3157 2024-09-12T14:11:17.613 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:11:17.614 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T14:11:17.614 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:11:17.614 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:11:17.614 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T14:11:17.614 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T14:11:17.635 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3156 2024-09-12T14:11:17.645 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:11:17.665 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:11:18.568 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3158 2024-09-12T14:11:18.597 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:11:18.607 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:11:18.608 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T14:11:18.608 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:11:18.608 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:11:18.608 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T14:11:18.608 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T14:11:18.659 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:11:18.659 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T14:11:18.659 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:11:18.659 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:11:18.659 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T14:11:18.660 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T14:11:18.674 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:11:18.675 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T14:11:18.675 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:11:18.675 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:11:18.675 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T14:11:18.675 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T14:11:18.689 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:11:18.689 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T14:11:18.689 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:11:18.689 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:11:18.689 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T14:11:18.689 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T14:11:19.627 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T14:11:19.627+0000 7f67dd000640 -1 rbd::mirror::image_replayer::snapshot::Replayer: 0x560556aac800 handle_unlink_peer: failed to unlink local peer from remote image: (22) Invalid argument 2024-09-12T14:11:19.659 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3160 2024-09-12T14:11:19.663 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:11:19.663 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T14:11:19.663 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:11:19.663 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:11:19.664 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T14:11:19.664 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T14:11:19.669 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:11:19.669 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T14:11:19.670 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:11:19.670 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:11:19.670 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T14:11:19.670 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T14:11:19.684 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:11:20.593 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3159 2024-09-12T14:11:20.636 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:11:20.651 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3163 2024-09-12T14:11:20.651 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:11:20.651 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T14:11:20.651 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:11:20.651 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:11:20.651 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T14:11:20.651 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T14:11:20.652 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3161 2024-09-12T14:11:20.669 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:11:20.669 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T14:11:20.669 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:11:20.669 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:11:20.669 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T14:11:20.669 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T14:11:20.692 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:11:20.706 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:11:21.589 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3162 2024-09-12T14:11:21.600 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:11:21.601 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T14:11:21.601 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:11:21.601 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:11:21.601 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T14:11:21.601 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T14:11:21.666 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:11:21.677 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3165 2024-09-12T14:11:21.687 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3164 2024-09-12T14:11:21.709 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:11:21.719 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:11:22.618 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3166 2024-09-12T14:11:22.640 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3167 2024-09-12T14:11:22.642 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:11:22.650 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:11:22.688 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:11:22.688 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T14:11:22.688 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:11:22.688 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:11:22.688 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T14:11:22.688 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T14:11:23.575 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3168 2024-09-12T14:11:23.602 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:11:23.640 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:11:23.640 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T14:11:23.640 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:11:23.640 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:11:23.640 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T14:11:23.640 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T14:11:23.697 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:11:23.697 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T14:11:23.697 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:11:23.697 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:11:23.697 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T14:11:23.697 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T14:11:23.709 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:11:23.709 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T14:11:23.709 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:11:23.709 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:11:23.709 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T14:11:23.709 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T14:11:24.626 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3169 2024-09-12T14:11:24.651 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:11:24.651 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3170 2024-09-12T14:11:24.668 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:11:24.671 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:11:24.671 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T14:11:24.671 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:11:24.671 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:11:24.671 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T14:11:24.671 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T14:11:24.714 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:11:24.714 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T14:11:24.714 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:11:24.714 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:11:24.714 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T14:11:24.714 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T14:11:24.724 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:11:24.724 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T14:11:24.725 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:11:24.725 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:11:24.725 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T14:11:24.725 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T14:11:25.645 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:11:25.645 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T14:11:25.646 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:11:25.646 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:11:25.646 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T14:11:25.646 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T14:11:25.653 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:11:25.653 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T14:11:25.653 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:11:25.653 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:11:25.653 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T14:11:25.653 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T14:11:25.680 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3173 2024-09-12T14:11:25.692 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:11:26.607 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:11:26.607 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T14:11:26.607 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:11:26.607 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:11:26.607 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T14:11:26.607 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T14:11:26.613 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3172 2024-09-12T14:11:26.621 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3171 2024-09-12T14:11:26.704 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:11:26.711 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:11:26.717 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3174 2024-09-12T14:11:26.753 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:11:27.596 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3175 2024-09-12T14:11:27.643 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:11:27.652 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3176 2024-09-12T14:11:27.653 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:11:27.653 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T14:11:27.653 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:11:27.653 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:11:27.654 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T14:11:27.654 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T14:11:27.670 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3177 2024-09-12T14:11:27.672 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:11:27.672 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T14:11:27.672 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:11:27.672 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:11:27.673 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T14:11:27.673 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T14:11:27.693 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:11:27.697 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:11:28.580 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3178 2024-09-12T14:11:28.607 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3180 2024-09-12T14:11:28.611 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:11:28.622 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:11:28.696 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:11:28.696 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T14:11:28.696 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:11:28.696 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:11:28.696 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T14:11:28.696 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T14:11:29.627 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3179 2024-09-12T14:11:29.644 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:11:29.707 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:11:29.707 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T14:11:29.707 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:11:29.708 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:11:29.708 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T14:11:29.708 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T14:11:29.714 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:11:29.714 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T14:11:29.714 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:11:29.714 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:11:29.714 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T14:11:29.714 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T14:11:29.756 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:11:29.756 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T14:11:29.756 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:11:29.756 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:11:29.756 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T14:11:29.756 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T14:11:30.646 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:11:30.647 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T14:11:30.647 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:11:30.647 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:11:30.647 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T14:11:30.647 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T14:11:30.663 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3182 2024-09-12T14:11:30.663 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3183 2024-09-12T14:11:30.679 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:11:30.679 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:11:30.701 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:11:30.701 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T14:11:30.701 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:11:30.701 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:11:30.701 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T14:11:30.701 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T14:11:30.703 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:11:30.703 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T14:11:30.703 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:11:30.703 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:11:30.703 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T14:11:30.703 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T14:11:31.615 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:11:31.615 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T14:11:31.615 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:11:31.615 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:11:31.616 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T14:11:31.616 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T14:11:31.624 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:11:31.625 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T14:11:31.625 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:11:31.625 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:11:31.625 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T14:11:31.625 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T14:11:31.626 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3181 2024-09-12T14:11:31.660 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:11:31.700 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3185 2024-09-12T14:11:31.711 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:11:32.615 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3184 2024-09-12T14:11:32.649 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:11:32.649 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T14:11:32.649 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:11:32.649 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:11:32.649 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T14:11:32.649 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T14:11:32.689 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:11:32.691 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3187 2024-09-12T14:11:32.708 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3186 2024-09-12T14:11:32.732 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3189 2024-09-12T14:11:32.745 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:11:32.749 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:11:32.755 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:11:33.563 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3188 2024-09-12T14:11:33.582 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:11:33.682 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:11:33.682 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T14:11:33.682 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:11:33.683 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:11:33.683 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T14:11:33.683 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T14:11:33.683 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:11:33.683 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T14:11:33.683 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:11:33.683 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:11:33.683 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T14:11:33.683 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T14:11:34.653 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3190 2024-09-12T14:11:34.663 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:11:34.663 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T14:11:34.663 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:11:34.663 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:11:34.663 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T14:11:34.664 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T14:11:34.677 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:11:34.714 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:11:34.714 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T14:11:34.714 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:11:34.715 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:11:34.715 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T14:11:34.715 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T14:11:35.694 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:11:35.694 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T14:11:35.695 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:11:35.695 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:11:35.695 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T14:11:35.695 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T14:11:35.702 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3193 2024-09-12T14:11:35.720 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:11:35.748 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:11:35.748 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T14:11:35.748 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:11:35.748 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:11:35.748 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T14:11:35.748 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T14:11:35.752 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:11:35.752 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T14:11:35.752 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:11:35.752 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:11:35.753 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T14:11:35.753 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T14:11:35.759 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:11:35.759 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T14:11:35.759 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:11:35.759 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:11:35.759 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T14:11:35.759 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T14:11:36.587 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:11:36.587 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T14:11:36.588 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:11:36.588 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:11:36.588 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T14:11:36.588 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T14:11:36.654 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3191 2024-09-12T14:11:36.662 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3192 2024-09-12T14:11:36.698 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:11:36.699 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:11:36.786 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3195 2024-09-12T14:11:36.789 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3196 2024-09-12T14:11:36.809 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:11:36.809 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:11:37.653 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3194 2024-09-12T14:11:37.680 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:11:37.680 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T14:11:37.681 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:11:37.681 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:11:37.681 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T14:11:37.681 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T14:11:37.705 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3198 2024-09-12T14:11:37.715 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:11:37.746 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:11:38.556 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3197 2024-09-12T14:11:38.591 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3199 2024-09-12T14:11:38.595 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:11:38.621 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:11:38.723 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:11:38.723 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T14:11:38.723 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:11:38.723 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:11:38.723 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T14:11:38.723 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T14:11:39.676 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3200 2024-09-12T14:11:39.693 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:11:39.701 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:11:39.701 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T14:11:39.701 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:11:39.701 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:11:39.701 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T14:11:39.701 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T14:11:39.701 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:11:39.701 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T14:11:39.701 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:11:39.701 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:11:39.701 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T14:11:39.702 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T14:11:39.811 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:11:39.812 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:11:39.812 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T14:11:39.812 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T14:11:39.812 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:11:39.812 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:11:39.812 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:11:39.812 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:11:39.812 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T14:11:39.812 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T14:11:39.812 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T14:11:39.812 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T14:11:40.696 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3203 2024-09-12T14:11:40.701 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3202 2024-09-12T14:11:40.713 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:11:40.716 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:11:40.718 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:11:40.718 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T14:11:40.718 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:11:40.718 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:11:40.718 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T14:11:40.718 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T14:11:40.750 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:11:40.750 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T14:11:40.750 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:11:40.750 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:11:40.750 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T14:11:40.751 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T14:11:41.599 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:11:41.600 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T14:11:41.600 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:11:41.600 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:11:41.600 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T14:11:41.600 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T14:11:41.627 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:11:41.627 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T14:11:41.627 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:11:41.627 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:11:41.628 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T14:11:41.628 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T14:11:41.645 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3205 2024-09-12T14:11:41.656 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3201 2024-09-12T14:11:41.684 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:11:41.704 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:11:41.733 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3206 2024-09-12T14:11:41.756 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:11:42.658 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3204 2024-09-12T14:11:42.696 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:11:42.696 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T14:11:42.696 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:11:42.696 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:11:42.696 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T14:11:42.696 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T14:11:42.697 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:11:42.771 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3208 2024-09-12T14:11:42.779 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:11:43.553 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3207 2024-09-12T14:11:43.588 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:11:43.610 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T14:11:43.612+0000 7f67dd000640 -1 rbd::mirror::image_replayer::snapshot::Replayer: 0x56055ba73000 handle_unlink_peer: failed to unlink local peer from remote image: (22) Invalid argument 2024-09-12T14:11:43.716 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:11:43.716 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T14:11:43.716 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:11:43.716 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:11:43.716 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T14:11:43.716 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T14:11:43.720 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:11:43.720 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T14:11:43.720 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:11:43.720 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:11:43.720 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T14:11:43.720 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T14:11:44.554 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3209 2024-09-12T14:11:44.593 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:11:44.603 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T14:11:44.604+0000 7f67dd000640 -1 rbd::mirror::image_replayer::snapshot::Replayer: 0x56055bb76c00 handle_unlink_peer: failed to unlink local peer from remote image: (22) Invalid argument 2024-09-12T14:11:44.687 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:11:44.687 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T14:11:44.687 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:11:44.687 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:11:44.687 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T14:11:44.688 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T14:11:44.707 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:11:44.707 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T14:11:44.707 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:11:44.707 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:11:44.707 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T14:11:44.707 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T14:11:44.758 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:11:44.759 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T14:11:44.759 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:11:44.759 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:11:44.759 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T14:11:44.759 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T14:11:45.552 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3210 2024-09-12T14:11:45.605 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:11:45.631 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T14:11:45.632+0000 7f67dd000640 -1 rbd::mirror::image_replayer::snapshot::Replayer: 0x560557a2c400 handle_unlink_peer: failed to unlink local peer from remote image: (22) Invalid argument 2024-09-12T14:11:45.648 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3214 2024-09-12T14:11:45.664 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:11:45.699 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:11:45.699 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T14:11:45.699 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:11:45.699 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:11:45.699 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T14:11:45.699 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T14:11:45.782 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:11:45.782 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T14:11:45.782 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:11:45.782 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:11:45.782 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T14:11:45.782 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T14:11:46.547 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3212 2024-09-12T14:11:46.553 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3213 2024-09-12T14:11:46.561 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3211 2024-09-12T14:11:46.591 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:11:46.593 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T14:11:46.593 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:11:46.593 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:11:46.593 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T14:11:46.593 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T14:11:46.624 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:11:46.625 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:11:46.629 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:11:46.633 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3215 2024-09-12T14:11:46.668 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:11:46.673 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3216 2024-09-12T14:11:46.690 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:11:47.592 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3217 2024-09-12T14:11:47.596 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:11:47.597 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T14:11:47.597 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:11:47.597 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:11:47.597 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T14:11:47.597 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T14:11:47.613 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:11:47.625 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3218 2024-09-12T14:11:47.659 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:11:48.586 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3219 2024-09-12T14:11:48.605 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:11:48.608 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:11:48.609 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T14:11:48.609 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:11:48.609 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:11:48.609 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T14:11:48.609 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T14:11:48.669 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:11:48.669 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T14:11:48.669 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:11:48.669 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:11:48.669 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T14:11:48.670 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T14:11:49.591 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3220 2024-09-12T14:11:49.611 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:11:49.627 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:11:49.627 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:11:49.627 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T14:11:49.627 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T14:11:49.627 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:11:49.627 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:11:49.627 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:11:49.628 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:11:49.628 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T14:11:49.628 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T14:11:49.628 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T14:11:49.628 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T14:11:49.632 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:11:49.632 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T14:11:49.632 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:11:49.632 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:11:49.632 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T14:11:49.632 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T14:11:49.670 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:11:49.671 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T14:11:49.671 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:11:49.671 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:11:49.671 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T14:11:49.671 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T14:11:49.692 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:11:49.692 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T14:11:49.693 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:11:49.693 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:11:49.693 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T14:11:49.693 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T14:11:50.615 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T14:11:50.615+0000 7f67dd000640 -1 rbd::mirror::image_replayer::snapshot::Replayer: 0x56055c2e1400 handle_unlink_peer: failed to unlink local peer from remote image: (22) Invalid argument 2024-09-12T14:11:50.618 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:11:50.618 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T14:11:50.618 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:11:50.618 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:11:50.618 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T14:11:50.618 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T14:11:50.642 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3223 2024-09-12T14:11:50.642 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3222 2024-09-12T14:11:50.658 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:11:50.658 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:11:50.662 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:11:50.662 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T14:11:50.662 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:11:50.662 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:11:50.662 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T14:11:50.662 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T14:11:51.573 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3221 2024-09-12T14:11:51.611 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:11:51.612 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T14:11:51.612 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:11:51.612 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:11:51.612 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T14:11:51.613 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T14:11:51.634 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3224 2024-09-12T14:11:51.652 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:11:51.654 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3225 2024-09-12T14:11:51.660 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3226 2024-09-12T14:11:51.678 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:11:51.702 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:11:51.705 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:11:52.596 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3228 2024-09-12T14:11:52.614 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:11:52.615 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T14:11:52.615 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:11:52.615 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:11:52.615 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T14:11:52.615 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T14:11:52.619 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3227 2024-09-12T14:11:52.634 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:11:52.646 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:11:53.582 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3229 2024-09-12T14:11:53.604 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:11:53.660 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:11:53.661 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:11:53.661 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T14:11:53.661 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T14:11:53.661 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:11:53.661 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:11:53.661 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:11:53.661 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:11:53.661 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T14:11:53.661 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T14:11:53.661 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T14:11:53.661 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T14:11:54.620 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3230 2024-09-12T14:11:54.639 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:11:54.658 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:11:54.658 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T14:11:54.658 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:11:54.658 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:11:54.658 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T14:11:54.658 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T14:11:54.678 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:11:54.679 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T14:11:54.679 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:11:54.679 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:11:54.679 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T14:11:54.679 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T14:11:54.698 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:11:54.698 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T14:11:54.698 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:11:54.698 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:11:54.698 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T14:11:54.698 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T14:11:54.703 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:11:54.703 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T14:11:54.703 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:11:54.703 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:11:54.703 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T14:11:54.703 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T14:11:55.640 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:11:55.640 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T14:11:55.640 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:11:55.640 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:11:55.640 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T14:11:55.640 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T14:11:55.651 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:11:55.651 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T14:11:55.651 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:11:55.651 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:11:55.651 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T14:11:55.651 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T14:11:55.670 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3233 2024-09-12T14:11:55.692 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:11:56.604 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3232 2024-09-12T14:11:56.609 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:11:56.609 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T14:11:56.609 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:11:56.609 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:11:56.609 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T14:11:56.609 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T14:11:56.614 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3231 2024-09-12T14:11:56.648 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:11:56.651 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:11:56.661 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3236 2024-09-12T14:11:56.668 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3235 2024-09-12T14:11:56.712 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:11:56.713 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:11:57.642 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:11:57.644 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T14:11:57.644 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:11:57.644 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:11:57.644 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T14:11:57.644 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T14:11:57.658 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3237 2024-09-12T14:11:57.669 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3238 2024-09-12T14:11:57.681 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:11:57.688 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:11:58.608 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3239 2024-09-12T14:11:58.632 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:11:58.695 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:11:58.695 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T14:11:58.695 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:11:58.695 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:11:58.695 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T14:11:58.696 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T14:11:59.600 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3234 2024-09-12T14:11:59.637 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3240 2024-09-12T14:11:59.647 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:11:59.651 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:11:59.651 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T14:11:59.651 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:11:59.651 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:11:59.651 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T14:11:59.651 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T14:11:59.653 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:11:59.653 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T14:11:59.654 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:11:59.654 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:11:59.654 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T14:11:59.654 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T14:11:59.667 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:11:59.671 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3241 2024-09-12T14:11:59.689 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:11:59.715 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:11:59.715 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T14:11:59.715 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:11:59.715 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:11:59.715 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T14:11:59.715 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T14:11:59.716 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:11:59.716 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T14:11:59.716 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:11:59.716 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:11:59.717 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T14:11:59.717 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T14:12:00.637 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3243 2024-09-12T14:12:00.647 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3242 2024-09-12T14:12:00.656 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:12:00.657 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:12:00.686 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:12:00.686 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T14:12:00.687 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:12:00.687 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:12:00.687 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T14:12:00.687 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T14:12:00.692 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:12:00.692 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T14:12:00.692 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:12:00.692 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:12:00.693 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T14:12:00.693 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T14:12:01.639 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:12:01.639 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T14:12:01.639 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:12:01.639 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:12:01.639 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T14:12:01.639 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T14:12:01.675 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T14:12:01.675+0000 7f67dd000640 -1 rbd::mirror::image_replayer::snapshot::Replayer: 0x56055b7cec00 handle_unlink_peer: failed to unlink local peer from remote image: (22) Invalid argument 2024-09-12T14:12:01.684 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T14:12:01.684+0000 7f67dd000640 -1 rbd::mirror::image_replayer::snapshot::Replayer: 0x56055b7cf400 handle_unlink_peer: failed to unlink local peer from remote image: (22) Invalid argument 2024-09-12T14:12:02.604 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3245 2024-09-12T14:12:02.627 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3244 2024-09-12T14:12:02.650 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:12:02.650 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T14:12:02.650 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:12:02.650 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:12:02.650 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T14:12:02.650 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T14:12:02.658 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:12:02.670 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:12:02.670 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T14:12:02.670 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:12:02.670 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:12:02.670 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T14:12:02.671 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T14:12:02.685 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:12:02.692 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:12:02.692 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T14:12:02.693 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:12:02.693 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:12:02.693 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T14:12:02.693 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T14:12:03.560 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3246 2024-09-12T14:12:03.565 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3247 2024-09-12T14:12:03.604 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:12:03.612 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:12:03.616 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3248 2024-09-12T14:12:03.649 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3250 2024-09-12T14:12:03.650 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:12:03.659 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:12:03.659 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T14:12:03.660 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:12:03.660 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:12:03.660 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T14:12:03.660 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T14:12:03.660 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:12:03.661 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T14:12:03.661 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:12:03.661 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:12:03.661 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T14:12:03.661 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T14:12:03.676 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:12:04.648 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3251 2024-09-12T14:12:04.678 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:12:05.619 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3249 2024-09-12T14:12:05.650 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:12:05.662 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:12:05.662 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T14:12:05.662 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:12:05.662 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:12:05.662 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T14:12:05.663 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T14:12:05.681 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T14:12:05.681+0000 7f67dd000640 -1 rbd::mirror::image_replayer::snapshot::Replayer: 0x56055bfdf400 handle_unlink_peer: failed to unlink local peer from remote image: (22) Invalid argument 2024-09-12T14:12:05.689 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:12:05.689 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T14:12:05.690 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:12:05.690 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:12:05.690 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T14:12:05.690 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T14:12:06.608 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:12:06.608 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T14:12:06.608 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:12:06.608 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:12:06.608 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T14:12:06.608 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T14:12:06.615 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:12:06.615 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T14:12:06.615 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:12:06.615 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:12:06.615 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T14:12:06.615 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T14:12:06.618 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3253 2024-09-12T14:12:06.625 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3252 2024-09-12T14:12:06.654 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:12:06.654 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T14:12:06.654 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:12:06.654 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:12:06.654 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T14:12:06.654 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T14:12:06.661 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:12:06.664 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:12:06.679 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:12:06.680 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T14:12:06.680 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:12:06.680 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:12:06.680 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T14:12:06.680 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T14:12:06.690 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3255 2024-09-12T14:12:06.695 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3254 2024-09-12T14:12:06.711 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:12:06.714 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:12:07.681 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:12:07.681 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T14:12:07.681 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:12:07.682 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:12:07.682 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T14:12:07.682 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T14:12:07.695 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3257 2024-09-12T14:12:07.700 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3259 2024-09-12T14:12:07.712 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:12:07.719 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3256 2024-09-12T14:12:07.719 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:12:07.729 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:12:08.576 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3258 2024-09-12T14:12:08.598 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:12:08.653 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:12:08.653 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T14:12:08.653 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:12:08.653 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:12:08.654 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T14:12:08.654 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T14:12:09.663 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3260 2024-09-12T14:12:09.665 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:12:09.665 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T14:12:09.665 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:12:09.665 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:12:09.665 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T14:12:09.665 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T14:12:09.668 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:12:09.668 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T14:12:09.668 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:12:09.668 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:12:09.668 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T14:12:09.668 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T14:12:09.698 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3261 2024-09-12T14:12:09.702 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:12:09.717 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:12:09.717 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T14:12:09.717 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:12:09.717 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:12:09.717 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:12:09.717 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T14:12:09.717 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:12:09.717 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:12:09.717 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:12:09.717 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T14:12:09.717 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T14:12:09.718 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T14:12:09.718 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T14:12:10.666 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3265 2024-09-12T14:12:10.668 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3263 2024-09-12T14:12:10.670 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3264 2024-09-12T14:12:10.676 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3262 2024-09-12T14:12:10.686 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:12:10.687 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:12:10.689 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:12:10.689 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:12:10.715 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:12:10.715 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T14:12:10.715 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:12:10.715 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:12:10.716 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T14:12:10.716 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T14:12:10.722 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:12:10.722 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary3 2024-09-12T14:12:10.722 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:12:10.722 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:12:10.722 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T14:12:10.722 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary3 2024-09-12T14:12:10.732 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:12:10.732 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T14:12:10.732 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:12:10.732 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:12:10.732 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T14:12:10.732 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T14:12:11.602 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:12:11.602 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T14:12:11.602 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:12:11.602 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:12:11.602 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T14:12:11.602 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T14:12:12.661 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3266 2024-09-12T14:12:12.702 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:12:12.708 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:12:12.708 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T14:12:12.709 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:12:12.709 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:12:12.709 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T14:12:12.709 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T14:12:12.720 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:12:12.720 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary1 2024-09-12T14:12:12.720 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:12:12.720 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:12:12.720 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T14:12:12.720 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary1 2024-09-12T14:12:12.731 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T14:12:12.731+0000 7f67dd000640 -1 rbd::mirror::image_replayer::snapshot::Replayer: 0x56055ba78c00 handle_unlink_peer: failed to unlink local peer from remote image: (22) Invalid argument 2024-09-12T14:12:13.564 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3268 2024-09-12T14:12:13.569 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3267 2024-09-12T14:12:13.603 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:12:13.615 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3269 2024-09-12T14:12:13.624 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:12:13.679 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:12:13.689 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:12:13.689 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary4 2024-09-12T14:12:13.689 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:12:13.689 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:12:13.690 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T14:12:13.690 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary4 2024-09-12T14:12:13.690 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:12:13.690 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary2 2024-09-12T14:12:13.690 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:12:13.690 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:12:13.690 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T14:12:13.690 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary2 2024-09-12T14:12:13.692 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:12:13.692 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T14:12:13.692 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:12:13.692 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:12:13.692 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T14:12:13.693 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T14:12:13.693 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:12:13.693 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T14:12:13.693 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:12:13.693 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:12:13.693 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T14:12:13.693 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T14:12:14.752 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3274 2024-09-12T14:12:14.755 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3273 2024-09-12T14:12:14.759 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3275 2024-09-12T14:12:14.775 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:12:14.775 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:12:14.777 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:12:15.654 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3270 2024-09-12T14:12:15.671 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3271 2024-09-12T14:12:15.694 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:12:15.703 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:12:16.606 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:12:16.606 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T14:12:16.606 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:12:16.606 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:12:16.606 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T14:12:16.606 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T14:12:16.627 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:12:16.627 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary5 2024-09-12T14:12:16.627 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:12:16.628 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:12:16.628 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T14:12:16.628 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary5 2024-09-12T14:12:16.650 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3272 2024-09-12T14:12:16.670 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:12:16.681 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:12:16.682 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary6 2024-09-12T14:12:16.682 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:12:16.682 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:12:16.682 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T14:12:16.682 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary6 2024-09-12T14:12:17.693 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3277 2024-09-12T14:12:17.694 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3276 2024-09-12T14:12:17.711 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:12:17.711 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:12:17.778 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:12:17.778 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary8 2024-09-12T14:12:17.778 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:12:17.778 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:12:17.779 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T14:12:17.779 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary8 2024-09-12T14:12:17.779 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:12:17.780 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary9 2024-09-12T14:12:17.780 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:12:17.780 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:12:17.780 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T14:12:17.780 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary9 2024-09-12T14:12:18.589 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3278 2024-09-12T14:12:18.619 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:12:18.632 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3280 2024-09-12T14:12:18.649 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:12:18.697 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:12:18.697 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary7 2024-09-12T14:12:18.697 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:12:18.697 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:12:18.697 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T14:12:18.697 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary7 2024-09-12T14:12:18.706 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${SNAP_PIDS[@]} 2024-09-12T14:12:18.706 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 142245 2024-09-12T14:12:18.840 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3279 2024-09-12T14:12:18.859 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:12:19.673 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${SNAP_PIDS[@]} 2024-09-12T14:12:19.673 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 142377 2024-09-12T14:12:19.673 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${SNAP_PIDS[@]} 2024-09-12T14:12:19.673 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 142509 2024-09-12T14:12:19.674 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${SNAP_PIDS[@]} 2024-09-12T14:12:19.674 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 142661 2024-09-12T14:12:19.691 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3281 2024-09-12T14:12:19.706 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:12:20.714 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${SNAP_PIDS[@]} 2024-09-12T14:12:20.714 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 142814 2024-09-12T14:12:20.714 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for i in {1..30} 2024-09-12T14:12:20.714 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ mirror_image_snapshot cluster1 mirror image-primary10 2024-09-12T14:12:20.714 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:12:20.715 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:12:20.715 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T14:12:20.715 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 mirror image snapshot mirror/image-primary10 2024-09-12T14:12:21.623 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${SNAP_PIDS[@]} 2024-09-12T14:12:21.624 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 142967 2024-09-12T14:12:22.694 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T14:12:22.694+0000 7f67dc600640 -1 rbd::mirror::image_replayer::snapshot::Replayer: 0x56055c380000 handle_unlink_peer: failed to unlink local peer from remote image: (22) Invalid argument 2024-09-12T14:12:22.709 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${SNAP_PIDS[@]} 2024-09-12T14:12:22.709 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 143120 2024-09-12T14:12:22.709 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${SNAP_PIDS[@]} 2024-09-12T14:12:22.709 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 143294 2024-09-12T14:12:22.709 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${SNAP_PIDS[@]} 2024-09-12T14:12:22.709 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 143468 2024-09-12T14:12:23.558 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Snapshot ID: 3282 2024-09-12T14:12:23.585 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 3 2024-09-12T14:12:26.587 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ (( RET != 0 )) 2024-09-12T14:12:26.587 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${WORKLOAD_PIDS[@]} 2024-09-12T14:12:26.587 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 142114 2024-09-12T14:14:52.290 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ ret=124 2024-09-12T14:14:52.291 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ (( ret != 124 )) 2024-09-12T14:14:52.291 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${WORKLOAD_PIDS[@]} 2024-09-12T14:14:52.291 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 142246 2024-09-12T14:14:53.239 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ ret=124 2024-09-12T14:14:53.240 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ (( ret != 124 )) 2024-09-12T14:14:53.240 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${WORKLOAD_PIDS[@]} 2024-09-12T14:14:53.240 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 142378 2024-09-12T14:14:54.265 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ ret=124 2024-09-12T14:14:54.265 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ (( ret != 124 )) 2024-09-12T14:14:54.266 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${WORKLOAD_PIDS[@]} 2024-09-12T14:14:54.266 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 142510 2024-09-12T14:14:55.444 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ ret=124 2024-09-12T14:14:55.445 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ (( ret != 124 )) 2024-09-12T14:14:55.445 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${WORKLOAD_PIDS[@]} 2024-09-12T14:14:55.445 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 142662 2024-09-12T14:14:56.471 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ ret=124 2024-09-12T14:14:56.471 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ (( ret != 124 )) 2024-09-12T14:14:56.472 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${WORKLOAD_PIDS[@]} 2024-09-12T14:14:56.472 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 142815 2024-09-12T14:14:57.479 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ ret=124 2024-09-12T14:14:57.480 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ (( ret != 124 )) 2024-09-12T14:14:57.480 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${WORKLOAD_PIDS[@]} 2024-09-12T14:14:57.480 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 142968 2024-09-12T14:14:58.279 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ ret=124 2024-09-12T14:14:58.279 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ (( ret != 124 )) 2024-09-12T14:14:58.279 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${WORKLOAD_PIDS[@]} 2024-09-12T14:14:58.279 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 143121 2024-09-12T14:14:59.409 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ ret=124 2024-09-12T14:14:59.410 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ (( ret != 124 )) 2024-09-12T14:14:59.410 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${WORKLOAD_PIDS[@]} 2024-09-12T14:14:59.410 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 143295 2024-09-12T14:15:00.439 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ ret=124 2024-09-12T14:15:00.439 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ (( ret != 124 )) 2024-09-12T14:15:00.439 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${WORKLOAD_PIDS[@]} 2024-09-12T14:15:00.440 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 143469 2024-09-12T14:15:01.643 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ ret=124 2024-09-12T14:15:01.643 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ (( ret != 124 )) 2024-09-12T14:15:01.643 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ (( RET != 0 )) 2024-09-12T14:15:01.643 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T14:15:01.645 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ COMPARE_PIDS+=($!) 2024-09-12T14:15:01.646 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T14:15:01.646 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ compare_demoted_promoted_image 1 2024-09-12T14:15:01.646 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local dev=/dev/rbd0 2024-09-12T14:15:01.646 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local img=image-primary1 2024-09-12T14:15:01.646 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mntpt=test-primary1 2024-09-12T14:15:01.646 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local demote_md5 promote_md5 2024-09-12T14:15:01.646 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo umount test-primary1 2024-09-12T14:15:01.646 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ COMPARE_PIDS+=($!) 2024-09-12T14:15:01.647 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T14:15:01.647 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ compare_demoted_promoted_image 2 2024-09-12T14:15:01.647 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local dev=/dev/rbd1 2024-09-12T14:15:01.647 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local img=image-primary2 2024-09-12T14:15:01.647 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mntpt=test-primary2 2024-09-12T14:15:01.647 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local demote_md5 promote_md5 2024-09-12T14:15:01.647 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo umount test-primary2 2024-09-12T14:15:01.647 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ COMPARE_PIDS+=($!) 2024-09-12T14:15:01.647 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T14:15:01.647 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ compare_demoted_promoted_image 3 2024-09-12T14:15:01.647 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local dev=/dev/rbd2 2024-09-12T14:15:01.647 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local img=image-primary3 2024-09-12T14:15:01.648 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mntpt=test-primary3 2024-09-12T14:15:01.648 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local demote_md5 promote_md5 2024-09-12T14:15:01.648 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo umount test-primary3 2024-09-12T14:15:01.648 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ COMPARE_PIDS+=($!) 2024-09-12T14:15:01.648 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ compare_demoted_promoted_image 4 2024-09-12T14:15:01.648 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T14:15:01.648 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local dev=/dev/rbd3 2024-09-12T14:15:01.648 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local img=image-primary4 2024-09-12T14:15:01.648 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mntpt=test-primary4 2024-09-12T14:15:01.648 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local demote_md5 promote_md5 2024-09-12T14:15:01.648 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo umount test-primary4 2024-09-12T14:15:01.649 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ COMPARE_PIDS+=($!) 2024-09-12T14:15:01.649 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T14:15:01.650 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ compare_demoted_promoted_image 5 2024-09-12T14:15:01.650 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local dev=/dev/rbd4 2024-09-12T14:15:01.650 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local img=image-primary5 2024-09-12T14:15:01.650 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mntpt=test-primary5 2024-09-12T14:15:01.650 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local demote_md5 promote_md5 2024-09-12T14:15:01.650 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo umount test-primary5 2024-09-12T14:15:01.650 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ COMPARE_PIDS+=($!) 2024-09-12T14:15:01.650 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T14:15:01.650 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ compare_demoted_promoted_image 6 2024-09-12T14:15:01.650 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local dev=/dev/rbd5 2024-09-12T14:15:01.650 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local img=image-primary6 2024-09-12T14:15:01.650 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mntpt=test-primary6 2024-09-12T14:15:01.650 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local demote_md5 promote_md5 2024-09-12T14:15:01.650 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo umount test-primary6 2024-09-12T14:15:01.651 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ COMPARE_PIDS+=($!) 2024-09-12T14:15:01.651 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T14:15:01.651 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ COMPARE_PIDS+=($!) 2024-09-12T14:15:01.651 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T14:15:01.652 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ compare_demoted_promoted_image 7 2024-09-12T14:15:01.652 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local dev=/dev/rbd6 2024-09-12T14:15:01.652 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local img=image-primary7 2024-09-12T14:15:01.652 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mntpt=test-primary7 2024-09-12T14:15:01.652 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ compare_demoted_promoted_image 8 2024-09-12T14:15:01.652 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local demote_md5 promote_md5 2024-09-12T14:15:01.653 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo umount test-primary7 2024-09-12T14:15:01.653 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local dev=/dev/rbd7 2024-09-12T14:15:01.654 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ compare_demoted_promoted_image 9 2024-09-12T14:15:01.654 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local img=image-primary8 2024-09-12T14:15:01.654 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mntpt=test-primary8 2024-09-12T14:15:01.655 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local dev=/dev/rbd8 2024-09-12T14:15:01.655 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local demote_md5 promote_md5 2024-09-12T14:15:01.655 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo umount test-primary8 2024-09-12T14:15:01.656 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ COMPARE_PIDS+=($!) 2024-09-12T14:15:01.656 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local img=image-primary9 2024-09-12T14:15:01.656 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mntpt=test-primary9 2024-09-12T14:15:01.656 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local demote_md5 promote_md5 2024-09-12T14:15:01.657 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T14:15:01.657 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo umount test-primary9 2024-09-12T14:15:01.659 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ COMPARE_PIDS+=($!) 2024-09-12T14:15:01.659 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${COMPARE_PIDS[@]} 2024-09-12T14:15:01.659 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 149431 2024-09-12T14:15:01.664 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ compare_demoted_promoted_image 10 2024-09-12T14:15:01.664 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local dev=/dev/rbd9 2024-09-12T14:15:01.664 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local img=image-primary10 2024-09-12T14:15:01.664 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local mntpt=test-primary10 2024-09-12T14:15:01.664 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local demote_md5 promote_md5 2024-09-12T14:15:01.664 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo umount test-primary10 2024-09-12T14:15:02.156 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ awk '{print $1}' 2024-09-12T14:15:02.156 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo md5sum /dev/rbd5 2024-09-12T14:15:02.219 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ awk '{print $1}' 2024-09-12T14:15:02.220 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo md5sum /dev/rbd1 2024-09-12T14:15:02.327 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo md5sum /dev/rbd0 2024-09-12T14:15:02.330 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ awk '{print $1}' 2024-09-12T14:15:02.342 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ awk '{print $1}' 2024-09-12T14:15:02.342 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo md5sum /dev/rbd6 2024-09-12T14:15:02.374 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ awk '{print $1}' 2024-09-12T14:15:02.377 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo md5sum /dev/rbd2 2024-09-12T14:15:02.403 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ awk '{print $1}' 2024-09-12T14:15:02.405 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo md5sum /dev/rbd7 2024-09-12T14:15:02.456 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo md5sum /dev/rbd4 2024-09-12T14:15:02.458 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ awk '{print $1}' 2024-09-12T14:15:02.538 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo md5sum /dev/rbd3 2024-09-12T14:15:02.540 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ awk '{print $1}' 2024-09-12T14:15:02.593 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo md5sum /dev/rbd9 2024-09-12T14:15:02.595 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ awk '{print $1}' 2024-09-12T14:15:02.677 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ awk '{print $1}' 2024-09-12T14:15:02.678 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo md5sum /dev/rbd8 2024-09-12T14:15:54.445 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ demote_md5=879019d4c94436f67080ba165eb700bd 2024-09-12T14:15:54.445 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo rbd --cluster cluster1 device unmap -t krbd mirror/image-primary9 2024-09-12T14:15:54.854 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ demote_image cluster1 mirror image-primary9 2024-09-12T14:15:54.855 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:15:54.855 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:15:54.855 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T14:15:54.855 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 mirror image demote mirror/image-primary9 2024-09-12T14:15:55.303 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ demote_md5=8e751ae38b30a1dbd73935dcc888ee87 2024-09-12T14:15:55.303 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo rbd --cluster cluster1 device unmap -t krbd mirror/image-primary5 2024-09-12T14:15:55.503 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ demote_image cluster1 mirror image-primary5 2024-09-12T14:15:55.503 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:15:55.503 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:15:55.504 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T14:15:55.504 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 mirror image demote mirror/image-primary5 2024-09-12T14:15:56.540 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ demote_md5=d83c2aa9b60f080de2d1a0b697f25201 2024-09-12T14:15:56.540 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo rbd --cluster cluster1 device unmap -t krbd mirror/image-primary3 2024-09-12T14:15:56.564 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ demote_md5=2c5c576a3e47d2ffc5f635ac482e1168 2024-09-12T14:15:56.564 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo rbd --cluster cluster1 device unmap -t krbd mirror/image-primary8 2024-09-12T14:15:56.899 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ demote_image cluster1 mirror image-primary3 2024-09-12T14:15:56.899 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:15:56.900 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:15:56.900 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T14:15:56.900 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 mirror image demote mirror/image-primary3 2024-09-12T14:15:56.952 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ demote_image cluster1 mirror image-primary8 2024-09-12T14:15:56.952 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:15:56.953 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:15:56.953 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T14:15:56.953 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 mirror image demote mirror/image-primary8 2024-09-12T14:15:57.183 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ demote_md5=e531d7f51e023d59799d74cb419aa852 2024-09-12T14:15:57.183 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ demote_md5=43be192ff1f4f2115159789cd54ac918 2024-09-12T14:15:57.183 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo rbd --cluster cluster1 device unmap -t krbd mirror/image-primary6 2024-09-12T14:15:57.183 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo rbd --cluster cluster1 device unmap -t krbd mirror/image-primary1 2024-09-12T14:15:57.360 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ demote_image cluster1 mirror image-primary1 2024-09-12T14:15:57.360 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:15:57.360 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:15:57.360 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T14:15:57.360 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 mirror image demote mirror/image-primary1 2024-09-12T14:15:57.404 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ demote_image cluster1 mirror image-primary6 2024-09-12T14:15:57.404 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:15:57.404 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:15:57.404 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T14:15:57.404 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 mirror image demote mirror/image-primary6 2024-09-12T14:15:57.519 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ demote_md5=2f7e271f77b6f892aa604d1d6246a9e9 2024-09-12T14:15:57.519 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo rbd --cluster cluster1 device unmap -t krbd mirror/image-primary7 2024-09-12T14:15:57.701 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Image demoted to non-primary 2024-09-12T14:15:57.703 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ demote_image cluster1 mirror image-primary7 2024-09-12T14:15:57.703 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:15:57.704 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:15:57.704 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T14:15:57.704 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 mirror image demote mirror/image-primary7 2024-09-12T14:15:57.727 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Image demoted to non-primary 2024-09-12T14:15:57.756 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary5 up+unknown 2024-09-12T14:15:57.757 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:15:57.757 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:15:57.757 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T14:15:57.757 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:15:57.757 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:15:57.757 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:15:57.757 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:15:57.757 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T14:15:57.777 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary1 up+unknown 2024-09-12T14:15:57.777 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:15:57.777 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:15:57.777 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T14:15:57.777 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:15:57.777 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:15:57.777 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:15:57.777 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:15:57.778 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T14:15:58.316 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ demote_md5=ef619673f0a848e6395c538696a537a7 2024-09-12T14:15:58.316 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo rbd --cluster cluster1 device unmap -t krbd mirror/image-primary2 2024-09-12T14:15:58.358 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ demote_md5=7756588463295cc74a757e122455f81a 2024-09-12T14:15:58.358 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo rbd --cluster cluster1 device unmap -t krbd mirror/image-primary10 2024-09-12T14:15:58.617 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ demote_image cluster1 mirror image-primary2 2024-09-12T14:15:58.617 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:15:58.617 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:15:58.617 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T14:15:58.617 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 mirror image demote mirror/image-primary2 2024-09-12T14:15:58.620 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ demote_image cluster1 mirror image-primary10 2024-09-12T14:15:58.620 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:15:58.623 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:15:58.624 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T14:15:58.624 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 mirror image demote mirror/image-primary10 2024-09-12T14:15:58.770 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary5 up+unknown '' '' 2024-09-12T14:15:58.770 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:15:58.770 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:15:58.770 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T14:15:58.770 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:15:58.770 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:15:58.770 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:15:58.770 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:15:58.774 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:15:58.775 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary5 2024-09-12T14:15:58.793 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary1 up+unknown '' '' 2024-09-12T14:15:58.793 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:15:58.793 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:15:58.793 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T14:15:58.793 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:15:58.793 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:15:58.793 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:15:58.794 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:15:58.797 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:15:58.799 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary1 2024-09-12T14:15:58.933 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Image demoted to non-primary 2024-09-12T14:15:59.033 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary6 up+unknown 2024-09-12T14:15:59.033 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:15:59.035 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:15:59.035 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T14:15:59.036 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:15:59.036 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:15:59.036 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:15:59.036 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:15:59.036 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T14:15:59.153 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary1: 2024-09-12T14:15:59.153 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 9b69126b-93e7-41c5-b8d9-d3784d684fd8 2024-09-12T14:15:59.153 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T14:15:59.153 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T14:15:59.153 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T14:15:59.153 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 14:15:47' 2024-09-12T14:15:59.154 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T14:15:59.174 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary5: 2024-09-12T14:15:59.174 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: d03e84bc-1230-4374-9749-7cc5f3588093 2024-09-12T14:15:59.174 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T14:15:59.175 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T14:15:59.175 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T14:15:59.176 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 14:15:47' 2024-09-12T14:15:59.177 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T14:15:59.333 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T14:15:59.333 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:15:59.333 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 2 2024-09-12T14:15:59.334 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T14:15:59.334 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:15:59.334 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 2 2024-09-12T14:15:59.705 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Image demoted to non-primary 2024-09-12T14:15:59.720 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Image demoted to non-primary 2024-09-12T14:15:59.755 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary7 up+unknown 2024-09-12T14:15:59.755 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:15:59.756 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:15:59.756 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T14:15:59.756 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:15:59.756 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:15:59.756 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:15:59.756 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:15:59.756 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T14:15:59.760 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Image demoted to non-primary 2024-09-12T14:15:59.762 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary9 up+unknown 2024-09-12T14:15:59.763 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:15:59.763 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:15:59.763 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T14:15:59.763 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:15:59.763 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:15:59.763 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:15:59.764 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:15:59.764 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T14:15:59.766 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Image demoted to non-primary 2024-09-12T14:15:59.789 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary2 up+unknown 2024-09-12T14:15:59.790 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:15:59.790 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:15:59.790 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T14:15:59.790 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:15:59.791 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:15:59.791 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:15:59.791 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:15:59.791 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T14:15:59.791 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary10 up+unknown 2024-09-12T14:15:59.792 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:15:59.792 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:15:59.792 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T14:15:59.792 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:15:59.792 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:15:59.792 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:15:59.792 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:15:59.793 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T14:16:00.064 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary6 up+unknown '' '' 2024-09-12T14:16:00.064 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:16:00.064 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:16:00.064 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T14:16:00.064 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:16:00.065 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:16:00.065 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:16:00.065 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:16:00.065 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:16:00.065 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary6 2024-09-12T14:16:00.142 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary6: 2024-09-12T14:16:00.143 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: b2ea2747-3b65-4670-9e03-38ab325e0695 2024-09-12T14:16:00.143 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T14:16:00.143 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T14:16:00.143 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T14:16:00.143 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 14:15:47' 2024-09-12T14:16:00.143 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T14:16:00.145 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T14:16:00.145 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:16:00.145 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 2 2024-09-12T14:16:00.151 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ demote_md5=03c898af3f96b670136c283af7036e56 2024-09-12T14:16:00.151 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo rbd --cluster cluster1 device unmap -t krbd mirror/image-primary4 2024-09-12T14:16:00.255 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ demote_image cluster1 mirror image-primary4 2024-09-12T14:16:00.255 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:16:00.255 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:16:00.256 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T14:16:00.256 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster1 mirror image demote mirror/image-primary4 2024-09-12T14:16:00.694 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Image demoted to non-primary 2024-09-12T14:16:00.721 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary8 up+unknown 2024-09-12T14:16:00.721 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:16:00.721 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:16:00.721 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T14:16:00.721 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:16:00.721 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:16:00.722 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:16:00.722 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:16:00.722 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T14:16:00.758 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary7 up+unknown '' '' 2024-09-12T14:16:00.758 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:16:00.759 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:16:00.759 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T14:16:00.759 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:16:00.760 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:16:00.760 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:16:00.760 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:16:00.761 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:16:00.761 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary7 2024-09-12T14:16:00.770 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary9 up+unknown '' '' 2024-09-12T14:16:00.770 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:16:00.771 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:16:00.772 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T14:16:00.772 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:16:00.772 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:16:00.772 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:16:00.772 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:16:00.773 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:16:00.773 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary9 2024-09-12T14:16:00.800 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary2 up+unknown '' '' 2024-09-12T14:16:00.800 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:16:00.800 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:16:00.801 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T14:16:00.801 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:16:00.801 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:16:00.801 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:16:00.801 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:16:00.802 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:16:00.802 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary2 2024-09-12T14:16:00.805 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary10 up+unknown '' '' 2024-09-12T14:16:00.805 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:16:00.805 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:16:00.805 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T14:16:00.805 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:16:00.805 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:16:00.806 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:16:00.806 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:16:00.809 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:16:00.809 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary10 2024-09-12T14:16:00.950 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary10: 2024-09-12T14:16:00.950 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: d6c75f78-0d2c-4921-8b6d-d39474248deb 2024-09-12T14:16:00.950 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T14:16:00.950 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T14:16:00.951 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T14:16:00.951 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary7: 2024-09-12T14:16:00.951 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 639c39b5-6a6c-4696-8c6f-e70477f65d39 2024-09-12T14:16:00.951 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T14:16:00.951 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T14:16:00.951 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T14:16:00.951 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 14:15:47' 2024-09-12T14:16:00.951 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T14:16:00.951 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 14:15:47' 2024-09-12T14:16:00.951 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T14:16:00.955 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T14:16:00.955 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:16:00.956 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T14:16:00.956 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 2 2024-09-12T14:16:00.956 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:16:00.956 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 2 2024-09-12T14:16:00.967 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary2: 2024-09-12T14:16:00.967 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 7d8bfe1f-b066-4c3e-95a4-c9c60426a59c 2024-09-12T14:16:00.967 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T14:16:00.967 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T14:16:00.968 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T14:16:00.968 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 14:15:47' 2024-09-12T14:16:00.968 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T14:16:00.971 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T14:16:00.971 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:16:00.971 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 2 2024-09-12T14:16:00.973 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary9: 2024-09-12T14:16:00.973 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 8c49b822-b2f9-4ec9-9270-a22fd92ddba3 2024-09-12T14:16:00.973 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T14:16:00.973 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T14:16:00.973 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T14:16:00.973 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 14:15:47' 2024-09-12T14:16:00.973 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T14:16:00.976 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T14:16:00.976 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:16:00.976 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 2 2024-09-12T14:16:01.338 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary1 up+unknown '' '' 2024-09-12T14:16:01.338 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:16:01.338 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:16:01.338 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T14:16:01.338 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:16:01.338 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:16:01.338 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:16:01.338 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:16:01.338 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary5 up+unknown '' '' 2024-09-12T14:16:01.338 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:16:01.338 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:16:01.339 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T14:16:01.339 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:16:01.339 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:16:01.339 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:16:01.339 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:16:01.339 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:16:01.339 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary1 2024-09-12T14:16:01.339 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:16:01.339 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary5 2024-09-12T14:16:01.411 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary1: 2024-09-12T14:16:01.411 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 9b69126b-93e7-41c5-b8d9-d3784d684fd8 2024-09-12T14:16:01.411 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T14:16:01.412 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T14:16:01.412 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T14:16:01.412 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 14:15:47' 2024-09-12T14:16:01.412 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T14:16:01.412 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary5: 2024-09-12T14:16:01.412 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: d03e84bc-1230-4374-9749-7cc5f3588093 2024-09-12T14:16:01.412 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T14:16:01.412 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T14:16:01.413 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T14:16:01.413 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 14:15:47' 2024-09-12T14:16:01.413 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T14:16:01.414 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T14:16:01.414 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:16:01.414 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 4 2024-09-12T14:16:01.414 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T14:16:01.415 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:16:01.415 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 4 2024-09-12T14:16:01.693 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Image demoted to non-primary 2024-09-12T14:16:01.714 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary3 up+unknown 2024-09-12T14:16:01.714 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:16:01.715 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:16:01.715 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T14:16:01.715 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:16:01.715 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:16:01.715 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:16:01.715 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:16:01.715 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T14:16:01.726 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary8 up+unknown '' '' 2024-09-12T14:16:01.726 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:16:01.726 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:16:01.726 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T14:16:01.726 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:16:01.726 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:16:01.726 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:16:01.726 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:16:01.728 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:16:01.728 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary8 2024-09-12T14:16:01.836 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary8: 2024-09-12T14:16:01.836 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 8c5b00c9-9b25-4271-b4e1-865fb28703a0 2024-09-12T14:16:01.836 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T14:16:01.836 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T14:16:01.836 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T14:16:01.836 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 14:15:47' 2024-09-12T14:16:01.836 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T14:16:01.839 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T14:16:01.839 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:16:01.839 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 2 2024-09-12T14:16:02.148 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary6 up+unknown '' '' 2024-09-12T14:16:02.148 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:16:02.148 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:16:02.148 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T14:16:02.148 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:16:02.148 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:16:02.148 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:16:02.148 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:16:02.149 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:16:02.149 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary6 2024-09-12T14:16:02.222 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary6: 2024-09-12T14:16:02.222 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: b2ea2747-3b65-4670-9e03-38ab325e0695 2024-09-12T14:16:02.222 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T14:16:02.222 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T14:16:02.222 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T14:16:02.222 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 14:15:47' 2024-09-12T14:16:02.222 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T14:16:02.225 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T14:16:02.225 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:16:02.225 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 4 2024-09-12T14:16:02.719 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary3 up+unknown '' '' 2024-09-12T14:16:02.719 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:16:02.719 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:16:02.719 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T14:16:02.719 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:16:02.719 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:16:02.720 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:16:02.720 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:16:02.720 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:16:02.720 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary3 2024-09-12T14:16:02.794 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary3: 2024-09-12T14:16:02.794 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 4f62e81b-1fb5-4b07-89a7-8a820b4d657d 2024-09-12T14:16:02.794 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T14:16:02.794 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T14:16:02.794 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T14:16:02.795 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 14:15:47' 2024-09-12T14:16:02.795 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T14:16:02.797 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T14:16:02.797 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:16:02.797 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 2 2024-09-12T14:16:02.961 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary10 up+unknown '' '' 2024-09-12T14:16:02.962 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:16:02.962 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:16:02.962 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T14:16:02.962 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:16:02.962 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:16:02.962 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:16:02.962 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:16:02.962 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary7 up+unknown '' '' 2024-09-12T14:16:02.962 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:16:02.962 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:16:02.962 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T14:16:02.963 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:16:02.963 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:16:02.963 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:16:02.963 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:16:02.963 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:16:02.963 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary10 2024-09-12T14:16:02.963 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:16:02.963 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary7 2024-09-12T14:16:02.973 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary2 up+unknown '' '' 2024-09-12T14:16:02.973 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:16:02.974 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:16:02.974 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T14:16:02.974 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:16:02.974 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:16:02.974 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:16:02.974 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:16:02.974 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:16:02.974 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary2 2024-09-12T14:16:02.978 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary9 up+unknown '' '' 2024-09-12T14:16:02.978 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:16:02.979 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:16:02.979 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T14:16:02.979 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:16:02.979 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:16:02.979 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:16:02.979 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:16:02.979 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:16:02.979 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary9 2024-09-12T14:16:03.039 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary7: 2024-09-12T14:16:03.039 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 639c39b5-6a6c-4696-8c6f-e70477f65d39 2024-09-12T14:16:03.039 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T14:16:03.040 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T14:16:03.040 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T14:16:03.040 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 14:15:47' 2024-09-12T14:16:03.040 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T14:16:03.040 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary10: 2024-09-12T14:16:03.040 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: d6c75f78-0d2c-4921-8b6d-d39474248deb 2024-09-12T14:16:03.040 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T14:16:03.041 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T14:16:03.041 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T14:16:03.041 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 14:15:47' 2024-09-12T14:16:03.041 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T14:16:03.042 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T14:16:03.042 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:16:03.042 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 4 2024-09-12T14:16:03.043 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T14:16:03.043 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:16:03.044 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 4 2024-09-12T14:16:03.056 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary9: 2024-09-12T14:16:03.056 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 8c49b822-b2f9-4ec9-9270-a22fd92ddba3 2024-09-12T14:16:03.056 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T14:16:03.057 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T14:16:03.057 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T14:16:03.057 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 14:15:47' 2024-09-12T14:16:03.057 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T14:16:03.057 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary2: 2024-09-12T14:16:03.057 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 7d8bfe1f-b066-4c3e-95a4-c9c60426a59c 2024-09-12T14:16:03.057 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T14:16:03.057 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T14:16:03.057 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T14:16:03.057 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 14:15:47' 2024-09-12T14:16:03.057 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T14:16:03.059 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T14:16:03.059 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:16:03.059 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 4 2024-09-12T14:16:03.059 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T14:16:03.059 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:16:03.059 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 4 2024-09-12T14:16:03.625 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Image demoted to non-primary 2024-09-12T14:16:03.636 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary4 up+unknown 2024-09-12T14:16:03.637 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:16:03.637 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:16:03.637 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T14:16:03.637 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:16:03.637 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:16:03.637 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:16:03.637 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:16:03.637 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T14:16:03.842 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary8 up+unknown '' '' 2024-09-12T14:16:03.842 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:16:03.842 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:16:03.842 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T14:16:03.842 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:16:03.842 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:16:03.842 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:16:03.843 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:16:03.843 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:16:03.843 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary8 2024-09-12T14:16:03.923 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary8: 2024-09-12T14:16:03.923 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 8c5b00c9-9b25-4271-b4e1-865fb28703a0 2024-09-12T14:16:03.923 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T14:16:03.923 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T14:16:03.923 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T14:16:03.923 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 14:15:47' 2024-09-12T14:16:03.923 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T14:16:03.925 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T14:16:03.925 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:16:03.926 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 4 2024-09-12T14:16:04.639 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary4 up+unknown '' '' 2024-09-12T14:16:04.639 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:16:04.639 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:16:04.639 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T14:16:04.639 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:16:04.639 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:16:04.639 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:16:04.640 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:16:04.640 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:16:04.640 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary4 2024-09-12T14:16:04.717 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary4: 2024-09-12T14:16:04.718 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: f1c9b79b-6866-4e42-bd0f-25d91fd7fdf5 2024-09-12T14:16:04.718 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T14:16:04.718 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T14:16:04.718 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T14:16:04.718 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 14:15:47' 2024-09-12T14:16:04.718 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T14:16:04.720 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T14:16:04.720 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:16:04.720 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 2 2024-09-12T14:16:04.799 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary3 up+unknown '' '' 2024-09-12T14:16:04.800 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:16:04.800 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:16:04.800 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T14:16:04.800 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:16:04.800 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:16:04.800 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:16:04.800 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:16:04.801 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:16:04.801 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary3 2024-09-12T14:16:04.876 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary3: 2024-09-12T14:16:04.876 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 4f62e81b-1fb5-4b07-89a7-8a820b4d657d 2024-09-12T14:16:04.876 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T14:16:04.876 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T14:16:04.876 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T14:16:04.876 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 14:15:47' 2024-09-12T14:16:04.877 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T14:16:04.879 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T14:16:04.879 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:16:04.879 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 4 2024-09-12T14:16:05.418 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary1 up+unknown '' '' 2024-09-12T14:16:05.418 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:16:05.418 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:16:05.418 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T14:16:05.418 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:16:05.418 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:16:05.418 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:16:05.419 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:16:05.419 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary5 up+unknown '' '' 2024-09-12T14:16:05.419 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:16:05.419 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:16:05.419 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T14:16:05.419 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:16:05.419 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:16:05.419 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:16:05.419 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:16:05.419 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:16:05.419 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary1 2024-09-12T14:16:05.419 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:16:05.420 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary5 2024-09-12T14:16:05.493 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary1: 2024-09-12T14:16:05.493 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 9b69126b-93e7-41c5-b8d9-d3784d684fd8 2024-09-12T14:16:05.493 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T14:16:05.493 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T14:16:05.493 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T14:16:05.494 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 14:15:47' 2024-09-12T14:16:05.494 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T14:16:05.494 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary5: 2024-09-12T14:16:05.495 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: d03e84bc-1230-4374-9749-7cc5f3588093 2024-09-12T14:16:05.495 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T14:16:05.495 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T14:16:05.495 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T14:16:05.495 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 14:15:47' 2024-09-12T14:16:05.495 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T14:16:05.496 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T14:16:05.496 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:16:05.496 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 8 2024-09-12T14:16:05.496 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T14:16:05.496 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:16:05.497 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 8 2024-09-12T14:16:06.227 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary6 up+unknown '' '' 2024-09-12T14:16:06.227 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:16:06.227 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:16:06.228 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T14:16:06.228 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:16:06.228 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:16:06.228 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:16:06.228 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:16:06.228 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:16:06.228 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary6 2024-09-12T14:16:06.308 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary6: 2024-09-12T14:16:06.309 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: b2ea2747-3b65-4670-9e03-38ab325e0695 2024-09-12T14:16:06.309 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T14:16:06.309 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T14:16:06.309 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T14:16:06.309 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 14:15:47' 2024-09-12T14:16:06.309 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T14:16:06.311 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T14:16:06.311 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:16:06.311 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 8 2024-09-12T14:16:06.723 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary4 up+unknown '' '' 2024-09-12T14:16:06.723 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:16:06.723 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:16:06.723 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T14:16:06.723 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:16:06.723 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:16:06.723 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:16:06.723 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:16:06.724 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:16:06.724 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary4 2024-09-12T14:16:06.803 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary4: 2024-09-12T14:16:06.803 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: f1c9b79b-6866-4e42-bd0f-25d91fd7fdf5 2024-09-12T14:16:06.803 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T14:16:06.803 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T14:16:06.803 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T14:16:06.803 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 14:15:47' 2024-09-12T14:16:06.803 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T14:16:06.805 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T14:16:06.806 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:16:06.806 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 4 2024-09-12T14:16:07.045 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary7 up+unknown '' '' 2024-09-12T14:16:07.045 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:16:07.045 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:16:07.045 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T14:16:07.045 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:16:07.046 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:16:07.046 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:16:07.046 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:16:07.046 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:16:07.047 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary7 2024-09-12T14:16:07.047 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary10 up+unknown '' '' 2024-09-12T14:16:07.047 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:16:07.047 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:16:07.047 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T14:16:07.047 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:16:07.047 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:16:07.047 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:16:07.047 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:16:07.047 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:16:07.047 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary10 2024-09-12T14:16:07.062 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary9 up+unknown '' '' 2024-09-12T14:16:07.062 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:16:07.062 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:16:07.062 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T14:16:07.062 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:16:07.062 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:16:07.062 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:16:07.062 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:16:07.062 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary2 up+unknown '' '' 2024-09-12T14:16:07.063 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:16:07.063 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:16:07.063 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T14:16:07.063 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:16:07.063 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:16:07.063 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:16:07.063 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:16:07.063 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:16:07.064 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary9 2024-09-12T14:16:07.064 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:16:07.064 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary2 2024-09-12T14:16:07.128 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary10: 2024-09-12T14:16:07.128 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: d6c75f78-0d2c-4921-8b6d-d39474248deb 2024-09-12T14:16:07.129 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T14:16:07.129 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T14:16:07.129 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T14:16:07.129 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 14:15:47' 2024-09-12T14:16:07.129 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T14:16:07.129 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary7: 2024-09-12T14:16:07.129 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 639c39b5-6a6c-4696-8c6f-e70477f65d39 2024-09-12T14:16:07.129 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T14:16:07.129 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T14:16:07.129 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T14:16:07.129 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 14:15:47' 2024-09-12T14:16:07.129 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T14:16:07.131 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T14:16:07.131 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:16:07.132 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 8 2024-09-12T14:16:07.132 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T14:16:07.132 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:16:07.132 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 8 2024-09-12T14:16:07.146 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary9: 2024-09-12T14:16:07.146 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 8c49b822-b2f9-4ec9-9270-a22fd92ddba3 2024-09-12T14:16:07.146 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T14:16:07.146 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T14:16:07.146 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T14:16:07.146 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 14:15:47' 2024-09-12T14:16:07.147 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary2: 2024-09-12T14:16:07.147 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 7d8bfe1f-b066-4c3e-95a4-c9c60426a59c 2024-09-12T14:16:07.147 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T14:16:07.147 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T14:16:07.147 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T14:16:07.147 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 14:15:47' 2024-09-12T14:16:07.147 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T14:16:07.147 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T14:16:07.149 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T14:16:07.149 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:16:07.149 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 8 2024-09-12T14:16:07.150 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T14:16:07.150 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:16:07.150 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 8 2024-09-12T14:16:07.928 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary8 up+unknown '' '' 2024-09-12T14:16:07.928 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:16:07.928 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:16:07.929 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T14:16:07.929 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:16:07.929 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:16:07.929 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:16:07.929 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:16:07.929 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:16:07.929 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary8 2024-09-12T14:16:08.011 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary8: 2024-09-12T14:16:08.011 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 8c5b00c9-9b25-4271-b4e1-865fb28703a0 2024-09-12T14:16:08.011 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T14:16:08.011 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T14:16:08.011 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T14:16:08.012 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 14:15:47' 2024-09-12T14:16:08.012 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T14:16:08.014 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T14:16:08.014 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:16:08.014 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 8 2024-09-12T14:16:08.881 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary3 up+unknown '' '' 2024-09-12T14:16:08.881 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:16:08.882 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:16:08.882 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T14:16:08.882 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:16:08.882 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:16:08.882 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:16:08.882 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:16:08.882 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:16:08.882 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary3 2024-09-12T14:16:08.972 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary3: 2024-09-12T14:16:08.972 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 4f62e81b-1fb5-4b07-89a7-8a820b4d657d 2024-09-12T14:16:08.972 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T14:16:08.972 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T14:16:08.972 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T14:16:08.972 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 14:15:47' 2024-09-12T14:16:08.972 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T14:16:08.975 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T14:16:08.975 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:16:08.975 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 8 2024-09-12T14:16:10.808 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary4 up+unknown '' '' 2024-09-12T14:16:10.809 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:16:10.809 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:16:10.809 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T14:16:10.809 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:16:10.809 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:16:10.809 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:16:10.809 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:16:10.809 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:16:10.810 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary4 2024-09-12T14:16:10.893 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary4: 2024-09-12T14:16:10.894 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: f1c9b79b-6866-4e42-bd0f-25d91fd7fdf5 2024-09-12T14:16:10.894 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T14:16:10.894 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T14:16:10.894 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T14:16:10.894 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 14:15:47' 2024-09-12T14:16:10.894 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T14:16:10.896 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T14:16:10.896 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:16:10.896 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 8 2024-09-12T14:16:13.499 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary1 up+unknown '' '' 2024-09-12T14:16:13.499 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:16:13.499 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:16:13.499 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T14:16:13.499 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:16:13.499 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:16:13.499 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:16:13.500 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:16:13.500 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary5 up+unknown '' '' 2024-09-12T14:16:13.500 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:16:13.500 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:16:13.500 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T14:16:13.500 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:16:13.500 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:16:13.500 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:16:13.500 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:16:13.501 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:16:13.501 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary1 2024-09-12T14:16:13.501 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:16:13.501 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary5 2024-09-12T14:16:13.586 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary5: 2024-09-12T14:16:13.586 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: d03e84bc-1230-4374-9749-7cc5f3588093 2024-09-12T14:16:13.586 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T14:16:13.586 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T14:16:13.586 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T14:16:13.586 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 14:15:47' 2024-09-12T14:16:13.586 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T14:16:13.587 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary1: 2024-09-12T14:16:13.587 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 9b69126b-93e7-41c5-b8d9-d3784d684fd8 2024-09-12T14:16:13.587 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T14:16:13.587 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T14:16:13.587 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T14:16:13.587 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 14:15:47' 2024-09-12T14:16:13.587 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T14:16:13.589 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T14:16:13.589 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T14:16:13.589 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:16:13.589 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 8 2024-09-12T14:16:13.589 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:16:13.589 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 8 2024-09-12T14:16:14.314 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary6 up+unknown '' '' 2024-09-12T14:16:14.314 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:16:14.314 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:16:14.315 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T14:16:14.315 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:16:14.315 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:16:14.315 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:16:14.315 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:16:14.315 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:16:14.315 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary6 2024-09-12T14:16:14.400 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary6: 2024-09-12T14:16:14.400 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: b2ea2747-3b65-4670-9e03-38ab325e0695 2024-09-12T14:16:14.401 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T14:16:14.401 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T14:16:14.401 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T14:16:14.401 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 14:15:47' 2024-09-12T14:16:14.401 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T14:16:14.403 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T14:16:14.403 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:16:14.403 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 8 2024-09-12T14:16:15.134 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary10 up+unknown '' '' 2024-09-12T14:16:15.135 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary7 up+unknown '' '' 2024-09-12T14:16:15.135 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:16:15.135 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:16:15.135 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T14:16:15.135 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:16:15.135 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:16:15.135 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:16:15.135 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:16:15.135 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:16:15.135 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:16:15.136 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T14:16:15.136 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:16:15.136 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:16:15.136 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:16:15.136 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:16:15.136 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:16:15.136 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary7 2024-09-12T14:16:15.136 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:16:15.137 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary10 2024-09-12T14:16:15.153 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary2 up+unknown '' '' 2024-09-12T14:16:15.154 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:16:15.154 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:16:15.154 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T14:16:15.154 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:16:15.154 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary9 up+unknown '' '' 2024-09-12T14:16:15.154 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:16:15.154 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:16:15.154 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:16:15.154 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:16:15.154 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:16:15.155 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T14:16:15.155 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:16:15.155 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:16:15.155 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:16:15.155 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:16:15.155 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:16:15.155 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary2 2024-09-12T14:16:15.155 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:16:15.155 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary9 2024-09-12T14:16:15.221 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary7: 2024-09-12T14:16:15.221 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 639c39b5-6a6c-4696-8c6f-e70477f65d39 2024-09-12T14:16:15.221 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T14:16:15.222 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T14:16:15.222 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T14:16:15.222 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 14:15:47' 2024-09-12T14:16:15.222 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T14:16:15.222 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary10: 2024-09-12T14:16:15.222 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: d6c75f78-0d2c-4921-8b6d-d39474248deb 2024-09-12T14:16:15.222 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T14:16:15.222 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T14:16:15.222 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T14:16:15.222 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 14:15:47' 2024-09-12T14:16:15.222 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T14:16:15.224 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T14:16:15.224 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T14:16:15.224 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:16:15.224 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 8 2024-09-12T14:16:15.224 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:16:15.224 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 8 2024-09-12T14:16:15.235 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary2: 2024-09-12T14:16:15.235 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 7d8bfe1f-b066-4c3e-95a4-c9c60426a59c 2024-09-12T14:16:15.235 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T14:16:15.235 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T14:16:15.235 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T14:16:15.235 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 14:15:47' 2024-09-12T14:16:15.235 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T14:16:15.238 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T14:16:15.238 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary9: 2024-09-12T14:16:15.238 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 8c49b822-b2f9-4ec9-9270-a22fd92ddba3 2024-09-12T14:16:15.238 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T14:16:15.238 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T14:16:15.238 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T14:16:15.238 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 14:15:47' 2024-09-12T14:16:15.238 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T14:16:15.238 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:16:15.239 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 8 2024-09-12T14:16:15.240 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T14:16:15.240 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:16:15.241 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 8 2024-09-12T14:16:16.017 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary8 up+unknown '' '' 2024-09-12T14:16:16.017 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:16:16.018 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:16:16.018 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T14:16:16.018 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:16:16.018 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:16:16.018 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:16:16.018 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:16:16.019 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:16:16.019 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary8 2024-09-12T14:16:16.193 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary8: 2024-09-12T14:16:16.193 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 8c5b00c9-9b25-4271-b4e1-865fb28703a0 2024-09-12T14:16:16.193 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T14:16:16.193 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T14:16:16.193 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T14:16:16.193 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 14:15:47' 2024-09-12T14:16:16.193 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T14:16:16.196 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T14:16:16.196 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:16:16.196 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 8 2024-09-12T14:16:16.978 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary3 up+unknown '' '' 2024-09-12T14:16:16.978 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:16:16.978 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:16:16.978 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T14:16:16.978 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:16:16.978 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:16:16.978 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:16:16.978 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:16:16.979 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:16:16.979 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary3 2024-09-12T14:16:17.062 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary3: 2024-09-12T14:16:17.062 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 4f62e81b-1fb5-4b07-89a7-8a820b4d657d 2024-09-12T14:16:17.062 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+stopped 2024-09-12T14:16:17.062 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: local image is primary 2024-09-12T14:16:17.062 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T14:16:17.062 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 14:15:47' 2024-09-12T14:16:17.062 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T14:16:17.065 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 1 2024-09-12T14:16:17.065 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:16:17.065 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 8 2024-09-12T14:16:18.899 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary4 up+unknown '' '' 2024-09-12T14:16:18.900 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:16:18.900 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:16:18.900 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T14:16:18.900 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:16:18.900 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:16:18.900 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:16:18.900 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:16:18.900 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:16:18.901 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary4 2024-09-12T14:16:18.982 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary4: 2024-09-12T14:16:18.982 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: f1c9b79b-6866-4e42-bd0f-25d91fd7fdf5 2024-09-12T14:16:18.982 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+unknown 2024-09-12T14:16:18.982 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: remote image is not primary 2024-09-12T14:16:18.982 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T14:16:18.983 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 14:16:17' 2024-09-12T14:16:18.983 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T14:16:18.984 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+unknown 2024-09-12T14:16:18.984 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T14:16:18.987 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: remote image is not primary 2024-09-12T14:16:18.987 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T14:16:18.988 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T14:16:18.988 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T14:16:18.990 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T14:16:18.991 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T14:16:18.993 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T14:16:18.994 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T14:16:18.994 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T14:16:18.998 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 14:16:17' 2024-09-12T14:16:18.998 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary4 up+unknown '2024-09-12 14:16:17' 2024-09-12T14:16:18.999 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:16:18.999 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:16:18.999 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T14:16:18.999 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:16:18.999 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 14:16:17' 2024-09-12T14:16:18.999 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:16:18.999 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:16:18.999 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-09-12T14:16:19.109 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041934232mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary19b69126b-93e7-41c5-b8d9-d3784d684fd8up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:16:17image-primary10d6c75f78-0d2c-4921-8b6d-d39474248debup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:16:17image-primary27d8bfe1f-b066-4c3e-95a4-c9c60426a59cup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:16:17image-primary34f62e81b-1fb5-4b07-89a7-8a820b4d657dup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:16:17image-primary4f1c9b79b-6866-4e42-bd0f-25d91fd7fdf5up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:16:17image-primary5d03e84bc-1230-4374-9749-7cc5f3588093up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:16:17image-primary6b2ea2747-3b65-4670-9e03-38ab325e0695up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:16:17image-primary7639c39b5-6a6c-4696-8c6f-e70477f65d39up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:16:17image-primary88c5b00c9-9b25-4271-b4e1-865fb28703a0up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:16:17image-primary98c49b822-b2f9-4ec9-9270-a22fd92ddba3up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:16:17' 2024-09-12T14:16:19.110 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T14:16:19.110 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary4'\'']/last_update' 2024-09-12T14:16:19.313 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 14:16:17' 2024-09-12T14:16:19.314 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary4'\'']/state' 2024-09-12T14:16:19.318 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+unknown 2024-09-12T14:16:19.318 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T14:16:19.318 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+unknown 2024-09-12T14:16:19.320 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T14:16:19.321 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:16:19.321 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:16:19.321 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster2 mirror image-primary4 up+unknown 2024-09-12T14:16:19.321 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T14:16:19.321 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:16:19.321 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T14:16:19.321 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:16:19.321 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:16:19.321 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:16:19.322 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:16:19.322 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T14:16:20.323 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster2 mirror image-primary4 up+unknown '' '' 2024-09-12T14:16:20.324 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T14:16:20.324 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:16:20.324 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T14:16:20.324 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:16:20.324 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:16:20.324 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:16:20.324 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:16:20.325 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:16:20.325 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster2 mirror image status mirror/image-primary4 2024-09-12T14:16:20.397 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary4: 2024-09-12T14:16:20.397 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: f1c9b79b-6866-4e42-bd0f-25d91fd7fdf5 2024-09-12T14:16:20.397 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+unknown 2024-09-12T14:16:20.397 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: remote image is not primary 2024-09-12T14:16:20.397 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T14:16:20.397 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 14:16:17' 2024-09-12T14:16:20.397 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T14:16:20.399 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+unknown 2024-09-12T14:16:20.400 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T14:16:20.402 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: remote image is not primary 2024-09-12T14:16:20.402 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T14:16:20.403 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T14:16:20.403 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T14:16:20.405 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T14:16:20.406 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T14:16:20.408 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T14:16:20.409 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T14:16:20.409 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T14:16:20.413 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 14:16:17' 2024-09-12T14:16:20.414 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster2 mirror image-primary4 up+unknown '2024-09-12 14:16:17' 2024-09-12T14:16:20.414 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T14:16:20.414 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:16:20.414 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T14:16:20.414 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:16:20.414 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 14:16:17' 2024-09-12T14:16:20.414 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:16:20.414 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:16:20.414 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2024-09-12T14:16:20.518 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041924231mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary19b69126b-93e7-41c5-b8d9-d3784d684fd8up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:16:17image-primary10d6c75f78-0d2c-4921-8b6d-d39474248debup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:16:17image-primary27d8bfe1f-b066-4c3e-95a4-c9c60426a59cup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:16:17image-primary34f62e81b-1fb5-4b07-89a7-8a820b4d657dup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:16:17image-primary4f1c9b79b-6866-4e42-bd0f-25d91fd7fdf5up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:16:17image-primary5d03e84bc-1230-4374-9749-7cc5f3588093up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:16:17image-primary6b2ea2747-3b65-4670-9e03-38ab325e0695up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:16:17image-primary7639c39b5-6a6c-4696-8c6f-e70477f65d39up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:16:17image-primary88c5b00c9-9b25-4271-b4e1-865fb28703a0up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:16:17image-primary98c49b822-b2f9-4ec9-9270-a22fd92ddba3up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:16:17' 2024-09-12T14:16:20.518 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T14:16:20.518 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary4'\'']/last_update' 2024-09-12T14:16:20.522 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 14:16:17' 2024-09-12T14:16:20.523 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary4'\'']/state' 2024-09-12T14:16:20.526 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+unknown 2024-09-12T14:16:20.527 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T14:16:20.527 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+unknown 2024-09-12T14:16:20.529 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T14:16:20.530 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:16:20.530 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:16:20.531 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ promote_image cluster2 mirror image-primary4 2024-09-12T14:16:20.531 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T14:16:20.531 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:16:20.531 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T14:16:20.531 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local force= 2024-09-12T14:16:20.531 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 mirror image promote mirror/image-primary4 2024-09-12T14:16:20.939 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Image promoted to primary 2024-09-12T14:16:20.952 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \n\b\d ]] 2024-09-12T14:16:20.952 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \k\r\b\d ]] 2024-09-12T14:16:20.953 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo rbd --cluster cluster2 device map -t krbd mirror/image-primary4 2024-09-12T14:16:21.082 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ dev=/dev/rbd0 2024-09-12T14:16:21.083 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo md5sum /dev/rbd0 2024-09-12T14:16:21.084 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ awk '{print $1}' 2024-09-12T14:16:21.592 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary1 up+unknown '' '' 2024-09-12T14:16:21.592 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary5 up+unknown '' '' 2024-09-12T14:16:21.592 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:16:21.592 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:16:21.592 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T14:16:21.592 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:16:21.592 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:16:21.592 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:16:21.592 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:16:21.592 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:16:21.592 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:16:21.592 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T14:16:21.593 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:16:21.593 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:16:21.593 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:16:21.593 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:16:21.593 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:16:21.593 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary5 2024-09-12T14:16:21.593 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:16:21.594 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary1 2024-09-12T14:16:21.668 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary5: 2024-09-12T14:16:21.668 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: d03e84bc-1230-4374-9749-7cc5f3588093 2024-09-12T14:16:21.668 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+unknown 2024-09-12T14:16:21.668 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: remote image is not primary 2024-09-12T14:16:21.668 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T14:16:21.669 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 14:16:17' 2024-09-12T14:16:21.669 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary1: 2024-09-12T14:16:21.669 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 9b69126b-93e7-41c5-b8d9-d3784d684fd8 2024-09-12T14:16:21.669 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+unknown 2024-09-12T14:16:21.669 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: remote image is not primary 2024-09-12T14:16:21.669 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T14:16:21.669 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 14:16:17' 2024-09-12T14:16:21.670 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T14:16:21.670 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T14:16:21.670 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+unknown 2024-09-12T14:16:21.670 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+unknown 2024-09-12T14:16:21.671 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T14:16:21.671 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T14:16:21.673 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: remote image is not primary 2024-09-12T14:16:21.673 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: remote image is not primary 2024-09-12T14:16:21.674 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T14:16:21.675 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T14:16:21.675 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T14:16:21.675 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T14:16:21.675 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T14:16:21.676 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T14:16:21.676 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T14:16:21.677 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T14:16:21.677 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T14:16:21.677 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T14:16:21.679 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T14:16:21.679 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T14:16:21.680 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T14:16:21.680 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T14:16:21.681 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T14:16:21.681 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T14:16:21.684 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 14:16:17' 2024-09-12T14:16:21.685 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 14:16:17' 2024-09-12T14:16:21.685 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary5 up+unknown '2024-09-12 14:16:17' 2024-09-12T14:16:21.685 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary1 up+unknown '2024-09-12 14:16:17' 2024-09-12T14:16:21.685 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:16:21.685 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:16:21.685 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:16:21.685 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T14:16:21.685 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:16:21.685 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:16:21.685 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T14:16:21.685 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 14:16:17' 2024-09-12T14:16:21.686 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:16:21.686 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:16:21.686 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 14:16:17' 2024-09-12T14:16:21.686 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:16:21.686 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:16:21.687 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-09-12T14:16:21.687 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:16:21.687 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-09-12T14:16:21.824 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041934232mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary19b69126b-93e7-41c5-b8d9-d3784d684fd8up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:16:17image-primary10d6c75f78-0d2c-4921-8b6d-d39474248debup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:16:17image-primary27d8bfe1f-b066-4c3e-95a4-c9c60426a59cup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:16:17image-primary34f62e81b-1fb5-4b07-89a7-8a820b4d657dup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:16:17image-primary4f1c9b79b-6866-4e42-bd0f-25d91fd7fdf5up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:16:17image-primary5d03e84bc-1230-4374-9749-7cc5f3588093up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:16:17image-primary6b2ea2747-3b65-4670-9e03-38ab325e0695up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:16:17image-primary7639c39b5-6a6c-4696-8c6f-e70477f65d39up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:16:17image-primary88c5b00c9-9b25-4271-b4e1-865fb28703a0up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:16:17image-primary98c49b822-b2f9-4ec9-9270-a22fd92ddba3up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:16:17' 2024-09-12T14:16:21.825 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T14:16:21.826 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary1'\'']/last_update' 2024-09-12T14:16:21.826 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041934232mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary19b69126b-93e7-41c5-b8d9-d3784d684fd8up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:16:17image-primary10d6c75f78-0d2c-4921-8b6d-d39474248debup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:16:17image-primary27d8bfe1f-b066-4c3e-95a4-c9c60426a59cup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:16:17image-primary34f62e81b-1fb5-4b07-89a7-8a820b4d657dup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:16:17image-primary4f1c9b79b-6866-4e42-bd0f-25d91fd7fdf5up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:16:17image-primary5d03e84bc-1230-4374-9749-7cc5f3588093up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:16:17image-primary6b2ea2747-3b65-4670-9e03-38ab325e0695up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:16:17image-primary7639c39b5-6a6c-4696-8c6f-e70477f65d39up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:16:17image-primary88c5b00c9-9b25-4271-b4e1-865fb28703a0up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:16:17image-primary98c49b822-b2f9-4ec9-9270-a22fd92ddba3up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:16:17' 2024-09-12T14:16:21.826 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T14:16:21.827 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary5'\'']/last_update' 2024-09-12T14:16:21.829 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 14:16:17' 2024-09-12T14:16:21.830 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary1'\'']/state' 2024-09-12T14:16:21.830 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 14:16:17' 2024-09-12T14:16:21.831 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary5'\'']/state' 2024-09-12T14:16:21.836 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+unknown 2024-09-12T14:16:21.836 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+unknown 2024-09-12T14:16:21.836 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T14:16:21.837 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+unknown 2024-09-12T14:16:21.837 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T14:16:21.837 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+unknown 2024-09-12T14:16:21.837 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T14:16:21.838 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:16:21.838 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:16:21.838 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster2 mirror image-primary1 up+unknown 2024-09-12T14:16:21.838 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T14:16:21.839 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:16:21.839 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T14:16:21.839 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:16:21.839 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:16:21.839 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:16:21.839 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:16:21.839 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T14:16:21.840 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T14:16:21.841 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:16:21.841 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:16:21.841 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster2 mirror image-primary5 up+unknown 2024-09-12T14:16:21.841 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T14:16:21.841 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:16:21.841 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T14:16:21.841 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:16:21.842 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:16:21.842 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:16:21.842 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:16:21.842 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T14:16:22.406 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary6 up+unknown '' '' 2024-09-12T14:16:22.406 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:16:22.406 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:16:22.406 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T14:16:22.407 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:16:22.407 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:16:22.407 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:16:22.407 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:16:22.407 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:16:22.407 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary6 2024-09-12T14:16:22.480 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary6: 2024-09-12T14:16:22.480 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: b2ea2747-3b65-4670-9e03-38ab325e0695 2024-09-12T14:16:22.480 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+unknown 2024-09-12T14:16:22.480 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: remote image is not primary 2024-09-12T14:16:22.480 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T14:16:22.480 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 14:16:17' 2024-09-12T14:16:22.480 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T14:16:22.482 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+unknown 2024-09-12T14:16:22.483 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T14:16:22.485 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: remote image is not primary 2024-09-12T14:16:22.485 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T14:16:22.486 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T14:16:22.486 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T14:16:22.488 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T14:16:22.488 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T14:16:22.491 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T14:16:22.491 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T14:16:22.492 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T14:16:22.496 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 14:16:17' 2024-09-12T14:16:22.496 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary6 up+unknown '2024-09-12 14:16:17' 2024-09-12T14:16:22.496 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:16:22.496 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:16:22.496 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T14:16:22.496 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:16:22.496 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 14:16:17' 2024-09-12T14:16:22.496 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:16:22.497 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:16:22.497 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-09-12T14:16:22.607 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041934232mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary19b69126b-93e7-41c5-b8d9-d3784d684fd8up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:16:17image-primary10d6c75f78-0d2c-4921-8b6d-d39474248debup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:16:17image-primary27d8bfe1f-b066-4c3e-95a4-c9c60426a59cup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:16:17image-primary34f62e81b-1fb5-4b07-89a7-8a820b4d657dup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:16:17image-primary4f1c9b79b-6866-4e42-bd0f-25d91fd7fdf5up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:16:17image-primary5d03e84bc-1230-4374-9749-7cc5f3588093up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:16:17image-primary6b2ea2747-3b65-4670-9e03-38ab325e0695up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:16:17image-primary7639c39b5-6a6c-4696-8c6f-e70477f65d39up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:16:17image-primary88c5b00c9-9b25-4271-b4e1-865fb28703a0up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:16:17image-primary98c49b822-b2f9-4ec9-9270-a22fd92ddba3up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:16:17' 2024-09-12T14:16:22.608 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T14:16:22.608 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary6'\'']/last_update' 2024-09-12T14:16:22.612 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 14:16:17' 2024-09-12T14:16:22.613 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary6'\'']/state' 2024-09-12T14:16:22.617 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+unknown 2024-09-12T14:16:22.618 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T14:16:22.618 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+unknown 2024-09-12T14:16:22.620 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T14:16:22.622 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:16:22.622 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:16:22.622 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster2 mirror image-primary6 up+unknown 2024-09-12T14:16:22.623 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T14:16:22.623 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:16:22.623 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T14:16:22.623 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:16:22.623 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:16:22.623 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:16:22.624 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:16:22.624 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T14:16:22.841 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster2 mirror image-primary1 up+unknown '' '' 2024-09-12T14:16:22.842 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T14:16:22.842 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:16:22.842 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T14:16:22.842 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:16:22.842 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:16:22.843 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:16:22.843 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:16:22.843 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster2 mirror image-primary5 up+unknown '' '' 2024-09-12T14:16:22.843 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T14:16:22.843 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:16:22.844 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T14:16:22.844 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:16:22.844 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:16:22.844 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:16:22.844 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:16:22.844 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:16:22.844 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster2 mirror image status mirror/image-primary1 2024-09-12T14:16:22.844 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:16:22.844 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster2 mirror image status mirror/image-primary5 2024-09-12T14:16:22.913 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary5: 2024-09-12T14:16:22.913 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: d03e84bc-1230-4374-9749-7cc5f3588093 2024-09-12T14:16:22.913 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+unknown 2024-09-12T14:16:22.913 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: remote image is not primary 2024-09-12T14:16:22.913 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T14:16:22.913 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 14:16:17' 2024-09-12T14:16:22.914 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T14:16:22.915 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary1: 2024-09-12T14:16:22.915 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 9b69126b-93e7-41c5-b8d9-d3784d684fd8 2024-09-12T14:16:22.916 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+unknown 2024-09-12T14:16:22.916 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: remote image is not primary 2024-09-12T14:16:22.916 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T14:16:22.916 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 14:16:17' 2024-09-12T14:16:22.916 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T14:16:22.917 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+unknown 2024-09-12T14:16:22.917 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T14:16:22.918 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+unknown 2024-09-12T14:16:22.919 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T14:16:22.920 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: remote image is not primary 2024-09-12T14:16:22.921 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T14:16:22.922 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T14:16:22.922 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: remote image is not primary 2024-09-12T14:16:22.922 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T14:16:22.922 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T14:16:22.923 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T14:16:22.923 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T14:16:22.924 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T14:16:22.925 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T14:16:22.926 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T14:16:22.926 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T14:16:22.928 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T14:16:22.928 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T14:16:22.929 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T14:16:22.929 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T14:16:22.929 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T14:16:22.930 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T14:16:22.932 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 14:16:17' 2024-09-12T14:16:22.932 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster2 mirror image-primary5 up+unknown '2024-09-12 14:16:17' 2024-09-12T14:16:22.933 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T14:16:22.933 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:16:22.933 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T14:16:22.933 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:16:22.933 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 14:16:17' 2024-09-12T14:16:22.933 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:16:22.934 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 14:16:17' 2024-09-12T14:16:22.934 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster2 mirror image-primary1 up+unknown '2024-09-12 14:16:17' 2024-09-12T14:16:22.934 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:16:22.934 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2024-09-12T14:16:22.934 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T14:16:22.934 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:16:22.934 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T14:16:22.934 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:16:22.934 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 14:16:17' 2024-09-12T14:16:22.934 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:16:22.934 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:16:22.934 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2024-09-12T14:16:23.075 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041924231mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary19b69126b-93e7-41c5-b8d9-d3784d684fd8up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:16:17image-primary10d6c75f78-0d2c-4921-8b6d-d39474248debup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:16:17image-primary27d8bfe1f-b066-4c3e-95a4-c9c60426a59cup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:16:17image-primary34f62e81b-1fb5-4b07-89a7-8a820b4d657dup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:16:17image-primary4f1c9b79b-6866-4e42-bd0f-25d91fd7fdf5up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:16:17image-primary5d03e84bc-1230-4374-9749-7cc5f3588093up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:16:17image-primary6b2ea2747-3b65-4670-9e03-38ab325e0695up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:16:17image-primary7639c39b5-6a6c-4696-8c6f-e70477f65d39up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:16:17image-primary88c5b00c9-9b25-4271-b4e1-865fb28703a0up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:16:17image-primary98c49b822-b2f9-4ec9-9270-a22fd92ddba3up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:16:17' 2024-09-12T14:16:23.075 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T14:16:23.076 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary1'\'']/last_update' 2024-09-12T14:16:23.076 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041924231mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary19b69126b-93e7-41c5-b8d9-d3784d684fd8up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:16:17image-primary10d6c75f78-0d2c-4921-8b6d-d39474248debup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:16:17image-primary27d8bfe1f-b066-4c3e-95a4-c9c60426a59cup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:16:17image-primary34f62e81b-1fb5-4b07-89a7-8a820b4d657dup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:16:17image-primary4f1c9b79b-6866-4e42-bd0f-25d91fd7fdf5up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:16:17image-primary5d03e84bc-1230-4374-9749-7cc5f3588093up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:16:17image-primary6b2ea2747-3b65-4670-9e03-38ab325e0695up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:16:17image-primary7639c39b5-6a6c-4696-8c6f-e70477f65d39up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:16:17image-primary88c5b00c9-9b25-4271-b4e1-865fb28703a0up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:16:17image-primary98c49b822-b2f9-4ec9-9270-a22fd92ddba3up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:16:17' 2024-09-12T14:16:23.076 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T14:16:23.077 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary5'\'']/last_update' 2024-09-12T14:16:23.080 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 14:16:17' 2024-09-12T14:16:23.081 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 14:16:17' 2024-09-12T14:16:23.081 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary1'\'']/state' 2024-09-12T14:16:23.082 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary5'\'']/state' 2024-09-12T14:16:23.085 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+unknown 2024-09-12T14:16:23.086 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+unknown 2024-09-12T14:16:23.086 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T14:16:23.086 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+unknown 2024-09-12T14:16:23.087 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T14:16:23.087 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+unknown 2024-09-12T14:16:23.088 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T14:16:23.089 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:16:23.089 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:16:23.089 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ promote_image cluster2 mirror image-primary1 2024-09-12T14:16:23.090 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T14:16:23.090 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:16:23.090 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T14:16:23.090 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local force= 2024-09-12T14:16:23.090 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 mirror image promote mirror/image-primary1 2024-09-12T14:16:23.090 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T14:16:23.091 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:16:23.091 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:16:23.091 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ promote_image cluster2 mirror image-primary5 2024-09-12T14:16:23.091 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T14:16:23.091 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:16:23.091 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T14:16:23.092 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local force= 2024-09-12T14:16:23.092 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 mirror image promote mirror/image-primary5 2024-09-12T14:16:23.227 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary10 up+unknown '' '' 2024-09-12T14:16:23.227 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary7 up+unknown '' '' 2024-09-12T14:16:23.227 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:16:23.227 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:16:23.227 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:16:23.227 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T14:16:23.227 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:16:23.227 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T14:16:23.227 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:16:23.228 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:16:23.228 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:16:23.228 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:16:23.228 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:16:23.228 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:16:23.228 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:16:23.228 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:16:23.228 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:16:23.229 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary10 2024-09-12T14:16:23.229 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:16:23.229 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary7 2024-09-12T14:16:23.240 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary2 up+unknown '' '' 2024-09-12T14:16:23.241 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:16:23.241 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:16:23.241 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T14:16:23.241 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:16:23.241 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:16:23.241 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:16:23.241 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:16:23.241 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:16:23.241 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary2 2024-09-12T14:16:23.243 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary9 up+unknown '' '' 2024-09-12T14:16:23.243 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:16:23.243 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:16:23.243 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T14:16:23.243 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:16:23.244 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:16:23.244 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:16:23.244 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:16:23.244 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:16:23.244 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary9 2024-09-12T14:16:23.310 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary7: 2024-09-12T14:16:23.310 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 639c39b5-6a6c-4696-8c6f-e70477f65d39 2024-09-12T14:16:23.310 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+unknown 2024-09-12T14:16:23.310 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: remote image is not primary 2024-09-12T14:16:23.310 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T14:16:23.310 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 14:16:17' 2024-09-12T14:16:23.310 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T14:16:23.311 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary10: 2024-09-12T14:16:23.311 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: d6c75f78-0d2c-4921-8b6d-d39474248deb 2024-09-12T14:16:23.311 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+unknown 2024-09-12T14:16:23.311 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: remote image is not primary 2024-09-12T14:16:23.311 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T14:16:23.311 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 14:16:17' 2024-09-12T14:16:23.311 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T14:16:23.314 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+unknown 2024-09-12T14:16:23.314 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+unknown 2024-09-12T14:16:23.315 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T14:16:23.315 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T14:16:23.317 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: remote image is not primary 2024-09-12T14:16:23.317 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: remote image is not primary 2024-09-12T14:16:23.318 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T14:16:23.318 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T14:16:23.318 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T14:16:23.319 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T14:16:23.319 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T14:16:23.319 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T14:16:23.321 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T14:16:23.321 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T14:16:23.322 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T14:16:23.322 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T14:16:23.324 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T14:16:23.324 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T14:16:23.325 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T14:16:23.325 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T14:16:23.325 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T14:16:23.326 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T14:16:23.329 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 14:16:17' 2024-09-12T14:16:23.329 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary10 up+unknown '2024-09-12 14:16:17' 2024-09-12T14:16:23.330 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:16:23.330 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:16:23.330 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T14:16:23.330 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:16:23.330 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 14:16:17' 2024-09-12T14:16:23.330 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:16:23.330 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 14:16:17' 2024-09-12T14:16:23.331 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary7 up+unknown '2024-09-12 14:16:17' 2024-09-12T14:16:23.331 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:16:23.331 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:16:23.331 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T14:16:23.331 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:16:23.331 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 14:16:17' 2024-09-12T14:16:23.331 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:16:23.331 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:16:23.331 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-09-12T14:16:23.332 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:16:23.332 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-09-12T14:16:23.343 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary9: 2024-09-12T14:16:23.344 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 8c49b822-b2f9-4ec9-9270-a22fd92ddba3 2024-09-12T14:16:23.344 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+unknown 2024-09-12T14:16:23.344 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: remote image is not primary 2024-09-12T14:16:23.344 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T14:16:23.344 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 14:16:17' 2024-09-12T14:16:23.344 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T14:16:23.346 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary2: 2024-09-12T14:16:23.346 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 7d8bfe1f-b066-4c3e-95a4-c9c60426a59c 2024-09-12T14:16:23.346 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+unknown 2024-09-12T14:16:23.346 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: remote image is not primary 2024-09-12T14:16:23.346 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T14:16:23.346 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 14:16:17' 2024-09-12T14:16:23.346 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T14:16:23.347 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+unknown 2024-09-12T14:16:23.347 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T14:16:23.350 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: remote image is not primary 2024-09-12T14:16:23.350 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+unknown 2024-09-12T14:16:23.350 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T14:16:23.350 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T14:16:23.351 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T14:16:23.352 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T14:16:23.354 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T14:16:23.354 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T14:16:23.354 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: remote image is not primary 2024-09-12T14:16:23.355 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T14:16:23.356 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T14:16:23.357 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T14:16:23.357 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T14:16:23.357 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T14:16:23.358 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T14:16:23.360 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T14:16:23.360 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T14:16:23.362 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 14:16:17' 2024-09-12T14:16:23.362 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary9 up+unknown '2024-09-12 14:16:17' 2024-09-12T14:16:23.362 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:16:23.363 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:16:23.363 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T14:16:23.363 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:16:23.363 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 14:16:17' 2024-09-12T14:16:23.363 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:16:23.363 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:16:23.363 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-09-12T14:16:23.365 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T14:16:23.365 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T14:16:23.366 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T14:16:23.372 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 14:16:17' 2024-09-12T14:16:23.373 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary2 up+unknown '2024-09-12 14:16:17' 2024-09-12T14:16:23.373 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:16:23.373 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:16:23.373 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T14:16:23.373 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:16:23.373 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 14:16:17' 2024-09-12T14:16:23.373 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:16:23.373 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:16:23.373 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-09-12T14:16:23.483 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041934232mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary19b69126b-93e7-41c5-b8d9-d3784d684fd8up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:16:17image-primary10d6c75f78-0d2c-4921-8b6d-d39474248debup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:16:17image-primary27d8bfe1f-b066-4c3e-95a4-c9c60426a59cup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:16:17image-primary34f62e81b-1fb5-4b07-89a7-8a820b4d657dup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:16:17image-primary4f1c9b79b-6866-4e42-bd0f-25d91fd7fdf5up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:16:17image-primary5d03e84bc-1230-4374-9749-7cc5f3588093up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:16:17image-primary6b2ea2747-3b65-4670-9e03-38ab325e0695up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:16:17image-primary7639c39b5-6a6c-4696-8c6f-e70477f65d39up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:16:17image-primary88c5b00c9-9b25-4271-b4e1-865fb28703a0up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:16:17image-primary98c49b822-b2f9-4ec9-9270-a22fd92ddba3up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:16:17' 2024-09-12T14:16:23.483 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T14:16:23.484 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary7'\'']/last_update' 2024-09-12T14:16:23.486 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041934232mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary19b69126b-93e7-41c5-b8d9-d3784d684fd8up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:16:17image-primary10d6c75f78-0d2c-4921-8b6d-d39474248debup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:16:17image-primary27d8bfe1f-b066-4c3e-95a4-c9c60426a59cup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:16:17image-primary34f62e81b-1fb5-4b07-89a7-8a820b4d657dup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:16:17image-primary4f1c9b79b-6866-4e42-bd0f-25d91fd7fdf5up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:16:17image-primary5d03e84bc-1230-4374-9749-7cc5f3588093up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:16:17image-primary6b2ea2747-3b65-4670-9e03-38ab325e0695up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:16:17image-primary7639c39b5-6a6c-4696-8c6f-e70477f65d39up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:16:17image-primary88c5b00c9-9b25-4271-b4e1-865fb28703a0up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:16:17image-primary98c49b822-b2f9-4ec9-9270-a22fd92ddba3up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:16:17' 2024-09-12T14:16:23.487 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T14:16:23.487 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary10'\'']/last_update' 2024-09-12T14:16:23.490 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 14:16:17' 2024-09-12T14:16:23.491 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary7'\'']/state' 2024-09-12T14:16:23.493 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 14:16:17' 2024-09-12T14:16:23.494 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary10'\'']/state' 2024-09-12T14:16:23.497 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+unknown 2024-09-12T14:16:23.497 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T14:16:23.498 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+unknown 2024-09-12T14:16:23.500 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+unknown 2024-09-12T14:16:23.501 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T14:16:23.502 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T14:16:23.502 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:16:23.502 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:16:23.502 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster2 mirror image-primary7 up+unknown 2024-09-12T14:16:23.502 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T14:16:23.502 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:16:23.502 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T14:16:23.502 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:16:23.502 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:16:23.502 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:16:23.503 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:16:23.503 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T14:16:23.503 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+unknown 2024-09-12T14:16:23.504 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T14:16:23.505 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:16:23.505 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:16:23.505 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster2 mirror image-primary10 up+unknown 2024-09-12T14:16:23.506 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T14:16:23.506 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:16:23.506 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T14:16:23.506 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:16:23.506 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:16:23.506 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:16:23.506 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:16:23.506 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T14:16:23.545 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041934232mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary19b69126b-93e7-41c5-b8d9-d3784d684fd8up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:16:17image-primary10d6c75f78-0d2c-4921-8b6d-d39474248debup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:16:17image-primary27d8bfe1f-b066-4c3e-95a4-c9c60426a59cup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:16:17image-primary34f62e81b-1fb5-4b07-89a7-8a820b4d657dup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:16:17image-primary4f1c9b79b-6866-4e42-bd0f-25d91fd7fdf5up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:16:17image-primary5d03e84bc-1230-4374-9749-7cc5f3588093up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:16:17image-primary6b2ea2747-3b65-4670-9e03-38ab325e0695up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:16:17image-primary7639c39b5-6a6c-4696-8c6f-e70477f65d39up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:16:17image-primary88c5b00c9-9b25-4271-b4e1-865fb28703a0up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:16:17image-primary98c49b822-b2f9-4ec9-9270-a22fd92ddba3up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:16:17' 2024-09-12T14:16:23.546 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T14:16:23.546 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary9'\'']/last_update' 2024-09-12T14:16:23.551 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 14:16:17' 2024-09-12T14:16:23.551 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041934232mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary19b69126b-93e7-41c5-b8d9-d3784d684fd8up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:16:17image-primary10d6c75f78-0d2c-4921-8b6d-d39474248debup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:16:17image-primary27d8bfe1f-b066-4c3e-95a4-c9c60426a59cup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:16:17image-primary34f62e81b-1fb5-4b07-89a7-8a820b4d657dup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:16:17image-primary4f1c9b79b-6866-4e42-bd0f-25d91fd7fdf5up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:16:17image-primary5d03e84bc-1230-4374-9749-7cc5f3588093up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:16:17image-primary6b2ea2747-3b65-4670-9e03-38ab325e0695up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:16:17image-primary7639c39b5-6a6c-4696-8c6f-e70477f65d39up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:16:17image-primary88c5b00c9-9b25-4271-b4e1-865fb28703a0up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:16:17image-primary98c49b822-b2f9-4ec9-9270-a22fd92ddba3up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:16:17' 2024-09-12T14:16:23.551 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T14:16:23.552 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary9'\'']/state' 2024-09-12T14:16:23.552 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary2'\'']/last_update' 2024-09-12T14:16:23.555 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+unknown 2024-09-12T14:16:23.555 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 14:16:17' 2024-09-12T14:16:23.555 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T14:16:23.556 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary2'\'']/state' 2024-09-12T14:16:23.556 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+unknown 2024-09-12T14:16:23.558 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T14:16:23.558 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:16:23.558 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:16:23.558 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster2 mirror image-primary9 up+unknown 2024-09-12T14:16:23.558 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T14:16:23.559 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:16:23.559 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T14:16:23.559 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:16:23.559 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:16:23.559 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:16:23.559 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:16:23.559 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T14:16:23.560 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+unknown 2024-09-12T14:16:23.561 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T14:16:23.561 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+unknown 2024-09-12T14:16:23.563 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T14:16:23.563 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:16:23.564 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:16:23.564 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster2 mirror image-primary2 up+unknown 2024-09-12T14:16:23.564 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T14:16:23.564 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:16:23.564 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T14:16:23.564 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:16:23.564 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:16:23.564 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:16:23.564 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:16:23.564 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T14:16:23.624 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster2 mirror image-primary6 up+unknown '' '' 2024-09-12T14:16:23.624 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T14:16:23.624 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:16:23.624 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T14:16:23.624 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:16:23.624 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:16:23.624 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:16:23.624 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:16:23.625 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:16:23.625 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster2 mirror image status mirror/image-primary6 2024-09-12T14:16:23.691 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary6: 2024-09-12T14:16:23.691 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: b2ea2747-3b65-4670-9e03-38ab325e0695 2024-09-12T14:16:23.691 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+unknown 2024-09-12T14:16:23.691 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: remote image is not primary 2024-09-12T14:16:23.692 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T14:16:23.692 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 14:16:17' 2024-09-12T14:16:23.692 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T14:16:23.693 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+unknown 2024-09-12T14:16:23.694 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T14:16:23.696 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: remote image is not primary 2024-09-12T14:16:23.697 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T14:16:23.697 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T14:16:23.698 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T14:16:23.699 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T14:16:23.700 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T14:16:23.703 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T14:16:23.703 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T14:16:23.704 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T14:16:23.707 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 14:16:17' 2024-09-12T14:16:23.708 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster2 mirror image-primary6 up+unknown '2024-09-12 14:16:17' 2024-09-12T14:16:23.708 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T14:16:23.708 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:16:23.708 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T14:16:23.708 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:16:23.708 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 14:16:17' 2024-09-12T14:16:23.708 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:16:23.708 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:16:23.709 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2024-09-12T14:16:23.820 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041924231mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary19b69126b-93e7-41c5-b8d9-d3784d684fd8up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:16:17image-primary10d6c75f78-0d2c-4921-8b6d-d39474248debup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:16:17image-primary27d8bfe1f-b066-4c3e-95a4-c9c60426a59cup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:16:17image-primary34f62e81b-1fb5-4b07-89a7-8a820b4d657dup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:16:17image-primary4f1c9b79b-6866-4e42-bd0f-25d91fd7fdf5up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:16:17image-primary5d03e84bc-1230-4374-9749-7cc5f3588093up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:16:17image-primary6b2ea2747-3b65-4670-9e03-38ab325e0695up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:16:17image-primary7639c39b5-6a6c-4696-8c6f-e70477f65d39up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:16:17image-primary88c5b00c9-9b25-4271-b4e1-865fb28703a0up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:16:17image-primary98c49b822-b2f9-4ec9-9270-a22fd92ddba3up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:16:17' 2024-09-12T14:16:23.820 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T14:16:23.821 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary6'\'']/last_update' 2024-09-12T14:16:23.825 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 14:16:17' 2024-09-12T14:16:23.826 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary6'\'']/state' 2024-09-12T14:16:23.830 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+unknown 2024-09-12T14:16:23.831 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T14:16:23.831 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+unknown 2024-09-12T14:16:23.833 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T14:16:23.834 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:16:23.834 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:16:23.834 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ promote_image cluster2 mirror image-primary6 2024-09-12T14:16:23.834 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T14:16:23.834 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:16:23.834 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T14:16:23.834 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local force= 2024-09-12T14:16:23.834 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 mirror image promote mirror/image-primary6 2024-09-12T14:16:23.957 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Image promoted to primary 2024-09-12T14:16:23.958 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Image promoted to primary 2024-09-12T14:16:23.958 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Image promoted to primary 2024-09-12T14:16:23.975 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \n\b\d ]] 2024-09-12T14:16:23.975 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \k\r\b\d ]] 2024-09-12T14:16:23.976 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \n\b\d ]] 2024-09-12T14:16:23.976 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \k\r\b\d ]] 2024-09-12T14:16:23.976 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo rbd --cluster cluster2 device map -t krbd mirror/image-primary5 2024-09-12T14:16:23.976 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo rbd --cluster cluster2 device map -t krbd mirror/image-primary6 2024-09-12T14:16:23.977 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \n\b\d ]] 2024-09-12T14:16:23.977 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \k\r\b\d ]] 2024-09-12T14:16:23.978 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo rbd --cluster cluster2 device map -t krbd mirror/image-primary1 2024-09-12T14:16:24.127 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ dev=/dev/rbd1 2024-09-12T14:16:24.128 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ dev=/dev/rbd2 2024-09-12T14:16:24.128 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo md5sum /dev/rbd1 2024-09-12T14:16:24.129 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ awk '{print $1}' 2024-09-12T14:16:24.129 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ dev=/dev/rbd3 2024-09-12T14:16:24.131 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo md5sum /dev/rbd2 2024-09-12T14:16:24.131 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ awk '{print $1}' 2024-09-12T14:16:24.131 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo md5sum /dev/rbd3 2024-09-12T14:16:24.131 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ awk '{print $1}' 2024-09-12T14:16:24.199 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary8 up+unknown '' '' 2024-09-12T14:16:24.199 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:16:24.199 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:16:24.199 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T14:16:24.199 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:16:24.199 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:16:24.199 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:16:24.200 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:16:24.200 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:16:24.200 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary8 2024-09-12T14:16:24.285 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary8: 2024-09-12T14:16:24.286 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 8c5b00c9-9b25-4271-b4e1-865fb28703a0 2024-09-12T14:16:24.286 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+unknown 2024-09-12T14:16:24.286 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: remote image is not primary 2024-09-12T14:16:24.286 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T14:16:24.286 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 14:16:17' 2024-09-12T14:16:24.286 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T14:16:24.290 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+unknown 2024-09-12T14:16:24.290 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T14:16:24.295 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: remote image is not primary 2024-09-12T14:16:24.295 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T14:16:24.297 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T14:16:24.297 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T14:16:24.301 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T14:16:24.301 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T14:16:24.306 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T14:16:24.306 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T14:16:24.307 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T14:16:24.314 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 14:16:17' 2024-09-12T14:16:24.314 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary8 up+unknown '2024-09-12 14:16:17' 2024-09-12T14:16:24.314 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:16:24.314 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:16:24.315 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T14:16:24.315 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:16:24.315 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 14:16:17' 2024-09-12T14:16:24.315 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:16:24.315 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:16:24.315 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-09-12T14:16:24.451 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041934232mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary19b69126b-93e7-41c5-b8d9-d3784d684fd8up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:16:17image-primary10d6c75f78-0d2c-4921-8b6d-d39474248debup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:16:17image-primary27d8bfe1f-b066-4c3e-95a4-c9c60426a59cup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:16:17image-primary34f62e81b-1fb5-4b07-89a7-8a820b4d657dup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:16:17image-primary4f1c9b79b-6866-4e42-bd0f-25d91fd7fdf5up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:16:17image-primary5d03e84bc-1230-4374-9749-7cc5f3588093up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:16:17image-primary6b2ea2747-3b65-4670-9e03-38ab325e0695up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:16:17image-primary7639c39b5-6a6c-4696-8c6f-e70477f65d39up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:16:17image-primary88c5b00c9-9b25-4271-b4e1-865fb28703a0up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:16:17image-primary98c49b822-b2f9-4ec9-9270-a22fd92ddba3up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:16:17' 2024-09-12T14:16:24.451 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T14:16:24.451 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary8'\'']/last_update' 2024-09-12T14:16:24.456 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 14:16:17' 2024-09-12T14:16:24.457 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary8'\'']/state' 2024-09-12T14:16:24.461 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+unknown 2024-09-12T14:16:24.461 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T14:16:24.462 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+unknown 2024-09-12T14:16:24.464 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T14:16:24.465 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:16:24.465 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:16:24.465 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster2 mirror image-primary8 up+unknown 2024-09-12T14:16:24.466 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T14:16:24.466 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:16:24.466 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T14:16:24.466 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:16:24.466 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:16:24.466 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:16:24.466 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:16:24.466 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T14:16:24.505 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster2 mirror image-primary7 up+unknown '' '' 2024-09-12T14:16:24.505 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T14:16:24.505 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:16:24.505 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T14:16:24.505 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:16:24.505 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:16:24.505 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:16:24.505 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:16:24.506 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:16:24.506 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster2 mirror image status mirror/image-primary7 2024-09-12T14:16:24.508 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster2 mirror image-primary10 up+unknown '' '' 2024-09-12T14:16:24.508 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T14:16:24.508 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:16:24.508 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T14:16:24.508 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:16:24.508 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:16:24.509 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:16:24.509 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:16:24.509 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:16:24.509 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster2 mirror image status mirror/image-primary10 2024-09-12T14:16:24.561 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster2 mirror image-primary9 up+unknown '' '' 2024-09-12T14:16:24.561 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T14:16:24.562 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:16:24.562 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T14:16:24.562 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:16:24.562 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:16:24.562 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:16:24.562 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:16:24.562 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:16:24.562 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster2 mirror image status mirror/image-primary9 2024-09-12T14:16:24.566 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster2 mirror image-primary2 up+unknown '' '' 2024-09-12T14:16:24.568 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T14:16:24.568 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:16:24.568 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T14:16:24.568 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:16:24.568 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:16:24.568 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:16:24.568 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:16:24.573 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:16:24.573 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster2 mirror image status mirror/image-primary2 2024-09-12T14:16:24.627 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary7: 2024-09-12T14:16:24.627 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 639c39b5-6a6c-4696-8c6f-e70477f65d39 2024-09-12T14:16:24.627 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+unknown 2024-09-12T14:16:24.627 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: remote image is not primary 2024-09-12T14:16:24.627 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T14:16:24.627 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 14:16:17' 2024-09-12T14:16:24.627 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T14:16:24.634 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+unknown 2024-09-12T14:16:24.635 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T14:16:24.638 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: remote image is not primary 2024-09-12T14:16:24.640 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T14:16:24.640 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T14:16:24.641 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T14:16:24.645 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T14:16:24.645 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary10: 2024-09-12T14:16:24.645 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: d6c75f78-0d2c-4921-8b6d-d39474248deb 2024-09-12T14:16:24.645 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+unknown 2024-09-12T14:16:24.646 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: remote image is not primary 2024-09-12T14:16:24.646 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T14:16:24.646 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 14:16:17' 2024-09-12T14:16:24.646 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T14:16:24.646 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T14:16:24.649 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T14:16:24.650 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T14:16:24.652 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+unknown 2024-09-12T14:16:24.652 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T14:16:24.653 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T14:16:24.659 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: remote image is not primary 2024-09-12T14:16:24.659 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T14:16:24.660 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 14:16:17' 2024-09-12T14:16:24.660 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster2 mirror image-primary7 up+unknown '2024-09-12 14:16:17' 2024-09-12T14:16:24.660 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T14:16:24.661 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:16:24.661 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T14:16:24.661 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:16:24.661 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 14:16:17' 2024-09-12T14:16:24.661 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:16:24.661 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T14:16:24.662 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:16:24.662 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2024-09-12T14:16:24.662 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T14:16:24.666 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T14:16:24.667 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T14:16:24.672 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T14:16:24.673 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T14:16:24.674 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T14:16:24.683 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 14:16:17' 2024-09-12T14:16:24.683 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster2 mirror image-primary10 up+unknown '2024-09-12 14:16:17' 2024-09-12T14:16:24.683 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T14:16:24.683 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:16:24.683 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T14:16:24.683 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:16:24.683 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 14:16:17' 2024-09-12T14:16:24.684 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:16:24.684 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:16:24.684 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2024-09-12T14:16:24.684 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary9: 2024-09-12T14:16:24.685 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 8c49b822-b2f9-4ec9-9270-a22fd92ddba3 2024-09-12T14:16:24.685 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+unknown 2024-09-12T14:16:24.685 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: remote image is not primary 2024-09-12T14:16:24.685 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T14:16:24.685 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 14:16:17' 2024-09-12T14:16:24.685 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T14:16:24.688 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+unknown 2024-09-12T14:16:24.690 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T14:16:24.697 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: remote image is not primary 2024-09-12T14:16:24.698 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T14:16:24.700 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T14:16:24.701 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T14:16:24.706 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T14:16:24.708 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T14:16:24.716 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T14:16:24.716 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T14:16:24.718 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T14:16:24.727 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 14:16:17' 2024-09-12T14:16:24.728 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster2 mirror image-primary9 up+unknown '2024-09-12 14:16:17' 2024-09-12T14:16:24.728 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T14:16:24.728 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:16:24.728 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T14:16:24.728 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:16:24.728 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 14:16:17' 2024-09-12T14:16:24.728 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:16:24.729 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:16:24.729 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2024-09-12T14:16:24.729 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary2: 2024-09-12T14:16:24.729 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 7d8bfe1f-b066-4c3e-95a4-c9c60426a59c 2024-09-12T14:16:24.730 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+unknown 2024-09-12T14:16:24.730 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: remote image is not primary 2024-09-12T14:16:24.730 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T14:16:24.730 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 14:16:17' 2024-09-12T14:16:24.730 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T14:16:24.735 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+unknown 2024-09-12T14:16:24.735 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T14:16:24.740 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: remote image is not primary 2024-09-12T14:16:24.740 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T14:16:24.742 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T14:16:24.743 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T14:16:24.748 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T14:16:24.749 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T14:16:24.754 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T14:16:24.755 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T14:16:24.758 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T14:16:24.765 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 14:16:17' 2024-09-12T14:16:24.765 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster2 mirror image-primary2 up+unknown '2024-09-12 14:16:17' 2024-09-12T14:16:24.765 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T14:16:24.765 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:16:24.765 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T14:16:24.765 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:16:24.765 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 14:16:17' 2024-09-12T14:16:24.765 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:16:24.766 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:16:24.766 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2024-09-12T14:16:24.926 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041924231mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary19b69126b-93e7-41c5-b8d9-d3784d684fd8up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:16:17image-primary10d6c75f78-0d2c-4921-8b6d-d39474248debup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:16:17image-primary27d8bfe1f-b066-4c3e-95a4-c9c60426a59cup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:16:17image-primary34f62e81b-1fb5-4b07-89a7-8a820b4d657dup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:16:17image-primary4f1c9b79b-6866-4e42-bd0f-25d91fd7fdf5up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:16:17image-primary5d03e84bc-1230-4374-9749-7cc5f3588093up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:16:17image-primary6b2ea2747-3b65-4670-9e03-38ab325e0695up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:16:17image-primary7639c39b5-6a6c-4696-8c6f-e70477f65d39up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:16:17image-primary88c5b00c9-9b25-4271-b4e1-865fb28703a0up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:16:17image-primary98c49b822-b2f9-4ec9-9270-a22fd92ddba3up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:16:17' 2024-09-12T14:16:24.926 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T14:16:24.927 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary7'\'']/last_update' 2024-09-12T14:16:24.955 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 14:16:17' 2024-09-12T14:16:24.961 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary7'\'']/state' 2024-09-12T14:16:24.978 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041924231mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary19b69126b-93e7-41c5-b8d9-d3784d684fd8up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:16:17image-primary10d6c75f78-0d2c-4921-8b6d-d39474248debup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:16:17image-primary27d8bfe1f-b066-4c3e-95a4-c9c60426a59cup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:16:17image-primary34f62e81b-1fb5-4b07-89a7-8a820b4d657dup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:16:17image-primary4f1c9b79b-6866-4e42-bd0f-25d91fd7fdf5up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:16:17image-primary5d03e84bc-1230-4374-9749-7cc5f3588093up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:16:17image-primary6b2ea2747-3b65-4670-9e03-38ab325e0695up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:16:17image-primary7639c39b5-6a6c-4696-8c6f-e70477f65d39up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:16:17image-primary88c5b00c9-9b25-4271-b4e1-865fb28703a0up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:16:17image-primary98c49b822-b2f9-4ec9-9270-a22fd92ddba3up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:16:17' 2024-09-12T14:16:24.979 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T14:16:24.985 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary10'\'']/last_update' 2024-09-12T14:16:24.987 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+unknown 2024-09-12T14:16:24.991 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+unknown 2024-09-12T14:16:24.991 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T14:16:24.998 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T14:16:25.000 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:16:25.000 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:16:25.000 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ promote_image cluster2 mirror image-primary7 2024-09-12T14:16:25.000 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T14:16:25.000 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:16:25.000 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T14:16:25.000 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local force= 2024-09-12T14:16:25.000 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 mirror image promote mirror/image-primary7 2024-09-12T14:16:25.002 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 14:16:17' 2024-09-12T14:16:25.002 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary10'\'']/state' 2024-09-12T14:16:25.019 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+unknown 2024-09-12T14:16:25.020 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T14:16:25.021 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+unknown 2024-09-12T14:16:25.025 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T14:16:25.027 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:16:25.027 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:16:25.027 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ promote_image cluster2 mirror image-primary10 2024-09-12T14:16:25.027 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T14:16:25.027 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:16:25.027 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T14:16:25.027 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local force= 2024-09-12T14:16:25.027 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 mirror image promote mirror/image-primary10 2024-09-12T14:16:25.046 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041924231mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary19b69126b-93e7-41c5-b8d9-d3784d684fd8up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:16:17image-primary10d6c75f78-0d2c-4921-8b6d-d39474248debup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:16:17image-primary27d8bfe1f-b066-4c3e-95a4-c9c60426a59cup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:16:17image-primary34f62e81b-1fb5-4b07-89a7-8a820b4d657dup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:16:17image-primary4f1c9b79b-6866-4e42-bd0f-25d91fd7fdf5up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:16:17image-primary5d03e84bc-1230-4374-9749-7cc5f3588093up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:16:17image-primary6b2ea2747-3b65-4670-9e03-38ab325e0695up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:16:17image-primary7639c39b5-6a6c-4696-8c6f-e70477f65d39up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:16:17image-primary88c5b00c9-9b25-4271-b4e1-865fb28703a0up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:16:17image-primary98c49b822-b2f9-4ec9-9270-a22fd92ddba3up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:16:17' 2024-09-12T14:16:25.046 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T14:16:25.051 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary9'\'']/last_update' 2024-09-12T14:16:25.064 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 14:16:17' 2024-09-12T14:16:25.066 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary9'\'']/state' 2024-09-12T14:16:25.068 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary3 up+unknown '' '' 2024-09-12T14:16:25.068 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:16:25.068 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:16:25.068 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T14:16:25.068 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:16:25.068 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:16:25.068 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:16:25.068 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:16:25.069 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:16:25.069 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary3 2024-09-12T14:16:25.075 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+unknown 2024-09-12T14:16:25.076 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T14:16:25.077 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+unknown 2024-09-12T14:16:25.080 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T14:16:25.081 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:16:25.081 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:16:25.081 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ promote_image cluster2 mirror image-primary9 2024-09-12T14:16:25.081 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T14:16:25.081 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:16:25.081 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T14:16:25.082 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local force= 2024-09-12T14:16:25.082 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 mirror image promote mirror/image-primary9 2024-09-12T14:16:25.092 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041924231mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary19b69126b-93e7-41c5-b8d9-d3784d684fd8up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:16:17image-primary10d6c75f78-0d2c-4921-8b6d-d39474248debup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:16:17image-primary27d8bfe1f-b066-4c3e-95a4-c9c60426a59cup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:16:17image-primary34f62e81b-1fb5-4b07-89a7-8a820b4d657dup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:16:17image-primary4f1c9b79b-6866-4e42-bd0f-25d91fd7fdf5up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:16:17image-primary5d03e84bc-1230-4374-9749-7cc5f3588093up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:16:17image-primary6b2ea2747-3b65-4670-9e03-38ab325e0695up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:16:17image-primary7639c39b5-6a6c-4696-8c6f-e70477f65d39up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:16:17image-primary88c5b00c9-9b25-4271-b4e1-865fb28703a0up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:16:17image-primary98c49b822-b2f9-4ec9-9270-a22fd92ddba3up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:16:17' 2024-09-12T14:16:25.092 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T14:16:25.093 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary2'\'']/last_update' 2024-09-12T14:16:25.103 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 14:16:17' 2024-09-12T14:16:25.106 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary2'\'']/state' 2024-09-12T14:16:25.117 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+unknown 2024-09-12T14:16:25.120 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T14:16:25.121 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+unknown 2024-09-12T14:16:25.123 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T14:16:25.124 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:16:25.124 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:16:25.124 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ promote_image cluster2 mirror image-primary2 2024-09-12T14:16:25.124 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T14:16:25.125 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:16:25.125 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T14:16:25.125 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local force= 2024-09-12T14:16:25.125 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 mirror image promote mirror/image-primary2 2024-09-12T14:16:25.211 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary3: 2024-09-12T14:16:25.211 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 4f62e81b-1fb5-4b07-89a7-8a820b4d657d 2024-09-12T14:16:25.211 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+unknown 2024-09-12T14:16:25.211 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: remote image is not primary 2024-09-12T14:16:25.211 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T14:16:25.211 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 14:16:17' 2024-09-12T14:16:25.211 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T14:16:25.214 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+unknown 2024-09-12T14:16:25.215 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T14:16:25.219 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: remote image is not primary 2024-09-12T14:16:25.221 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T14:16:25.222 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T14:16:25.223 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T14:16:25.226 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T14:16:25.227 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T14:16:25.231 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T14:16:25.232 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T14:16:25.233 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T14:16:25.240 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 14:16:17' 2024-09-12T14:16:25.240 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary3 up+unknown '2024-09-12 14:16:17' 2024-09-12T14:16:25.241 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:16:25.241 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:16:25.241 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T14:16:25.241 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:16:25.241 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 14:16:17' 2024-09-12T14:16:25.241 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:16:25.242 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:16:25.242 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-09-12T14:16:25.372 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041934232mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary19b69126b-93e7-41c5-b8d9-d3784d684fd8up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:16:17image-primary10d6c75f78-0d2c-4921-8b6d-d39474248debup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:16:17image-primary27d8bfe1f-b066-4c3e-95a4-c9c60426a59cup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:16:17image-primary34f62e81b-1fb5-4b07-89a7-8a820b4d657dup+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:16:17image-primary4f1c9b79b-6866-4e42-bd0f-25d91fd7fdf5up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:16:17image-primary5d03e84bc-1230-4374-9749-7cc5f3588093up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:16:17image-primary6b2ea2747-3b65-4670-9e03-38ab325e0695up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:16:17image-primary7639c39b5-6a6c-4696-8c6f-e70477f65d39up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:16:17image-primary88c5b00c9-9b25-4271-b4e1-865fb28703a0up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:16:17image-primary98c49b822-b2f9-4ec9-9270-a22fd92ddba3up+unknownremote image is not primary41934232mirror.0smithi1302024-09-12 14:16:17' 2024-09-12T14:16:25.373 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T14:16:25.373 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary3'\'']/last_update' 2024-09-12T14:16:25.377 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 14:16:17' 2024-09-12T14:16:25.378 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary3'\'']/state' 2024-09-12T14:16:25.382 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+unknown 2024-09-12T14:16:25.383 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T14:16:25.384 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+unknown 2024-09-12T14:16:25.386 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T14:16:25.387 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:16:25.387 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:16:25.387 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster2 mirror image-primary3 up+unknown 2024-09-12T14:16:25.387 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T14:16:25.387 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:16:25.387 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T14:16:25.387 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:16:25.388 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:16:25.388 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:16:25.388 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:16:25.388 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T14:16:25.469 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster2 mirror image-primary8 up+unknown '' '' 2024-09-12T14:16:25.469 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T14:16:25.469 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:16:25.469 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T14:16:25.470 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:16:25.470 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:16:25.470 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:16:25.470 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:16:25.470 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:16:25.470 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster2 mirror image status mirror/image-primary8 2024-09-12T14:16:25.580 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary8: 2024-09-12T14:16:25.580 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 8c5b00c9-9b25-4271-b4e1-865fb28703a0 2024-09-12T14:16:25.580 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+unknown 2024-09-12T14:16:25.580 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: remote image is not primary 2024-09-12T14:16:25.580 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T14:16:25.581 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 14:16:17' 2024-09-12T14:16:25.581 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T14:16:25.585 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+unknown 2024-09-12T14:16:25.585 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T14:16:25.590 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: remote image is not primary 2024-09-12T14:16:25.590 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T14:16:25.592 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T14:16:25.592 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T14:16:25.596 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T14:16:25.596 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T14:16:25.601 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T14:16:25.601 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T14:16:25.603 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T14:16:25.609 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 14:16:17' 2024-09-12T14:16:25.609 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster2 mirror image-primary8 up+unknown '2024-09-12 14:16:17' 2024-09-12T14:16:25.609 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T14:16:25.609 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:16:25.609 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T14:16:25.609 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:16:25.609 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 14:16:17' 2024-09-12T14:16:25.610 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:16:25.610 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:16:25.610 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2024-09-12T14:16:25.788 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041924231mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary19b69126b-93e7-41c5-b8d9-d3784d684fd8up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:16:17image-primary10d6c75f78-0d2c-4921-8b6d-d39474248debup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:16:17image-primary27d8bfe1f-b066-4c3e-95a4-c9c60426a59cup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:16:17image-primary34f62e81b-1fb5-4b07-89a7-8a820b4d657dup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:16:17image-primary4f1c9b79b-6866-4e42-bd0f-25d91fd7fdf5up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:16:17image-primary5d03e84bc-1230-4374-9749-7cc5f3588093up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:16:17image-primary6b2ea2747-3b65-4670-9e03-38ab325e0695up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:16:17image-primary7639c39b5-6a6c-4696-8c6f-e70477f65d39up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:16:17image-primary88c5b00c9-9b25-4271-b4e1-865fb28703a0up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:16:17image-primary98c49b822-b2f9-4ec9-9270-a22fd92ddba3up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:16:17' 2024-09-12T14:16:25.788 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T14:16:25.789 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary8'\'']/last_update' 2024-09-12T14:16:25.793 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 14:16:17' 2024-09-12T14:16:25.794 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary8'\'']/state' 2024-09-12T14:16:25.798 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+unknown 2024-09-12T14:16:25.799 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T14:16:25.800 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+unknown 2024-09-12T14:16:25.802 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T14:16:25.803 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:16:25.803 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:16:25.803 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ promote_image cluster2 mirror image-primary8 2024-09-12T14:16:25.803 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T14:16:25.803 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:16:25.803 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T14:16:25.804 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local force= 2024-09-12T14:16:25.804 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 mirror image promote mirror/image-primary8 2024-09-12T14:16:26.015 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Image promoted to primary 2024-09-12T14:16:26.016 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Image promoted to primary 2024-09-12T14:16:26.017 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Image promoted to primary 2024-09-12T14:16:26.018 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Image promoted to primary 2024-09-12T14:16:26.024 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Image promoted to primary 2024-09-12T14:16:26.050 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \n\b\d ]] 2024-09-12T14:16:26.050 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \k\r\b\d ]] 2024-09-12T14:16:26.051 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \n\b\d ]] 2024-09-12T14:16:26.051 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \k\r\b\d ]] 2024-09-12T14:16:26.052 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo rbd --cluster cluster2 device map -t krbd mirror/image-primary8 2024-09-12T14:16:26.053 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo rbd --cluster cluster2 device map -t krbd mirror/image-primary9 2024-09-12T14:16:26.053 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \n\b\d ]] 2024-09-12T14:16:26.054 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \k\r\b\d ]] 2024-09-12T14:16:26.055 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo rbd --cluster cluster2 device map -t krbd mirror/image-primary2 2024-09-12T14:16:26.057 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \n\b\d ]] 2024-09-12T14:16:26.057 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \k\r\b\d ]] 2024-09-12T14:16:26.060 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo rbd --cluster cluster2 device map -t krbd mirror/image-primary7 2024-09-12T14:16:26.060 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \n\b\d ]] 2024-09-12T14:16:26.060 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \k\r\b\d ]] 2024-09-12T14:16:26.061 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo rbd --cluster cluster2 device map -t krbd mirror/image-primary10 2024-09-12T14:16:26.349 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ dev=/dev/rbd4 2024-09-12T14:16:26.352 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo md5sum /dev/rbd4 2024-09-12T14:16:26.353 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ awk '{print $1}' 2024-09-12T14:16:26.373 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ dev=/dev/rbd5 2024-09-12T14:16:26.378 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ awk '{print $1}' 2024-09-12T14:16:26.380 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo md5sum /dev/rbd5 2024-09-12T14:16:26.394 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster2 mirror image-primary3 up+unknown '' '' 2024-09-12T14:16:26.394 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T14:16:26.394 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:16:26.394 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T14:16:26.394 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:16:26.394 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:16:26.394 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:16:26.395 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:16:26.395 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:16:26.395 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster2 mirror image status mirror/image-primary3 2024-09-12T14:16:26.410 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ dev=/dev/rbd7 2024-09-12T14:16:26.411 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ dev=/dev/rbd6 2024-09-12T14:16:26.416 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo md5sum /dev/rbd6 2024-09-12T14:16:26.417 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo md5sum /dev/rbd7 2024-09-12T14:16:26.418 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ awk '{print $1}' 2024-09-12T14:16:26.418 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ awk '{print $1}' 2024-09-12T14:16:26.455 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ dev=/dev/rbd8 2024-09-12T14:16:26.459 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo md5sum /dev/rbd8 2024-09-12T14:16:26.460 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ awk '{print $1}' 2024-09-12T14:16:26.639 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary3: 2024-09-12T14:16:26.640 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 4f62e81b-1fb5-4b07-89a7-8a820b4d657d 2024-09-12T14:16:26.640 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+unknown 2024-09-12T14:16:26.640 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: remote image is not primary 2024-09-12T14:16:26.640 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T14:16:26.640 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 14:16:17' 2024-09-12T14:16:26.640 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+unknown' 2024-09-12T14:16:26.647 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+unknown 2024-09-12T14:16:26.657 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T14:16:26.661 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: remote image is not primary 2024-09-12T14:16:26.668 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T14:16:26.669 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T14:16:26.672 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T14:16:26.678 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T14:16:26.678 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T14:16:26.688 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T14:16:26.688 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T14:16:26.692 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T14:16:26.709 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 14:16:17' 2024-09-12T14:16:26.709 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster2 mirror image-primary3 up+unknown '2024-09-12 14:16:17' 2024-09-12T14:16:26.710 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T14:16:26.710 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:16:26.710 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T14:16:26.710 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+unknown 2024-09-12T14:16:26.710 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 14:16:17' 2024-09-12T14:16:26.710 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:16:26.713 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:16:26.713 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster2 mirror pool status mirror --verbose --format xml 2024-09-12T14:16:27.084 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041924231mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary19b69126b-93e7-41c5-b8d9-d3784d684fd8up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:16:17image-primary10d6c75f78-0d2c-4921-8b6d-d39474248debup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:16:17image-primary27d8bfe1f-b066-4c3e-95a4-c9c60426a59cup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:16:17image-primary34f62e81b-1fb5-4b07-89a7-8a820b4d657dup+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:16:17image-primary4f1c9b79b-6866-4e42-bd0f-25d91fd7fdf5up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:16:17image-primary5d03e84bc-1230-4374-9749-7cc5f3588093up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:16:17image-primary6b2ea2747-3b65-4670-9e03-38ab325e0695up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:16:17image-primary7639c39b5-6a6c-4696-8c6f-e70477f65d39up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:16:17image-primary88c5b00c9-9b25-4271-b4e1-865fb28703a0up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:16:17image-primary98c49b822-b2f9-4ec9-9270-a22fd92ddba3up+unknownremote image is not primary41924231mirror.0smithi1302024-09-12 14:16:17' 2024-09-12T14:16:27.084 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T14:16:27.086 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary3'\'']/last_update' 2024-09-12T14:16:27.099 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 14:16:17' 2024-09-12T14:16:27.104 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary3'\'']/state' 2024-09-12T14:16:27.121 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+unknown 2024-09-12T14:16:27.124 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+unknown 2024-09-12T14:16:27.126 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+unknown 2024-09-12T14:16:27.134 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+unknown 2024-09-12T14:16:27.134 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:16:27.134 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:16:27.134 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ promote_image cluster2 mirror image-primary3 2024-09-12T14:16:27.135 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T14:16:27.135 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:16:27.135 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T14:16:27.135 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local force= 2024-09-12T14:16:27.135 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 mirror image promote mirror/image-primary3 2024-09-12T14:16:28.043 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:Image promoted to primary 2024-09-12T14:16:28.092 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \n\b\d ]] 2024-09-12T14:16:28.093 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ krbd == \k\r\b\d ]] 2024-09-12T14:16:28.095 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo rbd --cluster cluster2 device map -t krbd mirror/image-primary3 2024-09-12T14:16:28.513 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ dev=/dev/rbd9 2024-09-12T14:16:28.518 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ awk '{print $1}' 2024-09-12T14:16:28.518 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sudo md5sum /dev/rbd9 2024-09-12T14:17:13.482 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ promote_md5=03c898af3f96b670136c283af7036e56 2024-09-12T14:17:13.482 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo rbd --cluster cluster2 device unmap -t krbd /dev/rbd0 2024-09-12T14:17:13.804 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ 03c898af3f96b670136c283af7036e56 != \0\3\c\8\9\8\a\f\3\f\9\6\b\6\7\0\1\3\6\c\2\8\3\a\f\7\0\3\6\e\5\6 ]] 2024-09-12T14:17:17.602 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ promote_md5=8e751ae38b30a1dbd73935dcc888ee87 2024-09-12T14:17:17.603 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo rbd --cluster cluster2 device unmap -t krbd /dev/rbd1 2024-09-12T14:17:17.920 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ 8e751ae38b30a1dbd73935dcc888ee87 != \8\e\7\5\1\a\e\3\8\b\3\0\a\1\d\b\d\7\3\9\3\5\d\c\c\8\8\8\e\e\8\7 ]] 2024-09-12T14:17:18.817 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ promote_md5=43be192ff1f4f2115159789cd54ac918 2024-09-12T14:17:18.818 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo rbd --cluster cluster2 device unmap -t krbd /dev/rbd3 2024-09-12T14:17:19.022 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ 43be192ff1f4f2115159789cd54ac918 != \4\3\b\e\1\9\2\f\f\1\f\4\f\2\1\1\5\1\5\9\7\8\9\c\d\5\4\a\c\9\1\8 ]] 2024-09-12T14:17:19.022 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${COMPARE_PIDS[@]} 2024-09-12T14:17:19.022 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 149432 2024-09-12T14:17:19.133 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ promote_md5=ef619673f0a848e6395c538696a537a7 2024-09-12T14:17:19.133 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo rbd --cluster cluster2 device unmap -t krbd /dev/rbd8 2024-09-12T14:17:19.287 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ ef619673f0a848e6395c538696a537a7 != \e\f\6\1\9\6\7\3\f\0\a\8\4\8\e\6\3\9\5\c\5\3\8\6\9\6\a\5\3\7\a\7 ]] 2024-09-12T14:17:19.288 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${COMPARE_PIDS[@]} 2024-09-12T14:17:19.288 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 149434 2024-09-12T14:17:19.553 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ promote_md5=e531d7f51e023d59799d74cb419aa852 2024-09-12T14:17:19.553 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo rbd --cluster cluster2 device unmap -t krbd /dev/rbd2 2024-09-12T14:17:19.821 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ e531d7f51e023d59799d74cb419aa852 != \e\5\3\1\d\7\f\5\1\e\0\2\3\d\5\9\7\9\9\d\7\4\c\b\4\1\9\a\a\8\5\2 ]] 2024-09-12T14:17:20.141 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ promote_md5=2c5c576a3e47d2ffc5f635ac482e1168 2024-09-12T14:17:20.141 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo rbd --cluster cluster2 device unmap -t krbd /dev/rbd5 2024-09-12T14:17:20.207 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ promote_md5=2f7e271f77b6f892aa604d1d6246a9e9 2024-09-12T14:17:20.207 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo rbd --cluster cluster2 device unmap -t krbd /dev/rbd7 2024-09-12T14:17:20.326 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ 2c5c576a3e47d2ffc5f635ac482e1168 != \2\c\5\c\5\7\6\a\3\e\4\7\d\2\f\f\c\5\f\6\3\5\a\c\4\8\2\e\1\1\6\8 ]] 2024-09-12T14:17:20.400 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ 2f7e271f77b6f892aa604d1d6246a9e9 != \2\f\7\e\2\7\1\f\7\7\b\6\f\8\9\2\a\a\6\0\4\d\1\d\6\2\4\6\a\9\e\9 ]] 2024-09-12T14:17:20.667 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ promote_md5=7756588463295cc74a757e122455f81a 2024-09-12T14:17:20.668 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo rbd --cluster cluster2 device unmap -t krbd /dev/rbd6 2024-09-12T14:17:20.794 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ 7756588463295cc74a757e122455f81a != \7\7\5\6\5\8\8\4\6\3\2\9\5\c\c\7\4\a\7\5\7\e\1\2\2\4\5\5\f\8\1\a ]] 2024-09-12T14:17:21.104 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ promote_md5=879019d4c94436f67080ba165eb700bd 2024-09-12T14:17:21.104 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo rbd --cluster cluster2 device unmap -t krbd /dev/rbd4 2024-09-12T14:17:21.207 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ 879019d4c94436f67080ba165eb700bd != \8\7\9\0\1\9\d\4\c\9\4\4\3\6\f\6\7\0\8\0\b\a\1\6\5\e\b\7\0\0\b\d ]] 2024-09-12T14:17:22.442 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ promote_md5=d83c2aa9b60f080de2d1a0b697f25201 2024-09-12T14:17:22.442 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sudo rbd --cluster cluster2 device unmap -t krbd /dev/rbd9 2024-09-12T14:17:22.539 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ [[ d83c2aa9b60f080de2d1a0b697f25201 != \d\8\3\c\2\a\a\9\b\6\0\f\0\8\0\d\e\2\d\1\a\0\b\6\9\7\f\2\5\2\0\1 ]] 2024-09-12T14:17:22.540 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${COMPARE_PIDS[@]} 2024-09-12T14:17:22.540 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 149436 2024-09-12T14:17:22.540 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${COMPARE_PIDS[@]} 2024-09-12T14:17:22.540 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 149438 2024-09-12T14:17:22.540 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${COMPARE_PIDS[@]} 2024-09-12T14:17:22.540 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 149440 2024-09-12T14:17:22.540 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${COMPARE_PIDS[@]} 2024-09-12T14:17:22.540 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 149442 2024-09-12T14:17:22.540 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${COMPARE_PIDS[@]} 2024-09-12T14:17:22.540 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 149444 2024-09-12T14:17:22.541 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${COMPARE_PIDS[@]} 2024-09-12T14:17:22.541 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 149445 2024-09-12T14:17:22.541 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for pid in ${COMPARE_PIDS[@]} 2024-09-12T14:17:22.541 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait 149449 2024-09-12T14:17:22.541 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ (( RET != 0 )) 2024-09-12T14:17:22.541 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T14:17:22.541 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ IMG=image-primary1 2024-09-12T14:17:22.541 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_replaying_status_in_pool_dir cluster1 mirror image-primary1 2024-09-12T14:17:22.541 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:17:22.541 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:17:22.541 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T14:17:22.541 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' snapshot = journal ']' 2024-09-12T14:17:22.542 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary1 up+replaying 2024-09-12T14:17:22.542 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:17:22.542 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:17:22.542 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T14:17:22.542 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T14:17:22.542 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:17:22.542 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:17:22.542 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:17:22.542 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T14:17:23.542 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary1 up+replaying '' '' 2024-09-12T14:17:23.542 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:17:23.542 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:17:23.542 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T14:17:23.542 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T14:17:23.542 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:17:23.542 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:17:23.543 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:17:23.543 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:17:23.543 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary1 2024-09-12T14:17:23.620 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary1: 2024-09-12T14:17:23.620 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 9b69126b-93e7-41c5-b8d9-d3784d684fd8 2024-09-12T14:17:23.620 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+replaying 2024-09-12T14:17:23.621 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: replaying, {"bytes_per_second":3168460.8,"bytes_per_snapshot":63369216.0,"last_snapshot_bytes":63369216,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726150583,"remote_snapshot_timestamp":1726150583,"replay_state":"idle"} 2024-09-12T14:17:23.621 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T14:17:23.621 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 14:17:17' 2024-09-12T14:17:23.621 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+replaying' 2024-09-12T14:17:23.623 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+replaying 2024-09-12T14:17:23.623 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T14:17:23.626 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: replaying, {"bytes_per_second":3168460.8,"bytes_per_snapshot":63369216.0,"last_snapshot_bytes":63369216,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726150583,"remote_snapshot_timestamp":1726150583,"replay_state":"idle"} 2024-09-12T14:17:23.626 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T14:17:23.627 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+replaying 2024-09-12T14:17:23.627 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T14:17:23.629 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+replaying 2024-09-12T14:17:23.629 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T14:17:23.632 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T14:17:23.632 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T14:17:23.633 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T14:17:23.637 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 14:17:17' 2024-09-12T14:17:23.637 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary1 up+replaying '2024-09-12 14:17:17' 2024-09-12T14:17:23.637 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:17:23.637 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:17:23.637 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T14:17:23.637 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T14:17:23.637 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 14:17:17' 2024-09-12T14:17:23.638 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:17:23.638 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:17:23.638 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-09-12T14:17:23.748 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING1041934232mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary19b69126b-93e7-41c5-b8d9-d3784d684fd8up+replayingreplaying, {"bytes_per_second":3168460.8,"bytes_per_snapshot":63369216.0,"last_snapshot_bytes":63369216,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726150583,"remote_snapshot_timestamp":1726150583,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 14:17:17image-primary10d6c75f78-0d2c-4921-8b6d-d39474248debup+replayingreplaying, {"bytes_per_second":3136921.6,"bytes_per_snapshot":62738432.0,"last_snapshot_bytes":62738432,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726150585,"remote_snapshot_timestamp":1726150585,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 14:17:17image-primary27d8bfe1f-b066-4c3e-95a4-c9c60426a59cup+replayingreplaying, {"bytes_per_second":3260825.6,"bytes_per_snapshot":65216512.0,"last_snapshot_bytes":65216512,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726150585,"remote_snapshot_timestamp":1726150585,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 14:17:17image-primary34f62e81b-1fb5-4b07-89a7-8a820b4d657dup+replayingreplaying, {"bytes_per_second":3365683.2,"bytes_per_snapshot":67313664.0,"last_snapshot_bytes":67313664,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726150587,"remote_snapshot_timestamp":1726150587,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 14:17:17image-primary4f1c9b79b-6866-4e42-bd0f-25d91fd7fdf5up+replayingreplaying, {"bytes_per_second":3313868.8,"bytes_per_snapshot":66277376.0,"last_snapshot_bytes":66277376,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726150580,"remote_snapshot_timestamp":1726150580,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 14:17:17image-primary5d03e84bc-1230-4374-9749-7cc5f3588093up+replayingreplaying, {"bytes_per_second":3244032.0,"bytes_per_snapshot":64880640.0,"last_snapshot_bytes":64880640,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726150583,"remote_snapshot_timestamp":1726150583,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 14:17:17image-primary6b2ea2747-3b65-4670-9e03-38ab325e0695up+replayingreplaying, {"bytes_per_second":3409100.8,"bytes_per_snapshot":68182016.0,"last_snapshot_bytes":68182016,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726150583,"remote_snapshot_timestamp":1726150583,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 14:17:17image-primary7639c39b5-6a6c-4696-8c6f-e70477f65d39up+replayingreplaying, {"bytes_per_second":3258982.4,"bytes_per_snapshot":65179648.0,"last_snapshot_bytes":65179648,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726150585,"remote_snapshot_timestamp":1726150585,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 14:17:17image-primary88c5b00c9-9b25-4271-b4e1-865fb28703a0up+replayingreplaying, {"bytes_per_second":3366707.2,"bytes_per_snapshot":67334144.0,"last_snapshot_bytes":67334144,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726150585,"remote_snapshot_timestamp":1726150585,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 14:17:17image-primary98c49b822-b2f9-4ec9-9270-a22fd92ddba3up+replayingreplaying, {"bytes_per_second":3383296.0,"bytes_per_snapshot":67665920.0,"last_snapshot_bytes":67665920,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726150585,"remote_snapshot_timestamp":1726150585,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 14:17:17' 2024-09-12T14:17:23.749 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T14:17:23.749 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary1'\'']/last_update' 2024-09-12T14:17:23.754 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 14:17:17' 2024-09-12T14:17:23.755 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary1'\'']/state' 2024-09-12T14:17:23.759 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+replaying 2024-09-12T14:17:23.759 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+replaying 2024-09-12T14:17:23.760 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+replaying 2024-09-12T14:17:23.762 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+replaying 2024-09-12T14:17:23.762 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:17:23.762 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:17:23.763 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ remove_image cluster2 mirror image-primary1 2024-09-12T14:17:23.763 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T14:17:23.763 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:17:23.763 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T14:17:23.763 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 snap purge mirror/image-primary1 2024-09-12T14:17:23.827 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 rm mirror/image-primary1 2024-09-12T14:17:23.984 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T14:17:23.984+0000 7f67e5200640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x5605560fd380 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-09-12T14:17:23.984 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T14:17:23.984+0000 7f67ed400640 -1 rbd::mirror::ImageReplayer: 0x560556cef180 [3/9b69126b-93e7-41c5-b8d9-d3784d684fd8] operator(): start failed: (85) Interrupted system call should be restarted 2024-09-12T14:17:25.128 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-09-12T14:17:25.134 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_image_removal cluster1 mirror image-primary1 2024-09-12T14:17:25.134 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:17:25.134 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:17:25.134 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary1 2024-09-12T14:17:25.134 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:17:25.135 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 ls mirror 2024-09-12T14:17:25.135 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep -wq image-primary1 2024-09-12T14:17:25.185 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:17:25.185 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rm -rf test-primary1 2024-09-12T14:17:25.268 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T14:17:25.268 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ IMG=image-primary2 2024-09-12T14:17:25.268 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_replaying_status_in_pool_dir cluster1 mirror image-primary2 2024-09-12T14:17:25.268 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:17:25.268 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:17:25.268 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T14:17:25.268 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' snapshot = journal ']' 2024-09-12T14:17:25.268 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary2 up+replaying 2024-09-12T14:17:25.269 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:17:25.269 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:17:25.269 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T14:17:25.269 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T14:17:25.269 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:17:25.269 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:17:25.269 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:17:25.269 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T14:17:26.270 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary2 up+replaying '' '' 2024-09-12T14:17:26.270 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:17:26.270 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:17:26.270 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T14:17:26.270 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T14:17:26.270 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:17:26.270 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:17:26.271 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:17:26.271 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:17:26.271 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary2 2024-09-12T14:17:26.353 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary2: 2024-09-12T14:17:26.353 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 7d8bfe1f-b066-4c3e-95a4-c9c60426a59c 2024-09-12T14:17:26.353 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+replaying 2024-09-12T14:17:26.353 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: replaying, {"bytes_per_second":3260825.6,"bytes_per_snapshot":65216512.0,"last_snapshot_bytes":65216512,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726150585,"remote_snapshot_timestamp":1726150585,"replay_state":"idle"} 2024-09-12T14:17:26.353 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T14:17:26.353 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 14:17:17' 2024-09-12T14:17:26.353 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+replaying' 2024-09-12T14:17:26.355 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+replaying 2024-09-12T14:17:26.355 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T14:17:26.358 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: replaying, {"bytes_per_second":3260825.6,"bytes_per_snapshot":65216512.0,"last_snapshot_bytes":65216512,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726150585,"remote_snapshot_timestamp":1726150585,"replay_state":"idle"} 2024-09-12T14:17:26.358 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T14:17:26.359 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+replaying 2024-09-12T14:17:26.359 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T14:17:26.361 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+replaying 2024-09-12T14:17:26.362 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T14:17:26.364 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T14:17:26.365 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T14:17:26.366 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T14:17:26.369 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 14:17:17' 2024-09-12T14:17:26.370 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary2 up+replaying '2024-09-12 14:17:17' 2024-09-12T14:17:26.370 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:17:26.370 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:17:26.370 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T14:17:26.370 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T14:17:26.370 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 14:17:17' 2024-09-12T14:17:26.370 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:17:26.370 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:17:26.371 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-09-12T14:17:26.479 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING941934232mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary10d6c75f78-0d2c-4921-8b6d-d39474248debup+replayingreplaying, {"bytes_per_second":3136921.6,"bytes_per_snapshot":62738432.0,"last_snapshot_bytes":62738432,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726150585,"remote_snapshot_timestamp":1726150585,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 14:17:17image-primary27d8bfe1f-b066-4c3e-95a4-c9c60426a59cup+replayingreplaying, {"bytes_per_second":3260825.6,"bytes_per_snapshot":65216512.0,"last_snapshot_bytes":65216512,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726150585,"remote_snapshot_timestamp":1726150585,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 14:17:17image-primary34f62e81b-1fb5-4b07-89a7-8a820b4d657dup+replayingreplaying, {"bytes_per_second":3365683.2,"bytes_per_snapshot":67313664.0,"last_snapshot_bytes":67313664,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726150587,"remote_snapshot_timestamp":1726150587,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 14:17:17image-primary4f1c9b79b-6866-4e42-bd0f-25d91fd7fdf5up+replayingreplaying, {"bytes_per_second":3313868.8,"bytes_per_snapshot":66277376.0,"last_snapshot_bytes":66277376,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726150580,"remote_snapshot_timestamp":1726150580,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 14:17:17image-primary5d03e84bc-1230-4374-9749-7cc5f3588093up+replayingreplaying, {"bytes_per_second":3244032.0,"bytes_per_snapshot":64880640.0,"last_snapshot_bytes":64880640,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726150583,"remote_snapshot_timestamp":1726150583,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 14:17:17image-primary6b2ea2747-3b65-4670-9e03-38ab325e0695up+replayingreplaying, {"bytes_per_second":3409100.8,"bytes_per_snapshot":68182016.0,"last_snapshot_bytes":68182016,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726150583,"remote_snapshot_timestamp":1726150583,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 14:17:17image-primary7639c39b5-6a6c-4696-8c6f-e70477f65d39up+replayingreplaying, {"bytes_per_second":3258982.4,"bytes_per_snapshot":65179648.0,"last_snapshot_bytes":65179648,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726150585,"remote_snapshot_timestamp":1726150585,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 14:17:17image-primary88c5b00c9-9b25-4271-b4e1-865fb28703a0up+replayingreplaying, {"bytes_per_second":3366707.2,"bytes_per_snapshot":67334144.0,"last_snapshot_bytes":67334144,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726150585,"remote_snapshot_timestamp":1726150585,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 14:17:17image-primary98c49b822-b2f9-4ec9-9270-a22fd92ddba3up+replayingreplaying, {"bytes_per_second":3383296.0,"bytes_per_snapshot":67665920.0,"last_snapshot_bytes":67665920,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726150585,"remote_snapshot_timestamp":1726150585,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 14:17:17' 2024-09-12T14:17:26.480 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T14:17:26.480 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary2'\'']/last_update' 2024-09-12T14:17:26.484 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 14:17:17' 2024-09-12T14:17:26.485 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary2'\'']/state' 2024-09-12T14:17:26.488 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+replaying 2024-09-12T14:17:26.489 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+replaying 2024-09-12T14:17:26.489 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+replaying 2024-09-12T14:17:26.491 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+replaying 2024-09-12T14:17:26.492 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:17:26.492 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:17:26.492 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ remove_image cluster2 mirror image-primary2 2024-09-12T14:17:26.492 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T14:17:26.492 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:17:26.492 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T14:17:26.492 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 snap purge mirror/image-primary2 2024-09-12T14:17:26.553 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 rm mirror/image-primary2 2024-09-12T14:17:26.697 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T14:17:26.698+0000 7f67e5200640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x56055ba3e000 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-09-12T14:17:26.697 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T14:17:26.698+0000 7f67ed400640 -1 rbd::mirror::ImageReplayer: 0x56055c198500 [3/7d8bfe1f-b066-4c3e-95a4-c9c60426a59c] operator(): start failed: (85) Interrupted system call should be restarted 2024-09-12T14:17:28.141 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-09-12T14:17:28.146 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_image_removal cluster1 mirror image-primary2 2024-09-12T14:17:28.146 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:17:28.147 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:17:28.147 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary2 2024-09-12T14:17:28.147 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:17:28.147 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 ls mirror 2024-09-12T14:17:28.147 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep -wq image-primary2 2024-09-12T14:17:28.198 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:17:28.198 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rm -rf test-primary2 2024-09-12T14:17:28.201 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T14:17:28.201 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ IMG=image-primary3 2024-09-12T14:17:28.201 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_replaying_status_in_pool_dir cluster1 mirror image-primary3 2024-09-12T14:17:28.201 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:17:28.202 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:17:28.202 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T14:17:28.202 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' snapshot = journal ']' 2024-09-12T14:17:28.202 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary3 up+replaying 2024-09-12T14:17:28.202 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:17:28.202 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:17:28.202 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T14:17:28.202 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T14:17:28.202 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:17:28.202 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:17:28.202 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:17:28.202 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T14:17:29.203 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary3 up+replaying '' '' 2024-09-12T14:17:29.203 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:17:29.203 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:17:29.203 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T14:17:29.203 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T14:17:29.204 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:17:29.204 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:17:29.204 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:17:29.204 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:17:29.204 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary3 2024-09-12T14:17:29.284 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary3: 2024-09-12T14:17:29.284 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 4f62e81b-1fb5-4b07-89a7-8a820b4d657d 2024-09-12T14:17:29.284 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+replaying 2024-09-12T14:17:29.284 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: replaying, {"bytes_per_second":3365683.2,"bytes_per_snapshot":67313664.0,"last_snapshot_bytes":67313664,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726150587,"remote_snapshot_timestamp":1726150587,"replay_state":"idle"} 2024-09-12T14:17:29.284 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T14:17:29.284 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 14:17:17' 2024-09-12T14:17:29.285 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+replaying' 2024-09-12T14:17:29.286 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+replaying 2024-09-12T14:17:29.287 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T14:17:29.289 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: replaying, {"bytes_per_second":3365683.2,"bytes_per_snapshot":67313664.0,"last_snapshot_bytes":67313664,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726150587,"remote_snapshot_timestamp":1726150587,"replay_state":"idle"} 2024-09-12T14:17:29.289 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T14:17:29.290 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+replaying 2024-09-12T14:17:29.290 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T14:17:29.292 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+replaying 2024-09-12T14:17:29.292 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T14:17:29.295 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T14:17:29.295 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T14:17:29.296 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T14:17:29.300 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 14:17:17' 2024-09-12T14:17:29.300 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary3 up+replaying '2024-09-12 14:17:17' 2024-09-12T14:17:29.300 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:17:29.300 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:17:29.300 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T14:17:29.300 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T14:17:29.300 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 14:17:17' 2024-09-12T14:17:29.300 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:17:29.301 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:17:29.301 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-09-12T14:17:29.408 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING841934232mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary10d6c75f78-0d2c-4921-8b6d-d39474248debup+replayingreplaying, {"bytes_per_second":3136921.6,"bytes_per_snapshot":62738432.0,"last_snapshot_bytes":62738432,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726150585,"remote_snapshot_timestamp":1726150585,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 14:17:17image-primary34f62e81b-1fb5-4b07-89a7-8a820b4d657dup+replayingreplaying, {"bytes_per_second":3365683.2,"bytes_per_snapshot":67313664.0,"last_snapshot_bytes":67313664,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726150587,"remote_snapshot_timestamp":1726150587,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 14:17:17image-primary4f1c9b79b-6866-4e42-bd0f-25d91fd7fdf5up+replayingreplaying, {"bytes_per_second":3313868.8,"bytes_per_snapshot":66277376.0,"last_snapshot_bytes":66277376,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726150580,"remote_snapshot_timestamp":1726150580,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 14:17:17image-primary5d03e84bc-1230-4374-9749-7cc5f3588093up+replayingreplaying, {"bytes_per_second":3244032.0,"bytes_per_snapshot":64880640.0,"last_snapshot_bytes":64880640,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726150583,"remote_snapshot_timestamp":1726150583,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 14:17:17image-primary6b2ea2747-3b65-4670-9e03-38ab325e0695up+replayingreplaying, {"bytes_per_second":3409100.8,"bytes_per_snapshot":68182016.0,"last_snapshot_bytes":68182016,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726150583,"remote_snapshot_timestamp":1726150583,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 14:17:17image-primary7639c39b5-6a6c-4696-8c6f-e70477f65d39up+replayingreplaying, {"bytes_per_second":3258982.4,"bytes_per_snapshot":65179648.0,"last_snapshot_bytes":65179648,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726150585,"remote_snapshot_timestamp":1726150585,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 14:17:17image-primary88c5b00c9-9b25-4271-b4e1-865fb28703a0up+replayingreplaying, {"bytes_per_second":3366707.2,"bytes_per_snapshot":67334144.0,"last_snapshot_bytes":67334144,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726150585,"remote_snapshot_timestamp":1726150585,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 14:17:17image-primary98c49b822-b2f9-4ec9-9270-a22fd92ddba3up+replayingreplaying, {"bytes_per_second":3383296.0,"bytes_per_snapshot":67665920.0,"last_snapshot_bytes":67665920,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726150585,"remote_snapshot_timestamp":1726150585,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 14:17:17' 2024-09-12T14:17:29.408 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T14:17:29.409 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary3'\'']/last_update' 2024-09-12T14:17:29.413 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 14:17:17' 2024-09-12T14:17:29.413 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary3'\'']/state' 2024-09-12T14:17:29.417 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+replaying 2024-09-12T14:17:29.418 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+replaying 2024-09-12T14:17:29.418 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+replaying 2024-09-12T14:17:29.420 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+replaying 2024-09-12T14:17:29.421 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:17:29.421 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:17:29.421 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ remove_image cluster2 mirror image-primary3 2024-09-12T14:17:29.421 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T14:17:29.421 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:17:29.421 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T14:17:29.421 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 snap purge mirror/image-primary3 2024-09-12T14:17:29.481 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 rm mirror/image-primary3 2024-09-12T14:17:29.667 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T14:17:29.668+0000 7f67e4800640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x56055bce9a00 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-09-12T14:17:29.667 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T14:17:29.668+0000 7f67ed400640 -1 rbd::mirror::ImageReplayer: 0x56055c198780 [3/4f62e81b-1fb5-4b07-89a7-8a820b4d657d] operator(): start failed: (85) Interrupted system call should be restarted 2024-09-12T14:17:30.733 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-09-12T14:17:30.739 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_image_removal cluster1 mirror image-primary3 2024-09-12T14:17:30.739 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:17:30.739 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:17:30.740 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary3 2024-09-12T14:17:30.740 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:17:30.740 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 ls mirror 2024-09-12T14:17:30.740 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep -wq image-primary3 2024-09-12T14:17:30.797 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:17:30.797 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rm -rf test-primary3 2024-09-12T14:17:30.799 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T14:17:30.800 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ IMG=image-primary4 2024-09-12T14:17:30.800 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_replaying_status_in_pool_dir cluster1 mirror image-primary4 2024-09-12T14:17:30.800 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:17:30.800 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:17:30.800 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T14:17:30.800 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' snapshot = journal ']' 2024-09-12T14:17:30.800 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary4 up+replaying 2024-09-12T14:17:30.800 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:17:30.800 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:17:30.800 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T14:17:30.800 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T14:17:30.800 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:17:30.801 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:17:30.801 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:17:30.801 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T14:17:31.802 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary4 up+replaying '' '' 2024-09-12T14:17:31.802 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:17:31.802 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:17:31.802 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T14:17:31.802 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T14:17:31.802 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:17:31.802 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:17:31.803 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:17:31.803 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:17:31.803 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary4 2024-09-12T14:17:31.906 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary4: 2024-09-12T14:17:31.906 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: f1c9b79b-6866-4e42-bd0f-25d91fd7fdf5 2024-09-12T14:17:31.906 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+replaying 2024-09-12T14:17:31.906 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: replaying, {"bytes_per_second":3313868.8,"bytes_per_snapshot":66277376.0,"last_snapshot_bytes":66277376,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726150580,"remote_snapshot_timestamp":1726150580,"replay_state":"idle"} 2024-09-12T14:17:31.906 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T14:17:31.906 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 14:17:17' 2024-09-12T14:17:31.906 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+replaying' 2024-09-12T14:17:31.908 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+replaying 2024-09-12T14:17:31.908 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T14:17:31.911 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: replaying, {"bytes_per_second":3313868.8,"bytes_per_snapshot":66277376.0,"last_snapshot_bytes":66277376,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726150580,"remote_snapshot_timestamp":1726150580,"replay_state":"idle"} 2024-09-12T14:17:31.911 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T14:17:31.912 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+replaying 2024-09-12T14:17:31.912 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T14:17:31.914 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+replaying 2024-09-12T14:17:31.914 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T14:17:31.917 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T14:17:31.917 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T14:17:31.918 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T14:17:31.921 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 14:17:17' 2024-09-12T14:17:31.922 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary4 up+replaying '2024-09-12 14:17:17' 2024-09-12T14:17:31.922 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:17:31.922 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:17:31.922 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T14:17:31.922 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T14:17:31.922 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 14:17:17' 2024-09-12T14:17:31.922 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:17:31.922 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:17:31.922 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-09-12T14:17:32.070 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING741934232mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary10d6c75f78-0d2c-4921-8b6d-d39474248debup+replayingreplaying, {"bytes_per_second":3136921.6,"bytes_per_snapshot":62738432.0,"last_snapshot_bytes":62738432,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726150585,"remote_snapshot_timestamp":1726150585,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 14:17:17image-primary4f1c9b79b-6866-4e42-bd0f-25d91fd7fdf5up+replayingreplaying, {"bytes_per_second":3313868.8,"bytes_per_snapshot":66277376.0,"last_snapshot_bytes":66277376,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726150580,"remote_snapshot_timestamp":1726150580,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 14:17:17image-primary5d03e84bc-1230-4374-9749-7cc5f3588093up+replayingreplaying, {"bytes_per_second":3244032.0,"bytes_per_snapshot":64880640.0,"last_snapshot_bytes":64880640,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726150583,"remote_snapshot_timestamp":1726150583,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 14:17:17image-primary6b2ea2747-3b65-4670-9e03-38ab325e0695up+replayingreplaying, {"bytes_per_second":3409100.8,"bytes_per_snapshot":68182016.0,"last_snapshot_bytes":68182016,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726150583,"remote_snapshot_timestamp":1726150583,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 14:17:17image-primary7639c39b5-6a6c-4696-8c6f-e70477f65d39up+replayingreplaying, {"bytes_per_second":3258982.4,"bytes_per_snapshot":65179648.0,"last_snapshot_bytes":65179648,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726150585,"remote_snapshot_timestamp":1726150585,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 14:17:17image-primary88c5b00c9-9b25-4271-b4e1-865fb28703a0up+replayingreplaying, {"bytes_per_second":3366707.2,"bytes_per_snapshot":67334144.0,"last_snapshot_bytes":67334144,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726150585,"remote_snapshot_timestamp":1726150585,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 14:17:17image-primary98c49b822-b2f9-4ec9-9270-a22fd92ddba3up+replayingreplaying, {"bytes_per_second":3383296.0,"bytes_per_snapshot":67665920.0,"last_snapshot_bytes":67665920,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726150585,"remote_snapshot_timestamp":1726150585,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 14:17:17' 2024-09-12T14:17:32.071 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T14:17:32.071 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary4'\'']/last_update' 2024-09-12T14:17:32.075 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 14:17:17' 2024-09-12T14:17:32.076 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary4'\'']/state' 2024-09-12T14:17:32.080 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+replaying 2024-09-12T14:17:32.080 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+replaying 2024-09-12T14:17:32.081 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+replaying 2024-09-12T14:17:32.082 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+replaying 2024-09-12T14:17:32.083 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:17:32.083 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:17:32.083 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ remove_image cluster2 mirror image-primary4 2024-09-12T14:17:32.083 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T14:17:32.083 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:17:32.084 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T14:17:32.084 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 snap purge mirror/image-primary4 2024-09-12T14:17:32.146 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 rm mirror/image-primary4 2024-09-12T14:17:32.376 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T14:17:32.376+0000 7f67e5200640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x56055bce8b60 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-09-12T14:17:32.376 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T14:17:32.376+0000 7f67eca00640 -1 rbd::mirror::ImageReplayer: 0x56055c199680 [3/f1c9b79b-6866-4e42-bd0f-25d91fd7fdf5] operator(): start failed: (85) Interrupted system call should be restarted 2024-09-12T14:17:33.715 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-09-12T14:17:33.721 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_image_removal cluster1 mirror image-primary4 2024-09-12T14:17:33.721 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:17:33.721 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:17:33.721 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary4 2024-09-12T14:17:33.722 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:17:33.722 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 ls mirror 2024-09-12T14:17:33.722 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep -wq image-primary4 2024-09-12T14:17:33.770 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:17:33.770 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rm -rf test-primary4 2024-09-12T14:17:33.773 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T14:17:33.773 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ IMG=image-primary5 2024-09-12T14:17:33.773 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_replaying_status_in_pool_dir cluster1 mirror image-primary5 2024-09-12T14:17:33.774 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:17:33.774 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:17:33.774 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T14:17:33.774 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' snapshot = journal ']' 2024-09-12T14:17:33.774 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary5 up+replaying 2024-09-12T14:17:33.774 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:17:33.774 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:17:33.774 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T14:17:33.774 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T14:17:33.774 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:17:33.774 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:17:33.774 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:17:33.774 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T14:17:34.776 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary5 up+replaying '' '' 2024-09-12T14:17:34.776 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:17:34.776 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:17:34.776 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T14:17:34.776 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T14:17:34.776 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:17:34.776 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:17:34.776 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:17:34.777 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:17:34.777 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary5 2024-09-12T14:17:34.863 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary5: 2024-09-12T14:17:34.863 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: d03e84bc-1230-4374-9749-7cc5f3588093 2024-09-12T14:17:34.863 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+replaying 2024-09-12T14:17:34.863 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: replaying, {"bytes_per_second":3244032.0,"bytes_per_snapshot":64880640.0,"last_snapshot_bytes":64880640,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726150583,"remote_snapshot_timestamp":1726150583,"replay_state":"idle"} 2024-09-12T14:17:34.863 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T14:17:34.863 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 14:17:17' 2024-09-12T14:17:34.863 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+replaying' 2024-09-12T14:17:34.865 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+replaying 2024-09-12T14:17:34.866 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T14:17:34.868 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: replaying, {"bytes_per_second":3244032.0,"bytes_per_snapshot":64880640.0,"last_snapshot_bytes":64880640,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726150583,"remote_snapshot_timestamp":1726150583,"replay_state":"idle"} 2024-09-12T14:17:34.869 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T14:17:34.869 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+replaying 2024-09-12T14:17:34.870 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T14:17:34.872 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+replaying 2024-09-12T14:17:34.872 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T14:17:34.875 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T14:17:34.875 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T14:17:34.876 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T14:17:34.880 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 14:17:17' 2024-09-12T14:17:34.880 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary5 up+replaying '2024-09-12 14:17:17' 2024-09-12T14:17:34.880 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:17:34.880 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:17:34.880 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T14:17:34.880 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T14:17:34.881 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 14:17:17' 2024-09-12T14:17:34.881 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:17:34.881 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:17:34.881 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-09-12T14:17:34.979 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING641934232mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary10d6c75f78-0d2c-4921-8b6d-d39474248debup+replayingreplaying, {"bytes_per_second":3136921.6,"bytes_per_snapshot":62738432.0,"last_snapshot_bytes":62738432,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726150585,"remote_snapshot_timestamp":1726150585,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 14:17:17image-primary5d03e84bc-1230-4374-9749-7cc5f3588093up+replayingreplaying, {"bytes_per_second":3244032.0,"bytes_per_snapshot":64880640.0,"last_snapshot_bytes":64880640,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726150583,"remote_snapshot_timestamp":1726150583,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 14:17:17image-primary6b2ea2747-3b65-4670-9e03-38ab325e0695up+replayingreplaying, {"bytes_per_second":3409100.8,"bytes_per_snapshot":68182016.0,"last_snapshot_bytes":68182016,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726150583,"remote_snapshot_timestamp":1726150583,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 14:17:17image-primary7639c39b5-6a6c-4696-8c6f-e70477f65d39up+replayingreplaying, {"bytes_per_second":3258982.4,"bytes_per_snapshot":65179648.0,"last_snapshot_bytes":65179648,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726150585,"remote_snapshot_timestamp":1726150585,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 14:17:17image-primary88c5b00c9-9b25-4271-b4e1-865fb28703a0up+replayingreplaying, {"bytes_per_second":3366707.2,"bytes_per_snapshot":67334144.0,"last_snapshot_bytes":67334144,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726150585,"remote_snapshot_timestamp":1726150585,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 14:17:17image-primary98c49b822-b2f9-4ec9-9270-a22fd92ddba3up+replayingreplaying, {"bytes_per_second":3383296.0,"bytes_per_snapshot":67665920.0,"last_snapshot_bytes":67665920,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726150585,"remote_snapshot_timestamp":1726150585,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 14:17:17' 2024-09-12T14:17:34.979 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T14:17:34.979 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary5'\'']/last_update' 2024-09-12T14:17:34.983 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 14:17:17' 2024-09-12T14:17:34.984 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary5'\'']/state' 2024-09-12T14:17:34.988 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+replaying 2024-09-12T14:17:34.989 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+replaying 2024-09-12T14:17:34.989 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+replaying 2024-09-12T14:17:34.991 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+replaying 2024-09-12T14:17:34.992 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:17:34.992 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:17:34.992 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ remove_image cluster2 mirror image-primary5 2024-09-12T14:17:34.992 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T14:17:34.992 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:17:34.992 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T14:17:34.992 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 snap purge mirror/image-primary5 2024-09-12T14:17:35.053 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 rm mirror/image-primary5 2024-09-12T14:17:35.202 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T14:17:35.202+0000 7f67e5200640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x560556e1a9c0 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-09-12T14:17:35.202 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T14:17:35.202+0000 7f67ed400640 -1 rbd::mirror::ImageReplayer: 0x5605561ac280 [3/d03e84bc-1230-4374-9749-7cc5f3588093] operator(): start failed: (85) Interrupted system call should be restarted 2024-09-12T14:17:36.750 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-09-12T14:17:36.757 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_image_removal cluster1 mirror image-primary5 2024-09-12T14:17:36.757 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:17:36.757 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:17:36.757 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary5 2024-09-12T14:17:36.757 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:17:36.758 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 ls mirror 2024-09-12T14:17:36.758 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep -wq image-primary5 2024-09-12T14:17:36.808 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:17:36.808 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rm -rf test-primary5 2024-09-12T14:17:36.811 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T14:17:36.811 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ IMG=image-primary6 2024-09-12T14:17:36.811 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_replaying_status_in_pool_dir cluster1 mirror image-primary6 2024-09-12T14:17:36.811 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:17:36.811 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:17:36.811 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T14:17:36.811 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' snapshot = journal ']' 2024-09-12T14:17:36.811 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary6 up+replaying 2024-09-12T14:17:36.811 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:17:36.811 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:17:36.811 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T14:17:36.811 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T14:17:36.812 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:17:36.812 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:17:36.812 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:17:36.812 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T14:17:37.813 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary6 up+replaying '' '' 2024-09-12T14:17:37.813 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:17:37.813 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:17:37.814 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T14:17:37.814 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T14:17:37.814 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:17:37.814 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:17:37.814 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:17:37.814 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:17:37.814 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary6 2024-09-12T14:17:37.889 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary6: 2024-09-12T14:17:37.890 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: b2ea2747-3b65-4670-9e03-38ab325e0695 2024-09-12T14:17:37.890 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+replaying 2024-09-12T14:17:37.890 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: replaying, {"bytes_per_second":3409100.8,"bytes_per_snapshot":68182016.0,"last_snapshot_bytes":68182016,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726150583,"remote_snapshot_timestamp":1726150583,"replay_state":"idle"} 2024-09-12T14:17:37.890 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T14:17:37.890 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 14:17:17' 2024-09-12T14:17:37.890 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+replaying' 2024-09-12T14:17:37.892 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+replaying 2024-09-12T14:17:37.892 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T14:17:37.895 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: replaying, {"bytes_per_second":3409100.8,"bytes_per_snapshot":68182016.0,"last_snapshot_bytes":68182016,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726150583,"remote_snapshot_timestamp":1726150583,"replay_state":"idle"} 2024-09-12T14:17:37.895 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T14:17:37.896 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+replaying 2024-09-12T14:17:37.896 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T14:17:37.898 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+replaying 2024-09-12T14:17:37.898 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T14:17:37.901 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T14:17:37.901 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T14:17:37.902 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T14:17:37.908 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 14:17:17' 2024-09-12T14:17:37.908 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary6 up+replaying '2024-09-12 14:17:17' 2024-09-12T14:17:37.909 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:17:37.909 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:17:37.909 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T14:17:37.909 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T14:17:37.909 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 14:17:17' 2024-09-12T14:17:37.910 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:17:37.910 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:17:37.910 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-09-12T14:17:38.001 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING541934232mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary10d6c75f78-0d2c-4921-8b6d-d39474248debup+replayingreplaying, {"bytes_per_second":3136921.6,"bytes_per_snapshot":62738432.0,"last_snapshot_bytes":62738432,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726150585,"remote_snapshot_timestamp":1726150585,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 14:17:17image-primary6b2ea2747-3b65-4670-9e03-38ab325e0695up+replayingreplaying, {"bytes_per_second":3409100.8,"bytes_per_snapshot":68182016.0,"last_snapshot_bytes":68182016,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726150583,"remote_snapshot_timestamp":1726150583,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 14:17:17image-primary7639c39b5-6a6c-4696-8c6f-e70477f65d39up+replayingreplaying, {"bytes_per_second":3258982.4,"bytes_per_snapshot":65179648.0,"last_snapshot_bytes":65179648,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726150585,"remote_snapshot_timestamp":1726150585,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 14:17:17image-primary88c5b00c9-9b25-4271-b4e1-865fb28703a0up+replayingreplaying, {"bytes_per_second":3366707.2,"bytes_per_snapshot":67334144.0,"last_snapshot_bytes":67334144,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726150585,"remote_snapshot_timestamp":1726150585,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 14:17:17image-primary98c49b822-b2f9-4ec9-9270-a22fd92ddba3up+replayingreplaying, {"bytes_per_second":3383296.0,"bytes_per_snapshot":67665920.0,"last_snapshot_bytes":67665920,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726150585,"remote_snapshot_timestamp":1726150585,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 14:17:17' 2024-09-12T14:17:38.001 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T14:17:38.001 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary6'\'']/last_update' 2024-09-12T14:17:38.005 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 14:17:17' 2024-09-12T14:17:38.006 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary6'\'']/state' 2024-09-12T14:17:38.010 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+replaying 2024-09-12T14:17:38.010 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+replaying 2024-09-12T14:17:38.011 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+replaying 2024-09-12T14:17:38.013 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+replaying 2024-09-12T14:17:38.014 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:17:38.014 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:17:38.014 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ remove_image cluster2 mirror image-primary6 2024-09-12T14:17:38.014 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T14:17:38.014 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:17:38.014 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T14:17:38.014 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 snap purge mirror/image-primary6 2024-09-12T14:17:38.074 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 rm mirror/image-primary6 2024-09-12T14:17:38.222 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T14:17:38.222+0000 7f67e5200640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x56055b84d6c0 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-09-12T14:17:38.222 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T14:17:38.222+0000 7f67ed400640 -1 rbd::mirror::ImageReplayer: 0x5605561acf00 [3/b2ea2747-3b65-4670-9e03-38ab325e0695] operator(): start failed: (85) Interrupted system call should be restarted 2024-09-12T14:17:39.761 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-09-12T14:17:39.767 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_image_removal cluster1 mirror image-primary6 2024-09-12T14:17:39.767 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:17:39.767 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:17:39.767 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary6 2024-09-12T14:17:39.767 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:17:39.768 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 ls mirror 2024-09-12T14:17:39.768 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep -wq image-primary6 2024-09-12T14:17:39.826 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:17:39.826 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rm -rf test-primary6 2024-09-12T14:17:39.829 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T14:17:39.829 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ IMG=image-primary7 2024-09-12T14:17:39.830 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_replaying_status_in_pool_dir cluster1 mirror image-primary7 2024-09-12T14:17:39.830 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:17:39.830 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:17:39.830 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T14:17:39.830 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' snapshot = journal ']' 2024-09-12T14:17:39.830 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary7 up+replaying 2024-09-12T14:17:39.830 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:17:39.830 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:17:39.830 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T14:17:39.830 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T14:17:39.830 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:17:39.831 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:17:39.831 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:17:39.831 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T14:17:40.831 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary7 up+replaying '' '' 2024-09-12T14:17:40.831 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:17:40.831 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:17:40.831 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T14:17:40.832 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T14:17:40.832 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:17:40.832 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:17:40.832 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:17:40.832 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:17:40.832 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary7 2024-09-12T14:17:40.927 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary7: 2024-09-12T14:17:40.928 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 639c39b5-6a6c-4696-8c6f-e70477f65d39 2024-09-12T14:17:40.928 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+replaying 2024-09-12T14:17:40.928 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: replaying, {"bytes_per_second":3258982.4,"bytes_per_snapshot":65179648.0,"last_snapshot_bytes":65179648,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726150585,"remote_snapshot_timestamp":1726150585,"replay_state":"idle"} 2024-09-12T14:17:40.928 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T14:17:40.928 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 14:17:17' 2024-09-12T14:17:40.928 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+replaying' 2024-09-12T14:17:40.930 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+replaying 2024-09-12T14:17:40.930 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T14:17:40.933 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: replaying, {"bytes_per_second":3258982.4,"bytes_per_snapshot":65179648.0,"last_snapshot_bytes":65179648,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726150585,"remote_snapshot_timestamp":1726150585,"replay_state":"idle"} 2024-09-12T14:17:40.933 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T14:17:40.934 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+replaying 2024-09-12T14:17:40.934 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T14:17:40.936 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+replaying 2024-09-12T14:17:40.936 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T14:17:40.939 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T14:17:40.939 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T14:17:40.940 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T14:17:40.943 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 14:17:17' 2024-09-12T14:17:40.944 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary7 up+replaying '2024-09-12 14:17:17' 2024-09-12T14:17:40.944 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:17:40.944 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:17:40.944 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T14:17:40.944 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T14:17:40.944 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 14:17:17' 2024-09-12T14:17:40.944 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:17:40.944 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:17:40.945 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-09-12T14:17:41.086 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING441934232mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary10d6c75f78-0d2c-4921-8b6d-d39474248debup+replayingreplaying, {"bytes_per_second":3136921.6,"bytes_per_snapshot":62738432.0,"last_snapshot_bytes":62738432,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726150585,"remote_snapshot_timestamp":1726150585,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 14:17:17image-primary7639c39b5-6a6c-4696-8c6f-e70477f65d39up+replayingreplaying, {"bytes_per_second":3258982.4,"bytes_per_snapshot":65179648.0,"last_snapshot_bytes":65179648,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726150585,"remote_snapshot_timestamp":1726150585,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 14:17:17image-primary88c5b00c9-9b25-4271-b4e1-865fb28703a0up+replayingreplaying, {"bytes_per_second":3366707.2,"bytes_per_snapshot":67334144.0,"last_snapshot_bytes":67334144,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726150585,"remote_snapshot_timestamp":1726150585,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 14:17:17image-primary98c49b822-b2f9-4ec9-9270-a22fd92ddba3up+replayingreplaying, {"bytes_per_second":3383296.0,"bytes_per_snapshot":67665920.0,"last_snapshot_bytes":67665920,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726150585,"remote_snapshot_timestamp":1726150585,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 14:17:17' 2024-09-12T14:17:41.086 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T14:17:41.087 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary7'\'']/last_update' 2024-09-12T14:17:41.091 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 14:17:17' 2024-09-12T14:17:41.092 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary7'\'']/state' 2024-09-12T14:17:41.095 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+replaying 2024-09-12T14:17:41.096 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+replaying 2024-09-12T14:17:41.096 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+replaying 2024-09-12T14:17:41.098 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+replaying 2024-09-12T14:17:41.099 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:17:41.099 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:17:41.099 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ remove_image cluster2 mirror image-primary7 2024-09-12T14:17:41.099 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T14:17:41.099 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:17:41.100 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T14:17:41.100 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 snap purge mirror/image-primary7 2024-09-12T14:17:41.161 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 rm mirror/image-primary7 2024-09-12T14:17:41.405 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T14:17:41.405+0000 7f67e5200640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x56055bce84e0 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-09-12T14:17:41.405 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T14:17:41.405+0000 7f67eca00640 -1 rbd::mirror::ImageReplayer: 0x5605561ac000 [3/639c39b5-6a6c-4696-8c6f-e70477f65d39] operator(): start failed: (85) Interrupted system call should be restarted 2024-09-12T14:17:42.754 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-09-12T14:17:42.760 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_image_removal cluster1 mirror image-primary7 2024-09-12T14:17:42.760 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:17:42.760 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:17:42.760 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary7 2024-09-12T14:17:42.760 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:17:42.761 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 ls mirror 2024-09-12T14:17:42.761 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep -wq image-primary7 2024-09-12T14:17:42.810 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:17:42.810 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rm -rf test-primary7 2024-09-12T14:17:42.813 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T14:17:42.813 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ IMG=image-primary8 2024-09-12T14:17:42.814 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_replaying_status_in_pool_dir cluster1 mirror image-primary8 2024-09-12T14:17:42.814 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:17:42.814 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:17:42.814 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T14:17:42.814 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' snapshot = journal ']' 2024-09-12T14:17:42.814 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary8 up+replaying 2024-09-12T14:17:42.814 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:17:42.814 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:17:42.814 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T14:17:42.814 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T14:17:42.814 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:17:42.814 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:17:42.815 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:17:42.815 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T14:17:43.815 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary8 up+replaying '' '' 2024-09-12T14:17:43.816 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:17:43.816 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:17:43.816 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T14:17:43.816 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T14:17:43.816 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:17:43.816 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:17:43.816 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:17:43.816 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:17:43.817 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary8 2024-09-12T14:17:43.896 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary8: 2024-09-12T14:17:43.896 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 8c5b00c9-9b25-4271-b4e1-865fb28703a0 2024-09-12T14:17:43.896 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+replaying 2024-09-12T14:17:43.896 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: replaying, {"bytes_per_second":3366707.2,"bytes_per_snapshot":67334144.0,"last_snapshot_bytes":67334144,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726150585,"remote_snapshot_timestamp":1726150585,"replay_state":"idle"} 2024-09-12T14:17:43.896 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T14:17:43.896 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 14:17:17' 2024-09-12T14:17:43.897 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+replaying' 2024-09-12T14:17:43.899 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+replaying 2024-09-12T14:17:43.899 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T14:17:43.902 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: replaying, {"bytes_per_second":3366707.2,"bytes_per_snapshot":67334144.0,"last_snapshot_bytes":67334144,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726150585,"remote_snapshot_timestamp":1726150585,"replay_state":"idle"} 2024-09-12T14:17:43.902 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T14:17:43.903 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+replaying 2024-09-12T14:17:43.903 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T14:17:43.905 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+replaying 2024-09-12T14:17:43.905 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T14:17:43.908 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T14:17:43.908 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T14:17:43.909 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T14:17:43.913 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 14:17:17' 2024-09-12T14:17:43.913 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary8 up+replaying '2024-09-12 14:17:17' 2024-09-12T14:17:43.913 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:17:43.913 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:17:43.913 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T14:17:43.913 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T14:17:43.913 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 14:17:17' 2024-09-12T14:17:43.914 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:17:43.914 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:17:43.914 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-09-12T14:17:43.999 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING341934232mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary10d6c75f78-0d2c-4921-8b6d-d39474248debup+replayingreplaying, {"bytes_per_second":3136921.6,"bytes_per_snapshot":62738432.0,"last_snapshot_bytes":62738432,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726150585,"remote_snapshot_timestamp":1726150585,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 14:17:17image-primary88c5b00c9-9b25-4271-b4e1-865fb28703a0up+replayingreplaying, {"bytes_per_second":3366707.2,"bytes_per_snapshot":67334144.0,"last_snapshot_bytes":67334144,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726150585,"remote_snapshot_timestamp":1726150585,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 14:17:17image-primary98c49b822-b2f9-4ec9-9270-a22fd92ddba3up+replayingreplaying, {"bytes_per_second":3383296.0,"bytes_per_snapshot":67665920.0,"last_snapshot_bytes":67665920,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726150585,"remote_snapshot_timestamp":1726150585,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 14:17:17' 2024-09-12T14:17:44.000 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T14:17:44.000 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary8'\'']/last_update' 2024-09-12T14:17:44.005 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 14:17:17' 2024-09-12T14:17:44.005 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary8'\'']/state' 2024-09-12T14:17:44.009 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+replaying 2024-09-12T14:17:44.009 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+replaying 2024-09-12T14:17:44.010 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+replaying 2024-09-12T14:17:44.012 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+replaying 2024-09-12T14:17:44.012 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:17:44.012 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:17:44.012 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ remove_image cluster2 mirror image-primary8 2024-09-12T14:17:44.013 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T14:17:44.013 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:17:44.013 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T14:17:44.013 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 snap purge mirror/image-primary8 2024-09-12T14:17:44.074 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 rm mirror/image-primary8 2024-09-12T14:17:44.224 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T14:17:44.225+0000 7f67e4800640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x56055bbfda00 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-09-12T14:17:44.225 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T14:17:44.225+0000 7f67eca00640 -1 rbd::mirror::ImageReplayer: 0x56055c199180 [3/8c5b00c9-9b25-4271-b4e1-865fb28703a0] operator(): start failed: (85) Interrupted system call should be restarted 2024-09-12T14:17:45.732 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-09-12T14:17:45.738 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_image_removal cluster1 mirror image-primary8 2024-09-12T14:17:45.738 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:17:45.738 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:17:45.738 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary8 2024-09-12T14:17:45.738 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:17:45.739 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 ls mirror 2024-09-12T14:17:45.739 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep -wq image-primary8 2024-09-12T14:17:45.788 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:17:45.788 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rm -rf test-primary8 2024-09-12T14:17:45.791 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T14:17:45.791 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ IMG=image-primary9 2024-09-12T14:17:45.791 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_replaying_status_in_pool_dir cluster1 mirror image-primary9 2024-09-12T14:17:45.791 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:17:45.791 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:17:45.791 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T14:17:45.791 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' snapshot = journal ']' 2024-09-12T14:17:45.791 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary9 up+replaying 2024-09-12T14:17:45.791 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:17:45.791 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:17:45.791 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T14:17:45.791 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T14:17:45.792 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:17:45.792 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:17:45.792 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:17:45.792 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T14:17:46.793 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary9 up+replaying '' '' 2024-09-12T14:17:46.793 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:17:46.793 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:17:46.793 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T14:17:46.793 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T14:17:46.793 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:17:46.794 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:17:46.794 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:17:46.794 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:17:46.794 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary9 2024-09-12T14:17:46.876 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary9: 2024-09-12T14:17:46.876 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: 8c49b822-b2f9-4ec9-9270-a22fd92ddba3 2024-09-12T14:17:46.876 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+replaying 2024-09-12T14:17:46.877 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: replaying, {"bytes_per_second":3383296.0,"bytes_per_snapshot":67665920.0,"last_snapshot_bytes":67665920,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726150585,"remote_snapshot_timestamp":1726150585,"replay_state":"idle"} 2024-09-12T14:17:46.877 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T14:17:46.877 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 14:17:17' 2024-09-12T14:17:46.877 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+replaying' 2024-09-12T14:17:46.879 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+replaying 2024-09-12T14:17:46.879 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T14:17:46.882 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: replaying, {"bytes_per_second":3383296.0,"bytes_per_snapshot":67665920.0,"last_snapshot_bytes":67665920,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726150585,"remote_snapshot_timestamp":1726150585,"replay_state":"idle"} 2024-09-12T14:17:46.882 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T14:17:46.883 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+replaying 2024-09-12T14:17:46.883 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T14:17:46.885 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+replaying 2024-09-12T14:17:46.885 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T14:17:46.888 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T14:17:46.888 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T14:17:46.889 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T14:17:46.893 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 14:17:17' 2024-09-12T14:17:46.893 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary9 up+replaying '2024-09-12 14:17:17' 2024-09-12T14:17:46.893 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:17:46.893 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:17:46.893 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T14:17:46.893 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T14:17:46.893 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 14:17:17' 2024-09-12T14:17:46.894 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:17:46.894 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:17:46.894 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-09-12T14:17:46.979 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING241934232mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary10d6c75f78-0d2c-4921-8b6d-d39474248debup+replayingreplaying, {"bytes_per_second":3136921.6,"bytes_per_snapshot":62738432.0,"last_snapshot_bytes":62738432,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726150585,"remote_snapshot_timestamp":1726150585,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 14:17:17image-primary98c49b822-b2f9-4ec9-9270-a22fd92ddba3up+replayingreplaying, {"bytes_per_second":3383296.0,"bytes_per_snapshot":67665920.0,"last_snapshot_bytes":67665920,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726150585,"remote_snapshot_timestamp":1726150585,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 14:17:17' 2024-09-12T14:17:46.979 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T14:17:46.980 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary9'\'']/last_update' 2024-09-12T14:17:46.984 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 14:17:17' 2024-09-12T14:17:46.985 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary9'\'']/state' 2024-09-12T14:17:46.988 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+replaying 2024-09-12T14:17:46.989 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+replaying 2024-09-12T14:17:46.989 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+replaying 2024-09-12T14:17:46.991 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+replaying 2024-09-12T14:17:46.992 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:17:46.992 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:17:46.992 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ remove_image cluster2 mirror image-primary9 2024-09-12T14:17:46.992 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T14:17:46.992 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:17:46.992 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T14:17:46.992 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 snap purge mirror/image-primary9 2024-09-12T14:17:47.054 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 rm mirror/image-primary9 2024-09-12T14:17:47.203 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T14:17:47.203+0000 7f67e5200640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x56055b7ee340 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-09-12T14:17:47.203 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T14:17:47.203+0000 7f67eca00640 -1 rbd::mirror::ImageReplayer: 0x5605561ad680 [3/8c49b822-b2f9-4ec9-9270-a22fd92ddba3] operator(): start failed: (85) Interrupted system call should be restarted 2024-09-12T14:17:47.910 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T14:17:47.910+0000 7f67e5200640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x560556dfc680 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-09-12T14:17:47.910 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T14:17:47.910+0000 7f67ed400640 -1 rbd::mirror::ImageReplayer: 0x5605561ad680 [3/8c49b822-b2f9-4ec9-9270-a22fd92ddba3] operator(): start failed: (85) Interrupted system call should be restarted 2024-09-12T14:17:49.127 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-09-12T14:17:49.133 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_image_removal cluster1 mirror image-primary9 2024-09-12T14:17:49.133 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:17:49.134 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:17:49.134 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary9 2024-09-12T14:17:49.134 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:17:49.134 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 ls mirror 2024-09-12T14:17:49.135 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep -wq image-primary9 2024-09-12T14:17:49.189 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:17:49.189 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rm -rf test-primary9 2024-09-12T14:17:49.192 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for j in {1..10} 2024-09-12T14:17:49.192 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ IMG=image-primary10 2024-09-12T14:17:49.192 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_replaying_status_in_pool_dir cluster1 mirror image-primary10 2024-09-12T14:17:49.192 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:17:49.192 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:17:49.192 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T14:17:49.192 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' snapshot = journal ']' 2024-09-12T14:17:49.192 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_status_in_pool_dir cluster1 mirror image-primary10 up+replaying 2024-09-12T14:17:49.192 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:17:49.193 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:17:49.193 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T14:17:49.193 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T14:17:49.193 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:17:49.193 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:17:49.193 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:17:49.193 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ sleep 1 2024-09-12T14:17:50.196 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_status_in_pool_dir cluster1 mirror image-primary10 up+replaying '' '' 2024-09-12T14:17:50.197 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:17:50.197 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:17:50.197 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T14:17:50.197 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T14:17:50.197 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local description_pattern= 2024-09-12T14:17:50.197 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local service_pattern= 2024-09-12T14:17:50.197 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:17:50.197 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:17:50.198 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror image status mirror/image-primary10 2024-09-12T14:17:50.297 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='image-primary10: 2024-09-12T14:17:50.298 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: global_id: d6c75f78-0d2c-4921-8b6d-d39474248deb 2024-09-12T14:17:50.298 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: state: up+replaying 2024-09-12T14:17:50.298 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: description: replaying, {"bytes_per_second":0.0,"bytes_per_snapshot":62738432.0,"last_snapshot_bytes":62738432,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726150585,"remote_snapshot_timestamp":1726150585,"replay_state":"idle"} 2024-09-12T14:17:50.298 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: service: mirror.0 on smithi130 2024-09-12T14:17:50.298 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: last_update: 2024-09-12 14:17:47' 2024-09-12T14:17:50.298 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ state: .*up+replaying' 2024-09-12T14:17:50.300 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: state: up+replaying 2024-09-12T14:17:50.300 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^ description: .*' 2024-09-12T14:17:50.303 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: description: replaying, {"bytes_per_second":0.0,"bytes_per_snapshot":62738432.0,"last_snapshot_bytes":62738432,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726150585,"remote_snapshot_timestamp":1726150585,"replay_state":"idle"} 2024-09-12T14:17:50.303 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -n '' ']' 2024-09-12T14:17:50.304 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+replaying 2024-09-12T14:17:50.304 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep '^up+' 2024-09-12T14:17:50.306 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+replaying 2024-09-12T14:17:50.306 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep 'service: *mirror..* on ' 2024-09-12T14:17:50.309 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout: service: mirror.0 on smithi130 2024-09-12T14:17:50.309 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update 2024-09-12T14:17:50.310 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-09-12T14:17:50.314 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 14:17:47' 2024-09-12T14:17:50.314 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test_mirror_pool_status_verbose cluster1 mirror image-primary10 up+replaying '2024-09-12 14:17:47' 2024-09-12T14:17:50.314 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:17:50.314 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:17:50.314 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T14:17:50.314 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local state_pattern=up+replaying 2024-09-12T14:17:50.314 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local 'prev_last_update=2024-09-12 14:17:47' 2024-09-12T14:17:50.314 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local status 2024-09-12T14:17:50.315 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ CEPH_ARGS= 2024-09-12T14:17:50.315 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-09-12T14:17:50.428 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ status='WARNINGOKWARNING141934232mirror.0smithi13019.1.1-314-g086c737ftrueOKimage-primary10d6c75f78-0d2c-4921-8b6d-d39474248debup+replayingreplaying, {"bytes_per_second":0.0,"bytes_per_snapshot":62738432.0,"last_snapshot_bytes":62738432,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1726150585,"remote_snapshot_timestamp":1726150585,"replay_state":"idle"}41934232mirror.0smithi1302024-09-12 14:17:47' 2024-09-12T14:17:50.428 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local last_update state 2024-09-12T14:17:50.428 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary10'\'']/last_update' 2024-09-12T14:17:50.432 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ last_update='2024-09-12 14:17:47' 2024-09-12T14:17:50.433 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''image-primary10'\'']/state' 2024-09-12T14:17:50.436 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ state=up+replaying 2024-09-12T14:17:50.437 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo up+replaying 2024-09-12T14:17:50.437 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep up+replaying 2024-09-12T14:17:50.439 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:up+replaying 2024-09-12T14:17:50.440 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:17:50.440 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:17:50.440 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ remove_image cluster2 mirror image-primary10 2024-09-12T14:17:50.440 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T14:17:50.441 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:17:50.441 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T14:17:50.441 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 snap purge mirror/image-primary10 2024-09-12T14:17:50.503 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster=cluster2 rm mirror/image-primary10 2024-09-12T14:17:50.744 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T14:17:50.743+0000 7f67e5200640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x56055b864ea0 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2024-09-12T14:17:50.744 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T14:17:50.743+0000 7f67eca00640 -1 rbd::mirror::ImageReplayer: 0x5605573cbb80 [3/d6c75f78-0d2c-4921-8b6d-d39474248deb] operator(): start failed: (85) Interrupted system call should be restarted 2024-09-12T14:17:52.820 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-09-12T14:17:52.826 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ wait_for_image_removal cluster1 mirror image-primary10 2024-09-12T14:17:52.826 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:17:52.827 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local pool=mirror 2024-09-12T14:17:52.827 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local image=image-primary10 2024-09-12T14:17:52.827 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-09-12T14:17:52.827 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rbd --cluster cluster1 ls mirror 2024-09-12T14:17:52.827 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ grep -wq image-primary10 2024-09-12T14:17:52.877 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 0 2024-09-12T14:17:52.877 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rm -rf test-primary10 2024-09-12T14:17:52.879 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:OK 2024-09-12T14:17:52.879 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo OK 2024-09-12T14:17:52.880 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ cleanup 0 2024-09-12T14:17:52.880 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local error_code=0 2024-09-12T14:17:52.880 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ set +e 2024-09-12T14:17:52.880 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' 0 -ne 0 ']' 2024-09-12T14:17:52.880 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -z '' ']' 2024-09-12T14:17:52.880 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster instance 2024-09-12T14:17:52.880 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ CEPH_ARGS= 2024-09-12T14:17:52.880 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ ceph --cluster cluster1 osd pool rm mirror mirror --yes-i-really-really-mean-it 2024-09-12T14:17:53.600 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi130.stderr:2024-09-12T14:17:53.600+0000 7f62e6400640 -1 rbd::mirror::image_deleter::SnapshotPurgeRequest: 0x55993f7f1560 handle_snap_remove: failed to remove snapshot: (2) No such file or directory 2024-09-12T14:17:53.601 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi130.stderr:2024-09-12T14:17:53.601+0000 7f62c4800640 -1 rbd::mirror::image_deleter::TrashRemoveRequest: 0x55993b6f1ef0 handle_purge_snapshots: failed to purge image snapshots: (2) No such file or directory 2024-09-12T14:17:53.695 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:pool 'mirror' does not exist 2024-09-12T14:17:53.710 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ CEPH_ARGS= 2024-09-12T14:17:53.710 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ ceph --cluster cluster2 osd pool rm mirror mirror --yes-i-really-really-mean-it 2024-09-12T14:17:54.700 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:pool 'mirror' does not exist 2024-09-12T14:17:54.716 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ CEPH_ARGS= 2024-09-12T14:17:54.716 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ ceph --cluster cluster1 osd pool rm mirror_parent mirror_parent --yes-i-really-really-mean-it 2024-09-12T14:17:54.733 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T14:17:54.733+0000 7f67eca00640 -1 rbd::mirror::LeaderWatcher: 0x5605559bcf00 handle_notify_heartbeat: error notifying heartbeat: (2) No such file or directory, releasing leader 2024-09-12T14:17:54.734 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T14:17:54.734+0000 7f67e4800640 -1 rbd::mirror::image_deleter::TrashWatcher: 0x560555be8b40 handle_unregister_watcher: error unregistering watcher for trash directory: (2) No such file or directory 2024-09-12T14:17:54.734 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T14:17:54.734+0000 7f67e4800640 -1 rbd::mirror::image_deleter::TrashWatcher: 0x560556030000 handle_unregister_watcher: error unregistering watcher for trash directory: (2) No such file or directory 2024-09-12T14:17:54.734 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T14:17:54.734+0000 7f67e4800640 -1 rbd::mirror::image_deleter::TrashWatcher: 0x560555be8480 handle_unregister_watcher: error unregistering watcher for trash directory: (2) No such file or directory 2024-09-12T14:17:54.734 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T14:17:54.734+0000 7f67e5200640 -1 rbd::mirror::PoolWatcher: 0x560554a170e0 operator(): error unregistering watcher for rbd_mirroring object: (2) No such file or directory 2024-09-12T14:17:54.734 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T14:17:54.734+0000 7f67e5200640 -1 rbd::mirror::PoolWatcher: 0x560554a17680 operator(): error unregistering watcher for rbd_mirroring object: (2) No such file or directory 2024-09-12T14:17:54.734 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T14:17:54.734+0000 7f67e5200640 -1 rbd::mirror::PoolWatcher: 0x560554a16b40 operator(): error unregistering watcher for rbd_mirroring object: (2) No such file or directory 2024-09-12T14:17:54.735 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T14:17:54.735+0000 7f67ed400640 -1 rbd::mirror::LeaderWatcher: 0x5605559bcf00 handle_notify_lock_released: error notifying leader lock released: (2) No such file or directory 2024-09-12T14:17:54.735 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi130.stderr:2024-09-12T14:17:54.735+0000 7f67ed400640 -1 rbd::mirror::LeaderWatcher: 0x5605559bcf00 handle_release_leader_lock: error releasing lock: (2) No such file or directory 2024-09-12T14:17:55.045 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi130.stderr:2024-09-12T14:17:55.045+0000 7f62edc00640 -1 rbd::mirror::LeaderWatcher: 0x55993b1baf00 handle_notify_heartbeat: error notifying heartbeat: (2) No such file or directory, releasing leader 2024-09-12T14:17:55.046 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi130.stderr:2024-09-12T14:17:55.046+0000 7f62e5a00640 -1 rbd::mirror::image_deleter::TrashWatcher: 0x55993b3e8480 handle_unregister_watcher: error unregistering watcher for trash directory: (2) No such file or directory 2024-09-12T14:17:55.046 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi130.stderr:2024-09-12T14:17:55.046+0000 7f62e5a00640 -1 rbd::mirror::image_deleter::TrashWatcher: 0x55993b3e8b40 handle_unregister_watcher: error unregistering watcher for trash directory: (2) No such file or directory 2024-09-12T14:17:55.046 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi130.stderr:2024-09-12T14:17:55.046+0000 7f62e5a00640 -1 rbd::mirror::image_deleter::TrashWatcher: 0x55993b670480 handle_unregister_watcher: error unregistering watcher for trash directory: (2) No such file or directory 2024-09-12T14:17:55.046 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi130.stderr:2024-09-12T14:17:55.046+0000 7f62de200640 -1 rbd::mirror::PoolWatcher: 0x55993b5d63c0 operator(): error unregistering watcher for rbd_mirroring object: (2) No such file or directory 2024-09-12T14:17:55.046 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi130.stderr:2024-09-12T14:17:55.046+0000 7f62de200640 -1 rbd::mirror::PoolWatcher: 0x55993b5d61e0 operator(): error unregistering watcher for rbd_mirroring object: (2) No such file or directory 2024-09-12T14:17:55.047 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi130.stderr:2024-09-12T14:17:55.046+0000 7f62de200640 -1 rbd::mirror::PoolWatcher: 0x55993b5d65a0 operator(): error unregistering watcher for rbd_mirroring object: (2) No such file or directory 2024-09-12T14:17:55.047 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi130.stderr:2024-09-12T14:17:55.046+0000 7f62e6400640 -1 rbd::mirror::PoolWatcher: 0x55993a235c20 operator(): error unregistering watcher for rbd_mirroring object: (2) No such file or directory 2024-09-12T14:17:55.047 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi130.stderr:2024-09-12T14:17:55.046+0000 7f62e6400640 -1 rbd::mirror::PoolWatcher: 0x55993a235a40 operator(): error unregistering watcher for rbd_mirroring object: (2) No such file or directory 2024-09-12T14:17:55.047 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi130.stderr:2024-09-12T14:17:55.046+0000 7f62e6400640 -1 rbd::mirror::PoolWatcher: 0x55993b5d6000 operator(): error unregistering watcher for rbd_mirroring object: (2) No such file or directory 2024-09-12T14:17:55.047 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi130.stderr:2024-09-12T14:17:55.048+0000 7f62edc00640 -1 rbd::mirror::LeaderWatcher: 0x55993b1baf00 handle_notify_lock_released: error notifying leader lock released: (2) No such file or directory 2024-09-12T14:17:55.047 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi130.stderr:2024-09-12T14:17:55.048+0000 7f62edc00640 -1 rbd::mirror::LeaderWatcher: 0x55993b1baf00 handle_release_leader_lock: error releasing lock: (2) No such file or directory 2024-09-12T14:17:55.756 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:pool 'mirror_parent' does not exist 2024-09-12T14:17:55.770 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ CEPH_ARGS= 2024-09-12T14:17:55.771 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ ceph --cluster cluster2 osd pool rm mirror_parent mirror_parent --yes-i-really-really-mean-it 2024-09-12T14:17:56.678 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:pool 'mirror_parent' does not exist 2024-09-12T14:17:56.693 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for cluster in "${CLUSTER1}" "${CLUSTER2}" 2024-09-12T14:17:56.693 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ stop_mirrors cluster1 2024-09-12T14:17:56.693 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1 2024-09-12T14:17:56.693 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local sig= 2024-09-12T14:17:56.693 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ seq 0 0 2024-09-12T14:17:56.705 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-09-12T14:17:56.705 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ stop_mirror cluster1:0 '' 2024-09-12T14:17:56.705 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster1:0 2024-09-12T14:17:56.705 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local sig= 2024-09-12T14:17:56.705 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test -n 1 2024-09-12T14:17:56.705 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 2024-09-12T14:17:56.705 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for cluster in "${CLUSTER1}" "${CLUSTER2}" 2024-09-12T14:17:56.705 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ stop_mirrors cluster2 2024-09-12T14:17:56.706 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2 2024-09-12T14:17:56.706 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local sig= 2024-09-12T14:17:56.706 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:++ seq 0 0 2024-09-12T14:17:56.709 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ for instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-09-12T14:17:56.709 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ stop_mirror cluster2:0 '' 2024-09-12T14:17:56.709 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local cluster=cluster2:0 2024-09-12T14:17:56.709 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ local sig= 2024-09-12T14:17:56.709 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test -n 1 2024-09-12T14:17:56.709 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ return 2024-09-12T14:17:56.709 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' -z 1 ']' 2024-09-12T14:17:56.709 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ test '' = /tmp/tmp.PVdhy5vDz3 2024-09-12T14:17:56.709 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ rm -Rf /tmp/tmp.PVdhy5vDz3 2024-09-12T14:17:56.719 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stdout:OK 2024-09-12T14:17:56.719 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ '[' 0 -eq 0 ']' 2024-09-12T14:17:56.719 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ echo OK 2024-09-12T14:17:56.719 INFO:tasks.workunit.cluster1.client.mirror.smithi130.stderr:+ exit 0 2024-09-12T14:17:56.721 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-09-12T14:17:56.721 DEBUG:teuthology.orchestra.run.smithi130:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.cluster1.mirror/client.mirror/tmp 2024-09-12T14:17:57.033 INFO:tasks.workunit:Stopping ['rbd/compare_mirror_images.sh'] on cluster1.client.mirror... 2024-09-12T14:17:57.033 DEBUG:teuthology.orchestra.run.smithi130:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.cluster1.client.mirror /home/ubuntu/cephtest/clone.cluster1.client.mirror 2024-09-12T14:17:57.957 DEBUG:teuthology.parallel:result is None 2024-09-12T14:17:57.958 DEBUG:teuthology.orchestra.run.smithi130:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.cluster1.mirror/client.mirror 2024-09-12T14:17:58.017 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.cluster1.mirror/client.mirror 2024-09-12T14:17:58.017 DEBUG:teuthology.orchestra.run.smithi130:> rmdir -- /home/ubuntu/cephtest/mnt.cluster1.mirror 2024-09-12T14:17:58.063 INFO:tasks.workunit:Deleted artificial mount point /home/ubuntu/cephtest/mnt.cluster1.mirror/client.mirror 2024-09-12T14:17:58.063 DEBUG:teuthology.run_tasks:Unwinding manager rbd-mirror 2024-09-12T14:17:58.075 DEBUG:tasks.rbd_mirror.cluster2.client.mirror.0:waiting for process to exit 2024-09-12T14:17:58.075 INFO:teuthology.orchestra.run:waiting for 300 2024-09-12T14:17:58.127 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Stopped 2024-09-12T14:17:58.127 DEBUG:teuthology.run_tasks:Unwinding manager rbd-mirror 2024-09-12T14:17:58.144 DEBUG:tasks.rbd_mirror.cluster1.client.mirror.0:waiting for process to exit 2024-09-12T14:17:58.144 INFO:teuthology.orchestra.run:waiting for 300 2024-09-12T14:17:58.190 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Stopped 2024-09-12T14:17:58.190 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-09-12T14:17:58.201 INFO:tasks.ceph.ceph_manager.cluster2:waiting for clean 2024-09-12T14:17:58.201 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster2 pg dump --format=json 2024-09-12T14:17:58.597 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-09-12T14:17:58.597 INFO:teuthology.orchestra.run.smithi130.stderr:dumped all 2024-09-12T14:17:58.614 INFO:teuthology.orchestra.run.smithi130.stdout:{"pg_ready":true,"pg_map":{"version":4384,"stamp":"2024-09-12T14:17:56.881689+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":139,"num_write_kb":1896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":100,"ondisk_log_size":100,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":233,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2761988,"kb_used_data":398540,"kb_used_omap":4,"kb_used_meta":2363387,"kb_avail":280353532,"statfs":{"total":289910292480,"available":287082016768,"internally_reserved":0,"allocated":408104960,"data_stored":428554689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4785,"internal_metadata":2420108623},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-598597884,"num_objects":-245,"num_object_clones":-27,"num_object_copies":-490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-245,"num_whiteouts":0,"num_read":-515260,"num_read_kb":-36056813,"num_write":-132064,"num_write_kb":-29367345,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-8,"num_bytes_recovered":-45746,"num_keys_recovered":-4,"num_objects_omap":-15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.329030"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":3885,"reported_epoch":1930,"state":"active+clean","last_fresh":"2024-09-12T14:17:54.577402+0000","last_change":"2024-09-12T12:56:13.297872+0000","last_active":"2024-09-12T14:17:54.577402+0000","last_peered":"2024-09-12T14:17:54.577402+0000","last_clean":"2024-09-12T14:17:54.577402+0000","last_became_active":"2024-09-12T12:56:11.291035+0000","last_became_peered":"2024-09-12T12:56:11.291035+0000","last_unstale":"2024-09-12T14:17:54.577402+0000","last_undegraded":"2024-09-12T14:17:54.577402+0000","last_fullsized":"2024-09-12T14:17:54.577402+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:56:10.277688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:56:10.277688+0000","last_clean_scrub_stamp":"2024-09-12T12:56:10.277688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:44:09.505949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001875938,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":3885,"reported_epoch":1930,"state":"active+clean","last_fresh":"2024-09-12T14:17:54.599389+0000","last_change":"2024-09-12T12:56:13.297542+0000","last_active":"2024-09-12T14:17:54.599389+0000","last_peered":"2024-09-12T14:17:54.599389+0000","last_clean":"2024-09-12T14:17:54.599389+0000","last_became_active":"2024-09-12T12:56:11.294341+0000","last_became_peered":"2024-09-12T12:56:11.294341+0000","last_unstale":"2024-09-12T14:17:54.599389+0000","last_undegraded":"2024-09-12T14:17:54.599389+0000","last_fullsized":"2024-09-12T14:17:54.599389+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:56:10.277688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:56:10.277688+0000","last_clean_scrub_stamp":"2024-09-12T12:56:10.277688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:41:52.966407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016741390000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":3887,"reported_epoch":1930,"state":"active+clean","last_fresh":"2024-09-12T14:17:54.599398+0000","last_change":"2024-09-12T12:56:13.297678+0000","last_active":"2024-09-12T14:17:54.599398+0000","last_peered":"2024-09-12T14:17:54.599398+0000","last_clean":"2024-09-12T14:17:54.599398+0000","last_became_active":"2024-09-12T12:56:11.294629+0000","last_became_peered":"2024-09-12T12:56:11.294629+0000","last_unstale":"2024-09-12T14:17:54.599398+0000","last_undegraded":"2024-09-12T14:17:54.599398+0000","last_fullsized":"2024-09-12T14:17:54.599398+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:56:10.277688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:56:10.277688+0000","last_clean_scrub_stamp":"2024-09-12T12:56:10.277688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:27:43.345132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018622739999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":3887,"reported_epoch":1930,"state":"active+clean","last_fresh":"2024-09-12T14:17:54.577016+0000","last_change":"2024-09-12T12:56:13.297034+0000","last_active":"2024-09-12T14:17:54.577016+0000","last_peered":"2024-09-12T14:17:54.577016+0000","last_clean":"2024-09-12T14:17:54.577016+0000","last_became_active":"2024-09-12T12:56:11.291147+0000","last_became_peered":"2024-09-12T12:56:11.291147+0000","last_unstale":"2024-09-12T14:17:54.577016+0000","last_undegraded":"2024-09-12T14:17:54.577016+0000","last_fullsized":"2024-09-12T14:17:54.577016+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:56:10.277688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:56:10.277688+0000","last_clean_scrub_stamp":"2024-09-12T12:56:10.277688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:23:38.950871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001237943,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"15'2","reported_seq":3886,"reported_epoch":1928,"state":"active+clean","last_fresh":"2024-09-12T14:17:51.555465+0000","last_change":"2024-09-12T12:56:13.292357+0000","last_active":"2024-09-12T14:17:51.555465+0000","last_peered":"2024-09-12T14:17:51.555465+0000","last_clean":"2024-09-12T14:17:51.555465+0000","last_became_active":"2024-09-12T12:56:11.295878+0000","last_became_peered":"2024-09-12T12:56:11.295878+0000","last_unstale":"2024-09-12T14:17:51.555465+0000","last_undegraded":"2024-09-12T14:17:51.555465+0000","last_fullsized":"2024-09-12T14:17:51.555465+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:56:10.277688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:56:10.277688+0000","last_clean_scrub_stamp":"2024-09-12T12:56:10.277688+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:53:37.770371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054639100000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":3882,"reported_epoch":1928,"state":"active+clean","last_fresh":"2024-09-12T14:17:51.573166+0000","last_change":"2024-09-12T12:56:13.355367+0000","last_active":"2024-09-12T14:17:51.573166+0000","last_peered":"2024-09-12T14:17:51.573166+0000","last_clean":"2024-09-12T14:17:51.573166+0000","last_became_active":"2024-09-12T12:56:11.295181+0000","last_became_peered":"2024-09-12T12:56:11.295181+0000","last_unstale":"2024-09-12T14:17:51.573166+0000","last_undegraded":"2024-09-12T14:17:51.573166+0000","last_fullsized":"2024-09-12T14:17:51.573166+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:56:10.277688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:56:10.277688+0000","last_clean_scrub_stamp":"2024-09-12T12:56:10.277688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:20:30.198897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057238600000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":3883,"reported_epoch":1928,"state":"active+clean","last_fresh":"2024-09-12T14:17:51.572976+0000","last_change":"2024-09-12T12:56:13.355375+0000","last_active":"2024-09-12T14:17:51.572976+0000","last_peered":"2024-09-12T14:17:51.572976+0000","last_clean":"2024-09-12T14:17:51.572976+0000","last_became_active":"2024-09-12T12:56:11.293181+0000","last_became_peered":"2024-09-12T12:56:11.293181+0000","last_unstale":"2024-09-12T14:17:51.572976+0000","last_undegraded":"2024-09-12T14:17:51.572976+0000","last_fullsized":"2024-09-12T14:17:51.572976+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:56:10.277688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:56:10.277688+0000","last_clean_scrub_stamp":"2024-09-12T12:56:10.277688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:41:47.856672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058228100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"13'1","reported_seq":3888,"reported_epoch":1930,"state":"active+clean","last_fresh":"2024-09-12T14:17:54.599336+0000","last_change":"2024-09-12T12:56:13.297158+0000","last_active":"2024-09-12T14:17:54.599336+0000","last_peered":"2024-09-12T14:17:54.599336+0000","last_clean":"2024-09-12T14:17:54.599336+0000","last_became_active":"2024-09-12T12:56:11.290474+0000","last_became_peered":"2024-09-12T12:56:11.290474+0000","last_unstale":"2024-09-12T14:17:54.599336+0000","last_undegraded":"2024-09-12T14:17:54.599336+0000","last_fullsized":"2024-09-12T14:17:54.599336+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:56:10.277688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:56:10.277688+0000","last_clean_scrub_stamp":"2024-09-12T12:56:10.277688+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:29:42.607257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001374526,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":4014,"reported_epoch":1930,"state":"active+clean","last_fresh":"2024-09-12T14:17:54.594048+0000","last_change":"2024-09-12T12:56:09.442875+0000","last_active":"2024-09-12T14:17:54.594048+0000","last_peered":"2024-09-12T14:17:54.594048+0000","last_clean":"2024-09-12T14:17:54.594048+0000","last_became_active":"2024-09-12T12:56:09.442486+0000","last_became_peered":"2024-09-12T12:56:09.442486+0000","last_unstale":"2024-09-12T14:17:54.594048+0000","last_undegraded":"2024-09-12T14:17:54.594048+0000","last_fullsized":"2024-09-12T14:17:54.594048+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:56:08.256849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:56:08.256849+0000","last_clean_scrub_stamp":"2024-09-12T12:56:08.256849+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T00:45:42.906851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359739354,"num_pgs":82,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":974664,"kb_used_data":203512,"kb_used_omap":1,"kb_used_meta":771134,"kb_avail":93397176,"statfs":{"total":96636764160,"available":95638708224,"internally_reserved":0,"allocated":208396288,"data_stored":220042809,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":789641669},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":8,"seq":34359739353,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":717796,"kb_used_data":4944,"kb_used_omap":1,"kb_used_meta":712830,"kb_avail":93654044,"statfs":{"total":96636764160,"available":95901741056,"internally_reserved":0,"allocated":5062656,"data_stored":3066236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":729938373},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359739352,"num_pgs":95,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1069528,"kb_used_data":190084,"kb_used_omap":1,"kb_used_meta":879422,"kb_avail":93302312,"statfs":{"total":96636764160,"available":95541567488,"internally_reserved":0,"allocated":194646016,"data_stored":205445644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":900528581},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T14:17:58.615 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster2 pg dump --format=json 2024-09-12T14:17:58.981 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-09-12T14:17:58.981 INFO:teuthology.orchestra.run.smithi130.stderr:dumped all 2024-09-12T14:17:58.999 INFO:teuthology.orchestra.run.smithi130.stdout:{"pg_ready":true,"pg_map":{"version":4385,"stamp":"2024-09-12T14:17:58.882045+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":139,"num_write_kb":1896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":100,"ondisk_log_size":100,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":186,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2377696,"kb_used_data":14244,"kb_used_omap":4,"kb_used_meta":2363387,"kb_avail":280737824,"statfs":{"total":289910292480,"available":287475531776,"internally_reserved":0,"allocated":14585856,"data_stored":8608302,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4785,"internal_metadata":2420108623},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-298885848,"num_objects":-116,"num_object_clones":0,"num_object_copies":-232,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-116,"num_whiteouts":0,"num_read":-515744,"num_read_kb":-36057234,"num_write":-132236,"num_write_kb":-29367379,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-8,"num_bytes_recovered":-45746,"num_keys_recovered":-4,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.325323"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":3885,"reported_epoch":1930,"state":"active+clean","last_fresh":"2024-09-12T14:17:54.577402+0000","last_change":"2024-09-12T12:56:13.297872+0000","last_active":"2024-09-12T14:17:54.577402+0000","last_peered":"2024-09-12T14:17:54.577402+0000","last_clean":"2024-09-12T14:17:54.577402+0000","last_became_active":"2024-09-12T12:56:11.291035+0000","last_became_peered":"2024-09-12T12:56:11.291035+0000","last_unstale":"2024-09-12T14:17:54.577402+0000","last_undegraded":"2024-09-12T14:17:54.577402+0000","last_fullsized":"2024-09-12T14:17:54.577402+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:56:10.277688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:56:10.277688+0000","last_clean_scrub_stamp":"2024-09-12T12:56:10.277688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:44:09.505949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001875938,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":3885,"reported_epoch":1930,"state":"active+clean","last_fresh":"2024-09-12T14:17:54.599389+0000","last_change":"2024-09-12T12:56:13.297542+0000","last_active":"2024-09-12T14:17:54.599389+0000","last_peered":"2024-09-12T14:17:54.599389+0000","last_clean":"2024-09-12T14:17:54.599389+0000","last_became_active":"2024-09-12T12:56:11.294341+0000","last_became_peered":"2024-09-12T12:56:11.294341+0000","last_unstale":"2024-09-12T14:17:54.599389+0000","last_undegraded":"2024-09-12T14:17:54.599389+0000","last_fullsized":"2024-09-12T14:17:54.599389+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:56:10.277688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:56:10.277688+0000","last_clean_scrub_stamp":"2024-09-12T12:56:10.277688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:41:52.966407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016741390000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":3887,"reported_epoch":1930,"state":"active+clean","last_fresh":"2024-09-12T14:17:54.599398+0000","last_change":"2024-09-12T12:56:13.297678+0000","last_active":"2024-09-12T14:17:54.599398+0000","last_peered":"2024-09-12T14:17:54.599398+0000","last_clean":"2024-09-12T14:17:54.599398+0000","last_became_active":"2024-09-12T12:56:11.294629+0000","last_became_peered":"2024-09-12T12:56:11.294629+0000","last_unstale":"2024-09-12T14:17:54.599398+0000","last_undegraded":"2024-09-12T14:17:54.599398+0000","last_fullsized":"2024-09-12T14:17:54.599398+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:56:10.277688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:56:10.277688+0000","last_clean_scrub_stamp":"2024-09-12T12:56:10.277688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:27:43.345132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018622739999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":3887,"reported_epoch":1930,"state":"active+clean","last_fresh":"2024-09-12T14:17:54.577016+0000","last_change":"2024-09-12T12:56:13.297034+0000","last_active":"2024-09-12T14:17:54.577016+0000","last_peered":"2024-09-12T14:17:54.577016+0000","last_clean":"2024-09-12T14:17:54.577016+0000","last_became_active":"2024-09-12T12:56:11.291147+0000","last_became_peered":"2024-09-12T12:56:11.291147+0000","last_unstale":"2024-09-12T14:17:54.577016+0000","last_undegraded":"2024-09-12T14:17:54.577016+0000","last_fullsized":"2024-09-12T14:17:54.577016+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:56:10.277688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:56:10.277688+0000","last_clean_scrub_stamp":"2024-09-12T12:56:10.277688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:23:38.950871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001237943,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"15'2","reported_seq":3892,"reported_epoch":1931,"state":"active+clean","last_fresh":"2024-09-12T14:17:56.650015+0000","last_change":"2024-09-12T12:56:13.292357+0000","last_active":"2024-09-12T14:17:56.650015+0000","last_peered":"2024-09-12T14:17:56.650015+0000","last_clean":"2024-09-12T14:17:56.650015+0000","last_became_active":"2024-09-12T12:56:11.295878+0000","last_became_peered":"2024-09-12T12:56:11.295878+0000","last_unstale":"2024-09-12T14:17:56.650015+0000","last_undegraded":"2024-09-12T14:17:56.650015+0000","last_fullsized":"2024-09-12T14:17:56.650015+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:56:10.277688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:56:10.277688+0000","last_clean_scrub_stamp":"2024-09-12T12:56:10.277688+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:53:37.770371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054639100000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":3886,"reported_epoch":1930,"state":"active+clean","last_fresh":"2024-09-12T14:17:54.738092+0000","last_change":"2024-09-12T12:56:13.355367+0000","last_active":"2024-09-12T14:17:54.738092+0000","last_peered":"2024-09-12T14:17:54.738092+0000","last_clean":"2024-09-12T14:17:54.738092+0000","last_became_active":"2024-09-12T12:56:11.295181+0000","last_became_peered":"2024-09-12T12:56:11.295181+0000","last_unstale":"2024-09-12T14:17:54.738092+0000","last_undegraded":"2024-09-12T14:17:54.738092+0000","last_fullsized":"2024-09-12T14:17:54.738092+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:56:10.277688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:56:10.277688+0000","last_clean_scrub_stamp":"2024-09-12T12:56:10.277688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:20:30.198897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057238600000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":3887,"reported_epoch":1930,"state":"active+clean","last_fresh":"2024-09-12T14:17:54.742000+0000","last_change":"2024-09-12T12:56:13.355375+0000","last_active":"2024-09-12T14:17:54.742000+0000","last_peered":"2024-09-12T14:17:54.742000+0000","last_clean":"2024-09-12T14:17:54.742000+0000","last_became_active":"2024-09-12T12:56:11.293181+0000","last_became_peered":"2024-09-12T12:56:11.293181+0000","last_unstale":"2024-09-12T14:17:54.742000+0000","last_undegraded":"2024-09-12T14:17:54.742000+0000","last_fullsized":"2024-09-12T14:17:54.742000+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:56:10.277688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:56:10.277688+0000","last_clean_scrub_stamp":"2024-09-12T12:56:10.277688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:41:47.856672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058228100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"13'1","reported_seq":3888,"reported_epoch":1930,"state":"active+clean","last_fresh":"2024-09-12T14:17:54.599336+0000","last_change":"2024-09-12T12:56:13.297158+0000","last_active":"2024-09-12T14:17:54.599336+0000","last_peered":"2024-09-12T14:17:54.599336+0000","last_clean":"2024-09-12T14:17:54.599336+0000","last_became_active":"2024-09-12T12:56:11.290474+0000","last_became_peered":"2024-09-12T12:56:11.290474+0000","last_unstale":"2024-09-12T14:17:54.599336+0000","last_undegraded":"2024-09-12T14:17:54.599336+0000","last_fullsized":"2024-09-12T14:17:54.599336+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:56:10.277688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:56:10.277688+0000","last_clean_scrub_stamp":"2024-09-12T12:56:10.277688+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:29:42.607257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001374526,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":4014,"reported_epoch":1930,"state":"active+clean","last_fresh":"2024-09-12T14:17:54.594048+0000","last_change":"2024-09-12T12:56:09.442875+0000","last_active":"2024-09-12T14:17:54.594048+0000","last_peered":"2024-09-12T14:17:54.594048+0000","last_clean":"2024-09-12T14:17:54.594048+0000","last_became_active":"2024-09-12T12:56:09.442486+0000","last_became_peered":"2024-09-12T12:56:09.442486+0000","last_unstale":"2024-09-12T14:17:54.594048+0000","last_undegraded":"2024-09-12T14:17:54.594048+0000","last_fullsized":"2024-09-12T14:17:54.594048+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:56:08.256849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:56:08.256849+0000","last_clean_scrub_stamp":"2024-09-12T12:56:08.256849+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T00:45:42.906851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359739355,"num_pgs":82,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":775516,"kb_used_data":4360,"kb_used_omap":1,"kb_used_meta":771134,"kb_avail":93596324,"statfs":{"total":96636764160,"available":95842635776,"internally_reserved":0,"allocated":4464640,"data_stored":2475849,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":789641669},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":8,"seq":34359739353,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":717796,"kb_used_data":4944,"kb_used_omap":1,"kb_used_meta":712830,"kb_avail":93654044,"statfs":{"total":96636764160,"available":95901741056,"internally_reserved":0,"allocated":5062656,"data_stored":3066236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":729938373},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359739353,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":884384,"kb_used_data":4940,"kb_used_omap":1,"kb_used_meta":879422,"kb_avail":93487456,"statfs":{"total":96636764160,"available":95731154944,"internally_reserved":0,"allocated":5058560,"data_stored":3066217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":900528581},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T14:17:58.999 INFO:tasks.ceph.ceph_manager.cluster2:clean! 2024-09-12T14:17:59.000 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster2 pg dump --format=json 2024-09-12T14:17:59.349 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-09-12T14:17:59.349 INFO:teuthology.orchestra.run.smithi130.stderr:dumped all 2024-09-12T14:17:59.366 INFO:teuthology.orchestra.run.smithi130.stdout:{"pg_ready":true,"pg_map":{"version":4385,"stamp":"2024-09-12T14:17:58.882045+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":139,"num_write_kb":1896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":100,"ondisk_log_size":100,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":186,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2377696,"kb_used_data":14244,"kb_used_omap":4,"kb_used_meta":2363387,"kb_avail":280737824,"statfs":{"total":289910292480,"available":287475531776,"internally_reserved":0,"allocated":14585856,"data_stored":8608302,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4785,"internal_metadata":2420108623},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-298885848,"num_objects":-116,"num_object_clones":0,"num_object_copies":-232,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-116,"num_whiteouts":0,"num_read":-515744,"num_read_kb":-36057234,"num_write":-132236,"num_write_kb":-29367379,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-8,"num_bytes_recovered":-45746,"num_keys_recovered":-4,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.325323"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":3885,"reported_epoch":1930,"state":"active+clean","last_fresh":"2024-09-12T14:17:54.577402+0000","last_change":"2024-09-12T12:56:13.297872+0000","last_active":"2024-09-12T14:17:54.577402+0000","last_peered":"2024-09-12T14:17:54.577402+0000","last_clean":"2024-09-12T14:17:54.577402+0000","last_became_active":"2024-09-12T12:56:11.291035+0000","last_became_peered":"2024-09-12T12:56:11.291035+0000","last_unstale":"2024-09-12T14:17:54.577402+0000","last_undegraded":"2024-09-12T14:17:54.577402+0000","last_fullsized":"2024-09-12T14:17:54.577402+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:56:10.277688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:56:10.277688+0000","last_clean_scrub_stamp":"2024-09-12T12:56:10.277688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:44:09.505949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001875938,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":3885,"reported_epoch":1930,"state":"active+clean","last_fresh":"2024-09-12T14:17:54.599389+0000","last_change":"2024-09-12T12:56:13.297542+0000","last_active":"2024-09-12T14:17:54.599389+0000","last_peered":"2024-09-12T14:17:54.599389+0000","last_clean":"2024-09-12T14:17:54.599389+0000","last_became_active":"2024-09-12T12:56:11.294341+0000","last_became_peered":"2024-09-12T12:56:11.294341+0000","last_unstale":"2024-09-12T14:17:54.599389+0000","last_undegraded":"2024-09-12T14:17:54.599389+0000","last_fullsized":"2024-09-12T14:17:54.599389+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:56:10.277688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:56:10.277688+0000","last_clean_scrub_stamp":"2024-09-12T12:56:10.277688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:41:52.966407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016741390000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":3887,"reported_epoch":1930,"state":"active+clean","last_fresh":"2024-09-12T14:17:54.599398+0000","last_change":"2024-09-12T12:56:13.297678+0000","last_active":"2024-09-12T14:17:54.599398+0000","last_peered":"2024-09-12T14:17:54.599398+0000","last_clean":"2024-09-12T14:17:54.599398+0000","last_became_active":"2024-09-12T12:56:11.294629+0000","last_became_peered":"2024-09-12T12:56:11.294629+0000","last_unstale":"2024-09-12T14:17:54.599398+0000","last_undegraded":"2024-09-12T14:17:54.599398+0000","last_fullsized":"2024-09-12T14:17:54.599398+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:56:10.277688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:56:10.277688+0000","last_clean_scrub_stamp":"2024-09-12T12:56:10.277688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:27:43.345132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018622739999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":3887,"reported_epoch":1930,"state":"active+clean","last_fresh":"2024-09-12T14:17:54.577016+0000","last_change":"2024-09-12T12:56:13.297034+0000","last_active":"2024-09-12T14:17:54.577016+0000","last_peered":"2024-09-12T14:17:54.577016+0000","last_clean":"2024-09-12T14:17:54.577016+0000","last_became_active":"2024-09-12T12:56:11.291147+0000","last_became_peered":"2024-09-12T12:56:11.291147+0000","last_unstale":"2024-09-12T14:17:54.577016+0000","last_undegraded":"2024-09-12T14:17:54.577016+0000","last_fullsized":"2024-09-12T14:17:54.577016+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:56:10.277688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:56:10.277688+0000","last_clean_scrub_stamp":"2024-09-12T12:56:10.277688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:23:38.950871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001237943,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"15'2","reported_seq":3892,"reported_epoch":1931,"state":"active+clean","last_fresh":"2024-09-12T14:17:56.650015+0000","last_change":"2024-09-12T12:56:13.292357+0000","last_active":"2024-09-12T14:17:56.650015+0000","last_peered":"2024-09-12T14:17:56.650015+0000","last_clean":"2024-09-12T14:17:56.650015+0000","last_became_active":"2024-09-12T12:56:11.295878+0000","last_became_peered":"2024-09-12T12:56:11.295878+0000","last_unstale":"2024-09-12T14:17:56.650015+0000","last_undegraded":"2024-09-12T14:17:56.650015+0000","last_fullsized":"2024-09-12T14:17:56.650015+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:56:10.277688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:56:10.277688+0000","last_clean_scrub_stamp":"2024-09-12T12:56:10.277688+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:53:37.770371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054639100000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":3886,"reported_epoch":1930,"state":"active+clean","last_fresh":"2024-09-12T14:17:54.738092+0000","last_change":"2024-09-12T12:56:13.355367+0000","last_active":"2024-09-12T14:17:54.738092+0000","last_peered":"2024-09-12T14:17:54.738092+0000","last_clean":"2024-09-12T14:17:54.738092+0000","last_became_active":"2024-09-12T12:56:11.295181+0000","last_became_peered":"2024-09-12T12:56:11.295181+0000","last_unstale":"2024-09-12T14:17:54.738092+0000","last_undegraded":"2024-09-12T14:17:54.738092+0000","last_fullsized":"2024-09-12T14:17:54.738092+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:56:10.277688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:56:10.277688+0000","last_clean_scrub_stamp":"2024-09-12T12:56:10.277688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:20:30.198897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057238600000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":3887,"reported_epoch":1930,"state":"active+clean","last_fresh":"2024-09-12T14:17:54.742000+0000","last_change":"2024-09-12T12:56:13.355375+0000","last_active":"2024-09-12T14:17:54.742000+0000","last_peered":"2024-09-12T14:17:54.742000+0000","last_clean":"2024-09-12T14:17:54.742000+0000","last_became_active":"2024-09-12T12:56:11.293181+0000","last_became_peered":"2024-09-12T12:56:11.293181+0000","last_unstale":"2024-09-12T14:17:54.742000+0000","last_undegraded":"2024-09-12T14:17:54.742000+0000","last_fullsized":"2024-09-12T14:17:54.742000+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:56:10.277688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:56:10.277688+0000","last_clean_scrub_stamp":"2024-09-12T12:56:10.277688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:41:47.856672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058228100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"13'1","reported_seq":3888,"reported_epoch":1930,"state":"active+clean","last_fresh":"2024-09-12T14:17:54.599336+0000","last_change":"2024-09-12T12:56:13.297158+0000","last_active":"2024-09-12T14:17:54.599336+0000","last_peered":"2024-09-12T14:17:54.599336+0000","last_clean":"2024-09-12T14:17:54.599336+0000","last_became_active":"2024-09-12T12:56:11.290474+0000","last_became_peered":"2024-09-12T12:56:11.290474+0000","last_unstale":"2024-09-12T14:17:54.599336+0000","last_undegraded":"2024-09-12T14:17:54.599336+0000","last_fullsized":"2024-09-12T14:17:54.599336+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:56:10.277688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:56:10.277688+0000","last_clean_scrub_stamp":"2024-09-12T12:56:10.277688+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:29:42.607257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001374526,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":4014,"reported_epoch":1930,"state":"active+clean","last_fresh":"2024-09-12T14:17:54.594048+0000","last_change":"2024-09-12T12:56:09.442875+0000","last_active":"2024-09-12T14:17:54.594048+0000","last_peered":"2024-09-12T14:17:54.594048+0000","last_clean":"2024-09-12T14:17:54.594048+0000","last_became_active":"2024-09-12T12:56:09.442486+0000","last_became_peered":"2024-09-12T12:56:09.442486+0000","last_unstale":"2024-09-12T14:17:54.594048+0000","last_undegraded":"2024-09-12T14:17:54.594048+0000","last_fullsized":"2024-09-12T14:17:54.594048+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:56:08.256849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:56:08.256849+0000","last_clean_scrub_stamp":"2024-09-12T12:56:08.256849+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T00:45:42.906851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359739355,"num_pgs":82,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":775516,"kb_used_data":4360,"kb_used_omap":1,"kb_used_meta":771134,"kb_avail":93596324,"statfs":{"total":96636764160,"available":95842635776,"internally_reserved":0,"allocated":4464640,"data_stored":2475849,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":789641669},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":8,"seq":34359739353,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":717796,"kb_used_data":4944,"kb_used_omap":1,"kb_used_meta":712830,"kb_avail":93654044,"statfs":{"total":96636764160,"available":95901741056,"internally_reserved":0,"allocated":5062656,"data_stored":3066236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":729938373},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359739353,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":884384,"kb_used_data":4940,"kb_used_omap":1,"kb_used_meta":879422,"kb_avail":93487456,"statfs":{"total":96636764160,"available":95731154944,"internally_reserved":0,"allocated":5058560,"data_stored":3066217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":900528581},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T14:17:59.367 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster2 osd dump --format=json 2024-09-12T14:17:59.714 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-09-12T14:17:59.715 INFO:teuthology.orchestra.run.smithi130.stdout:{"epoch":1931,"fsid":"76d48f4f-9f5b-4bde-b048-7f763396fe3f","created":"2024-09-12T12:55:54.555731+0000","modified":"2024-09-12T14:17:56.566102+0000","last_up_change":"2024-09-12T12:56:06.254589+0000","last_in_change":"2024-09-12T12:55:55.652531+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T12:56:07.548658+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-12T12:56:10.277057+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":15,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"14760318-3ea5-426d-949a-87b78b1c5170","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6800","nonce":373021749},{"type":"v1","addr":"172.21.15.130:6801","nonce":373021749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6802","nonce":373021749},{"type":"v1","addr":"172.21.15.130:6803","nonce":373021749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6806","nonce":373021749},{"type":"v1","addr":"172.21.15.130:6807","nonce":373021749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6804","nonce":373021749},{"type":"v1","addr":"172.21.15.130:6805","nonce":373021749}]},"public_addr":"172.21.15.130:6801/373021749","cluster_addr":"172.21.15.130:6803/373021749","heartbeat_back_addr":"172.21.15.130:6807/373021749","heartbeat_front_addr":"172.21.15.130:6805/373021749","state":["exists","up"]},{"osd":1,"uuid":"5129799d-ce0e-4fbc-9bcc-2528285fa983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6808","nonce":1890016022},{"type":"v1","addr":"172.21.15.130:6809","nonce":1890016022}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6810","nonce":1890016022},{"type":"v1","addr":"172.21.15.130:6811","nonce":1890016022}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6817","nonce":1890016022},{"type":"v1","addr":"172.21.15.130:6819","nonce":1890016022}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6813","nonce":1890016022},{"type":"v1","addr":"172.21.15.130:6815","nonce":1890016022}]},"public_addr":"172.21.15.130:6809/1890016022","cluster_addr":"172.21.15.130:6811/1890016022","heartbeat_back_addr":"172.21.15.130:6819/1890016022","heartbeat_front_addr":"172.21.15.130:6815/1890016022","state":["exists","up"]},{"osd":2,"uuid":"3321b514-7dac-4085-9fe1-55714a27c720","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6812","nonce":2246267968},{"type":"v1","addr":"172.21.15.130:6814","nonce":2246267968}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6816","nonce":2246267968},{"type":"v1","addr":"172.21.15.130:6818","nonce":2246267968}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6822","nonce":2246267968},{"type":"v1","addr":"172.21.15.130:6823","nonce":2246267968}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6820","nonce":2246267968},{"type":"v1","addr":"172.21.15.130:6821","nonce":2246267968}]},"public_addr":"172.21.15.130:6814/2246267968","cluster_addr":"172.21.15.130:6818/2246267968","heartbeat_back_addr":"172.21.15.130:6823/2246267968","heartbeat_front_addr":"172.21.15.130:6821/2246267968","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:56:03.468853+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:56:03.255425+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:56:05.253102+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":3,"snaps":[{"begin":3225,"length":1},{"begin":3233,"length":1},{"begin":3240,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T14:18:00.735 INFO:tasks.ceph:Scrubbing cluster2.osd.0 2024-09-12T14:18:00.735 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster2 tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-09-12T14:18:00.916 INFO:teuthology.orchestra.run.smithi130.stdout:{ 2024-09-12T14:18:00.916 INFO:teuthology.orchestra.run.smithi130.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-12T14:18:00.916 INFO:teuthology.orchestra.run.smithi130.stdout:} 2024-09-12T14:18:00.931 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster2 osd deep-scrub 0 2024-09-12T14:18:01.271 INFO:teuthology.orchestra.run.smithi130.stderr:instructed osd(s) 0 to deep-scrub 2024-09-12T14:18:01.288 INFO:tasks.ceph:Scrubbing cluster2.osd.1 2024-09-12T14:18:01.289 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster2 tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-09-12T14:18:01.468 INFO:teuthology.orchestra.run.smithi130.stdout:{ 2024-09-12T14:18:01.468 INFO:teuthology.orchestra.run.smithi130.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-12T14:18:01.468 INFO:teuthology.orchestra.run.smithi130.stdout:} 2024-09-12T14:18:01.483 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster2 osd deep-scrub 1 2024-09-12T14:18:01.822 INFO:teuthology.orchestra.run.smithi130.stderr:instructed osd(s) 1 to deep-scrub 2024-09-12T14:18:01.841 INFO:tasks.ceph:Scrubbing cluster2.osd.2 2024-09-12T14:18:01.841 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster2 tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-09-12T14:18:02.021 INFO:teuthology.orchestra.run.smithi130.stdout:{ 2024-09-12T14:18:02.021 INFO:teuthology.orchestra.run.smithi130.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-12T14:18:02.021 INFO:teuthology.orchestra.run.smithi130.stdout:} 2024-09-12T14:18:02.036 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster2 osd deep-scrub 2 2024-09-12T14:18:02.374 INFO:teuthology.orchestra.run.smithi130.stderr:instructed osd(s) 2 to deep-scrub 2024-09-12T14:18:02.392 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster2 pg dump --format=json 2024-09-12T14:18:02.727 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-09-12T14:18:02.727 INFO:teuthology.orchestra.run.smithi130.stderr:dumped all 2024-09-12T14:18:02.744 INFO:teuthology.orchestra.run.smithi130.stdout:{"pg_ready":true,"pg_map":{"version":4386,"stamp":"2024-09-12T14:18:00.882348+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":139,"num_write_kb":1896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":100,"ondisk_log_size":100,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":186,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2377696,"kb_used_data":14244,"kb_used_omap":4,"kb_used_meta":2363387,"kb_avail":280737824,"statfs":{"total":289910292480,"available":287475531776,"internally_reserved":0,"allocated":14585856,"data_stored":8608302,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4785,"internal_metadata":2420108623},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-298885848,"num_objects":-116,"num_object_clones":0,"num_object_copies":-232,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-116,"num_whiteouts":0,"num_read":-515744,"num_read_kb":-36057234,"num_write":-132236,"num_write_kb":-29367379,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-8,"num_bytes_recovered":-45746,"num_keys_recovered":-4,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001350"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":3885,"reported_epoch":1930,"state":"active+clean","last_fresh":"2024-09-12T14:17:54.577402+0000","last_change":"2024-09-12T12:56:13.297872+0000","last_active":"2024-09-12T14:17:54.577402+0000","last_peered":"2024-09-12T14:17:54.577402+0000","last_clean":"2024-09-12T14:17:54.577402+0000","last_became_active":"2024-09-12T12:56:11.291035+0000","last_became_peered":"2024-09-12T12:56:11.291035+0000","last_unstale":"2024-09-12T14:17:54.577402+0000","last_undegraded":"2024-09-12T14:17:54.577402+0000","last_fullsized":"2024-09-12T14:17:54.577402+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:56:10.277688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:56:10.277688+0000","last_clean_scrub_stamp":"2024-09-12T12:56:10.277688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:44:09.505949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001875938,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":3885,"reported_epoch":1930,"state":"active+clean","last_fresh":"2024-09-12T14:17:54.599389+0000","last_change":"2024-09-12T12:56:13.297542+0000","last_active":"2024-09-12T14:17:54.599389+0000","last_peered":"2024-09-12T14:17:54.599389+0000","last_clean":"2024-09-12T14:17:54.599389+0000","last_became_active":"2024-09-12T12:56:11.294341+0000","last_became_peered":"2024-09-12T12:56:11.294341+0000","last_unstale":"2024-09-12T14:17:54.599389+0000","last_undegraded":"2024-09-12T14:17:54.599389+0000","last_fullsized":"2024-09-12T14:17:54.599389+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:56:10.277688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:56:10.277688+0000","last_clean_scrub_stamp":"2024-09-12T12:56:10.277688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:41:52.966407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016741390000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":3887,"reported_epoch":1930,"state":"active+clean","last_fresh":"2024-09-12T14:17:54.599398+0000","last_change":"2024-09-12T12:56:13.297678+0000","last_active":"2024-09-12T14:17:54.599398+0000","last_peered":"2024-09-12T14:17:54.599398+0000","last_clean":"2024-09-12T14:17:54.599398+0000","last_became_active":"2024-09-12T12:56:11.294629+0000","last_became_peered":"2024-09-12T12:56:11.294629+0000","last_unstale":"2024-09-12T14:17:54.599398+0000","last_undegraded":"2024-09-12T14:17:54.599398+0000","last_fullsized":"2024-09-12T14:17:54.599398+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:56:10.277688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:56:10.277688+0000","last_clean_scrub_stamp":"2024-09-12T12:56:10.277688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:27:43.345132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018622739999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":3887,"reported_epoch":1930,"state":"active+clean","last_fresh":"2024-09-12T14:17:54.577016+0000","last_change":"2024-09-12T12:56:13.297034+0000","last_active":"2024-09-12T14:17:54.577016+0000","last_peered":"2024-09-12T14:17:54.577016+0000","last_clean":"2024-09-12T14:17:54.577016+0000","last_became_active":"2024-09-12T12:56:11.291147+0000","last_became_peered":"2024-09-12T12:56:11.291147+0000","last_unstale":"2024-09-12T14:17:54.577016+0000","last_undegraded":"2024-09-12T14:17:54.577016+0000","last_fullsized":"2024-09-12T14:17:54.577016+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:56:10.277688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:56:10.277688+0000","last_clean_scrub_stamp":"2024-09-12T12:56:10.277688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:23:38.950871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001237943,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"15'2","reported_seq":3892,"reported_epoch":1931,"state":"active+clean","last_fresh":"2024-09-12T14:17:56.650015+0000","last_change":"2024-09-12T12:56:13.292357+0000","last_active":"2024-09-12T14:17:56.650015+0000","last_peered":"2024-09-12T14:17:56.650015+0000","last_clean":"2024-09-12T14:17:56.650015+0000","last_became_active":"2024-09-12T12:56:11.295878+0000","last_became_peered":"2024-09-12T12:56:11.295878+0000","last_unstale":"2024-09-12T14:17:56.650015+0000","last_undegraded":"2024-09-12T14:17:56.650015+0000","last_fullsized":"2024-09-12T14:17:56.650015+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:56:10.277688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:56:10.277688+0000","last_clean_scrub_stamp":"2024-09-12T12:56:10.277688+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:53:37.770371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054639100000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":3886,"reported_epoch":1930,"state":"active+clean","last_fresh":"2024-09-12T14:17:54.738092+0000","last_change":"2024-09-12T12:56:13.355367+0000","last_active":"2024-09-12T14:17:54.738092+0000","last_peered":"2024-09-12T14:17:54.738092+0000","last_clean":"2024-09-12T14:17:54.738092+0000","last_became_active":"2024-09-12T12:56:11.295181+0000","last_became_peered":"2024-09-12T12:56:11.295181+0000","last_unstale":"2024-09-12T14:17:54.738092+0000","last_undegraded":"2024-09-12T14:17:54.738092+0000","last_fullsized":"2024-09-12T14:17:54.738092+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:56:10.277688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:56:10.277688+0000","last_clean_scrub_stamp":"2024-09-12T12:56:10.277688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:20:30.198897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057238600000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":3887,"reported_epoch":1930,"state":"active+clean","last_fresh":"2024-09-12T14:17:54.742000+0000","last_change":"2024-09-12T12:56:13.355375+0000","last_active":"2024-09-12T14:17:54.742000+0000","last_peered":"2024-09-12T14:17:54.742000+0000","last_clean":"2024-09-12T14:17:54.742000+0000","last_became_active":"2024-09-12T12:56:11.293181+0000","last_became_peered":"2024-09-12T12:56:11.293181+0000","last_unstale":"2024-09-12T14:17:54.742000+0000","last_undegraded":"2024-09-12T14:17:54.742000+0000","last_fullsized":"2024-09-12T14:17:54.742000+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:56:10.277688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:56:10.277688+0000","last_clean_scrub_stamp":"2024-09-12T12:56:10.277688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:41:47.856672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058228100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"13'1","reported_seq":3888,"reported_epoch":1930,"state":"active+clean","last_fresh":"2024-09-12T14:17:54.599336+0000","last_change":"2024-09-12T12:56:13.297158+0000","last_active":"2024-09-12T14:17:54.599336+0000","last_peered":"2024-09-12T14:17:54.599336+0000","last_clean":"2024-09-12T14:17:54.599336+0000","last_became_active":"2024-09-12T12:56:11.290474+0000","last_became_peered":"2024-09-12T12:56:11.290474+0000","last_unstale":"2024-09-12T14:17:54.599336+0000","last_undegraded":"2024-09-12T14:17:54.599336+0000","last_fullsized":"2024-09-12T14:17:54.599336+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:56:10.277688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:56:10.277688+0000","last_clean_scrub_stamp":"2024-09-12T12:56:10.277688+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:29:42.607257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001374526,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":4014,"reported_epoch":1930,"state":"active+clean","last_fresh":"2024-09-12T14:17:54.594048+0000","last_change":"2024-09-12T12:56:09.442875+0000","last_active":"2024-09-12T14:17:54.594048+0000","last_peered":"2024-09-12T14:17:54.594048+0000","last_clean":"2024-09-12T14:17:54.594048+0000","last_became_active":"2024-09-12T12:56:09.442486+0000","last_became_peered":"2024-09-12T12:56:09.442486+0000","last_unstale":"2024-09-12T14:17:54.594048+0000","last_undegraded":"2024-09-12T14:17:54.594048+0000","last_fullsized":"2024-09-12T14:17:54.594048+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:56:08.256849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:56:08.256849+0000","last_clean_scrub_stamp":"2024-09-12T12:56:08.256849+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T00:45:42.906851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359739355,"num_pgs":82,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":775516,"kb_used_data":4360,"kb_used_omap":1,"kb_used_meta":771134,"kb_avail":93596324,"statfs":{"total":96636764160,"available":95842635776,"internally_reserved":0,"allocated":4464640,"data_stored":2475849,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":789641669},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":8,"seq":34359739353,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":717796,"kb_used_data":4944,"kb_used_omap":1,"kb_used_meta":712830,"kb_avail":93654044,"statfs":{"total":96636764160,"available":95901741056,"internally_reserved":0,"allocated":5062656,"data_stored":3066236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":729938373},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359739353,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":884384,"kb_used_data":4940,"kb_used_omap":1,"kb_used_meta":879422,"kb_avail":93487456,"statfs":{"total":96636764160,"available":95731154944,"internally_reserved":0,"allocated":5058560,"data_stored":3066217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":900528581},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T14:18:02.844 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2024-09-12T12:56:10.277688+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=12, tm_hour=12, tm_min=56, tm_sec=10, tm_wday=3, tm_yday=256, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=12, tm_hour=14, tm_min=17, tm_sec=59, tm_wday=3, tm_yday=256, tm_isdst=0) 2024-09-12T14:18:02.844 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2024-09-12T12:56:10.277688+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=12, tm_hour=12, tm_min=56, tm_sec=10, tm_wday=3, tm_yday=256, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=12, tm_hour=14, tm_min=17, tm_sec=59, tm_wday=3, tm_yday=256, tm_isdst=0) 2024-09-12T14:18:02.844 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2024-09-12T12:56:10.277688+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=12, tm_hour=12, tm_min=56, tm_sec=10, tm_wday=3, tm_yday=256, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=12, tm_hour=14, tm_min=17, tm_sec=59, tm_wday=3, tm_yday=256, tm_isdst=0) 2024-09-12T14:18:02.845 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2024-09-12T12:56:10.277688+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=12, tm_hour=12, tm_min=56, tm_sec=10, tm_wday=3, tm_yday=256, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=12, tm_hour=14, tm_min=17, tm_sec=59, tm_wday=3, tm_yday=256, tm_isdst=0) 2024-09-12T14:18:02.845 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2024-09-12T12:56:10.277688+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=12, tm_hour=12, tm_min=56, tm_sec=10, tm_wday=3, tm_yday=256, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=12, tm_hour=14, tm_min=17, tm_sec=59, tm_wday=3, tm_yday=256, tm_isdst=0) 2024-09-12T14:18:02.845 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2024-09-12T12:56:10.277688+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=12, tm_hour=12, tm_min=56, tm_sec=10, tm_wday=3, tm_yday=256, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=12, tm_hour=14, tm_min=17, tm_sec=59, tm_wday=3, tm_yday=256, tm_isdst=0) 2024-09-12T14:18:02.845 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2024-09-12T12:56:10.277688+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=12, tm_hour=12, tm_min=56, tm_sec=10, tm_wday=3, tm_yday=256, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=12, tm_hour=14, tm_min=17, tm_sec=59, tm_wday=3, tm_yday=256, tm_isdst=0) 2024-09-12T14:18:02.845 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2024-09-12T12:56:10.277688+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=12, tm_hour=12, tm_min=56, tm_sec=10, tm_wday=3, tm_yday=256, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=12, tm_hour=14, tm_min=17, tm_sec=59, tm_wday=3, tm_yday=256, tm_isdst=0) 2024-09-12T14:18:02.845 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-09-12T12:56:08.256849+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=12, tm_hour=12, tm_min=56, tm_sec=8, tm_wday=3, tm_yday=256, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=12, tm_hour=14, tm_min=17, tm_sec=59, tm_wday=3, tm_yday=256, tm_isdst=0) 2024-09-12T14:18:02.845 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-09-12T14:18:22.846 DEBUG:teuthology.orchestra.run.smithi130:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster2 pg dump --format=json 2024-09-12T14:18:23.193 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-09-12T14:18:23.194 INFO:teuthology.orchestra.run.smithi130.stderr:dumped all 2024-09-12T14:18:23.212 INFO:teuthology.orchestra.run.smithi130.stdout:{"pg_ready":true,"pg_map":{"version":4397,"stamp":"2024-09-12T14:18:22.886783+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":139,"num_write_kb":1896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":104,"ondisk_log_size":104,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2395760,"kb_used_data":14272,"kb_used_omap":4,"kb_used_meta":2381435,"kb_avail":280719760,"statfs":{"total":289910292480,"available":287457034240,"internally_reserved":0,"allocated":14614528,"data_stored":8619334,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4785,"internal_metadata":2438589775},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002382"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":3896,"reported_epoch":1931,"state":"active+clean","last_fresh":"2024-09-12T14:18:05.505696+0000","last_change":"2024-09-12T14:18:05.505719+0000","last_active":"2024-09-12T14:18:05.505696+0000","last_peered":"2024-09-12T14:18:05.505696+0000","last_clean":"2024-09-12T14:18:05.505696+0000","last_became_active":"2024-09-12T12:56:11.291035+0000","last_became_peered":"2024-09-12T12:56:11.291035+0000","last_unstale":"2024-09-12T14:18:05.505696+0000","last_undegraded":"2024-09-12T14:18:05.505696+0000","last_fullsized":"2024-09-12T14:18:05.505696+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T14:18:05.505562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T14:18:05.505562+0000","last_clean_scrub_stamp":"2024-09-12T14:18:05.505562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:42:50.343743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001875938,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":3896,"reported_epoch":1931,"state":"active+clean","last_fresh":"2024-09-12T14:18:06.523445+0000","last_change":"2024-09-12T14:18:06.523492+0000","last_active":"2024-09-12T14:18:06.523445+0000","last_peered":"2024-09-12T14:18:06.523445+0000","last_clean":"2024-09-12T14:18:06.523445+0000","last_became_active":"2024-09-12T12:56:11.294341+0000","last_became_peered":"2024-09-12T12:56:11.294341+0000","last_unstale":"2024-09-12T14:18:06.523445+0000","last_undegraded":"2024-09-12T14:18:06.523445+0000","last_fullsized":"2024-09-12T14:18:06.523445+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T14:18:06.523330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T14:18:06.523330+0000","last_clean_scrub_stamp":"2024-09-12T14:18:06.523330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:00:01.690262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016741390000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":3898,"reported_epoch":1931,"state":"active+clean","last_fresh":"2024-09-12T14:18:07.516265+0000","last_change":"2024-09-12T14:18:07.516289+0000","last_active":"2024-09-12T14:18:07.516265+0000","last_peered":"2024-09-12T14:18:07.516265+0000","last_clean":"2024-09-12T14:18:07.516265+0000","last_became_active":"2024-09-12T12:56:11.294629+0000","last_became_peered":"2024-09-12T12:56:11.294629+0000","last_unstale":"2024-09-12T14:18:07.516265+0000","last_undegraded":"2024-09-12T14:18:07.516265+0000","last_fullsized":"2024-09-12T14:18:07.516265+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T14:18:07.516167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T14:18:07.516167+0000","last_clean_scrub_stamp":"2024-09-12T14:18:07.516167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:34:13.428693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018622739999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":3898,"reported_epoch":1931,"state":"active+clean","last_fresh":"2024-09-12T14:18:04.479260+0000","last_change":"2024-09-12T14:18:04.479285+0000","last_active":"2024-09-12T14:18:04.479260+0000","last_peered":"2024-09-12T14:18:04.479260+0000","last_clean":"2024-09-12T14:18:04.479260+0000","last_became_active":"2024-09-12T12:56:11.291147+0000","last_became_peered":"2024-09-12T12:56:11.291147+0000","last_unstale":"2024-09-12T14:18:04.479260+0000","last_undegraded":"2024-09-12T14:18:04.479260+0000","last_fullsized":"2024-09-12T14:18:04.479260+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T14:18:04.479142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T14:18:04.479142+0000","last_clean_scrub_stamp":"2024-09-12T14:18:04.479142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:31:04.363423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001237943,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"1931'3","reported_seq":3902,"reported_epoch":1931,"state":"active+clean","last_fresh":"2024-09-12T14:18:01.529613+0000","last_change":"2024-09-12T14:18:01.529638+0000","last_active":"2024-09-12T14:18:01.529613+0000","last_peered":"2024-09-12T14:18:01.529613+0000","last_clean":"2024-09-12T14:18:01.529613+0000","last_became_active":"2024-09-12T12:56:11.295878+0000","last_became_peered":"2024-09-12T12:56:11.295878+0000","last_unstale":"2024-09-12T14:18:01.529613+0000","last_undegraded":"2024-09-12T14:18:01.529613+0000","last_fullsized":"2024-09-12T14:18:01.529613+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"1931'3","last_scrub_stamp":"2024-09-12T14:18:01.529542+0000","last_deep_scrub":"1931'3","last_deep_scrub_stamp":"2024-09-12T14:18:01.529542+0000","last_clean_scrub_stamp":"2024-09-12T14:18:01.529542+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T01:29:58.611663+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0.00054639100000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":3897,"reported_epoch":1931,"state":"active+clean","last_fresh":"2024-09-12T14:18:03.372603+0000","last_change":"2024-09-12T14:18:03.372626+0000","last_active":"2024-09-12T14:18:03.372603+0000","last_peered":"2024-09-12T14:18:03.372603+0000","last_clean":"2024-09-12T14:18:03.372603+0000","last_became_active":"2024-09-12T12:56:11.295181+0000","last_became_peered":"2024-09-12T12:56:11.295181+0000","last_unstale":"2024-09-12T14:18:03.372603+0000","last_undegraded":"2024-09-12T14:18:03.372603+0000","last_fullsized":"2024-09-12T14:18:03.372603+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T14:18:03.372517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T14:18:03.372517+0000","last_clean_scrub_stamp":"2024-09-12T14:18:03.372517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:54:03.599150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057238600000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":3898,"reported_epoch":1931,"state":"active+clean","last_fresh":"2024-09-12T14:18:02.414080+0000","last_change":"2024-09-12T14:18:02.414100+0000","last_active":"2024-09-12T14:18:02.414080+0000","last_peered":"2024-09-12T14:18:02.414080+0000","last_clean":"2024-09-12T14:18:02.414080+0000","last_became_active":"2024-09-12T12:56:11.293181+0000","last_became_peered":"2024-09-12T12:56:11.293181+0000","last_unstale":"2024-09-12T14:18:02.414080+0000","last_undegraded":"2024-09-12T14:18:02.414080+0000","last_fullsized":"2024-09-12T14:18:02.414080+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T14:18:02.414013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T14:18:02.414013+0000","last_clean_scrub_stamp":"2024-09-12T14:18:02.414013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:38:35.788823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058228100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"1931'2","reported_seq":3900,"reported_epoch":1931,"state":"active+clean","last_fresh":"2024-09-12T14:18:03.503762+0000","last_change":"2024-09-12T14:18:03.503825+0000","last_active":"2024-09-12T14:18:03.503762+0000","last_peered":"2024-09-12T14:18:03.503762+0000","last_clean":"2024-09-12T14:18:03.503762+0000","last_became_active":"2024-09-12T12:56:11.290474+0000","last_became_peered":"2024-09-12T12:56:11.290474+0000","last_unstale":"2024-09-12T14:18:03.503762+0000","last_undegraded":"2024-09-12T14:18:03.503762+0000","last_fullsized":"2024-09-12T14:18:03.503762+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"1931'2","last_scrub_stamp":"2024-09-12T14:18:03.503655+0000","last_deep_scrub":"1931'2","last_deep_scrub_stamp":"2024-09-12T14:18:03.503655+0000","last_clean_scrub_stamp":"2024-09-12T14:18:03.503655+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T02:04:16.677542+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0.001374526,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"1931'99","reported_seq":4027,"reported_epoch":1931,"state":"active+clean","last_fresh":"2024-09-12T14:18:02.539541+0000","last_change":"2024-09-12T14:18:02.539572+0000","last_active":"2024-09-12T14:18:02.539541+0000","last_peered":"2024-09-12T14:18:02.539541+0000","last_clean":"2024-09-12T14:18:02.539541+0000","last_became_active":"2024-09-12T12:56:09.442486+0000","last_became_peered":"2024-09-12T12:56:09.442486+0000","last_unstale":"2024-09-12T14:18:02.539541+0000","last_undegraded":"2024-09-12T14:18:02.539541+0000","last_fullsized":"2024-09-12T14:18:02.539541+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"1931'99","last_scrub_stamp":"2024-09-12T14:18:02.539482+0000","last_deep_scrub":"1931'99","last_deep_scrub_stamp":"2024-09-12T14:18:02.539482+0000","last_clean_scrub_stamp":"2024-09-12T14:18:02.539482+0000","objects_scrubbed":2,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:12:57.971287+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":99,"ondisk_log_size":99,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359739360,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":793568,"kb_used_data":4368,"kb_used_omap":1,"kb_used_meta":789182,"kb_avail":93578272,"statfs":{"total":96636764160,"available":95824150528,"internally_reserved":0,"allocated":4472832,"data_stored":2479520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":808122821},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":8,"seq":34359739358,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":717800,"kb_used_data":4952,"kb_used_omap":1,"kb_used_meta":712830,"kb_avail":93654040,"statfs":{"total":96636764160,"available":95901736960,"internally_reserved":0,"allocated":5070848,"data_stored":3069907,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":729938373},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359739358,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":884392,"kb_used_data":4952,"kb_used_omap":1,"kb_used_meta":879422,"kb_avail":93487448,"statfs":{"total":96636764160,"available":95731146752,"internally_reserved":0,"allocated":5070848,"data_stored":3069907,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":900528581},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T14:18:23.213 DEBUG:teuthology.orchestra.run.smithi130:> sudo ceph --cluster cluster2 config set global mon_health_to_clog false 2024-09-12T14:18:23.570 INFO:teuthology.misc:Shutting down mds daemons... 2024-09-12T14:18:23.570 INFO:teuthology.misc:Shutting down osd daemons... 2024-09-12T14:18:23.571 DEBUG:tasks.ceph.cluster2.osd.0:waiting for process to exit 2024-09-12T14:18:23.571 INFO:teuthology.orchestra.run:waiting for 300 2024-09-12T14:18:25.805 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon cluster2.osd.0 is failed for ~0s 2024-09-12T14:18:29.673 INFO:tasks.ceph.cluster2.osd.0:Stopped 2024-09-12T14:18:29.673 DEBUG:tasks.ceph.cluster2.osd.1:waiting for process to exit 2024-09-12T14:18:29.673 INFO:teuthology.orchestra.run:waiting for 300 2024-09-12T14:18:31.108 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon cluster2.osd.1 is failed for ~0s 2024-09-12T14:18:31.108 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon cluster2.osd.0 has been restored 2024-09-12T14:18:35.774 INFO:tasks.ceph.cluster2.osd.1:Stopped 2024-09-12T14:18:35.775 DEBUG:tasks.ceph.cluster2.osd.2:waiting for process to exit 2024-09-12T14:18:35.775 INFO:teuthology.orchestra.run:waiting for 300 2024-09-12T14:18:36.310 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon cluster2.osd.2 is failed for ~0s 2024-09-12T14:18:36.310 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon cluster2.osd.1 has been restored 2024-09-12T14:18:41.513 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon cluster2.osd.2 is failed for ~5s 2024-09-12T14:18:41.877 INFO:tasks.ceph.cluster2.osd.2:Stopped 2024-09-12T14:18:41.877 INFO:teuthology.misc:Shutting down mgr daemons... 2024-09-12T14:18:41.877 DEBUG:tasks.ceph.cluster2.mgr.x:waiting for process to exit 2024-09-12T14:18:41.877 INFO:teuthology.orchestra.run:waiting for 300 2024-09-12T14:18:41.962 INFO:tasks.ceph.cluster2.mgr.x:Stopped 2024-09-12T14:18:41.962 INFO:teuthology.misc:Shutting down mon daemons... 2024-09-12T14:18:41.962 DEBUG:tasks.ceph.cluster2.mon.a:waiting for process to exit 2024-09-12T14:18:41.962 INFO:teuthology.orchestra.run:waiting for 300 2024-09-12T14:18:42.000 INFO:tasks.ceph.cluster2.mon.a:Stopped 2024-09-12T14:18:42.000 INFO:tasks.ceph:Checking cluster log for badness... 2024-09-12T14:18:42.000 DEBUG:teuthology.orchestra.run.smithi130:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/cluster2.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | head -n 1 2024-09-12T14:18:42.174 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/cluster2-0 on ubuntu@smithi130.front.sepia.ceph.com 2024-09-12T14:18:42.174 DEBUG:teuthology.orchestra.run.smithi130:> sync && sudo umount -f /var/lib/ceph/osd/cluster2-0 2024-09-12T14:18:42.458 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/cluster2-1 on ubuntu@smithi130.front.sepia.ceph.com 2024-09-12T14:18:42.458 DEBUG:teuthology.orchestra.run.smithi130:> sync && sudo umount -f /var/lib/ceph/osd/cluster2-1 2024-09-12T14:18:42.599 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/cluster2-2 on ubuntu@smithi130.front.sepia.ceph.com 2024-09-12T14:18:42.600 DEBUG:teuthology.orchestra.run.smithi130:> sync && sudo umount -f /var/lib/ceph/osd/cluster2-2 2024-09-12T14:18:42.734 INFO:tasks.ceph:Archiving mon data... 2024-09-12T14:18:42.735 DEBUG:teuthology.misc:Transferring archived files from smithi130:/var/lib/ceph/mon/cluster2-a to /home/teuthworker/archive/teuthology-2024-09-11_21:48:03-krbd-squid-testing-default-smithi/7901765/data/cluster2.mon.a.tgz 2024-09-12T14:18:42.735 DEBUG:teuthology.orchestra.run.smithi130:> mktemp 2024-09-12T14:18:42.774 INFO:teuthology.orchestra.run.smithi130.stdout:/tmp/tmp.emNipSOQ3F 2024-09-12T14:18:42.774 DEBUG:teuthology.orchestra.run.smithi130:> sudo tar cz -f - -C /var/lib/ceph/mon/cluster2-a -- . > /tmp/tmp.emNipSOQ3F 2024-09-12T14:18:42.987 DEBUG:teuthology.orchestra.run.smithi130:> sudo chmod 0666 /tmp/tmp.emNipSOQ3F 2024-09-12T14:18:43.092 DEBUG:teuthology.orchestra.remote:smithi130:/tmp/tmp.emNipSOQ3F is 482KB 2024-09-12T14:18:43.187 DEBUG:teuthology.orchestra.run.smithi130:> rm -fr /tmp/tmp.emNipSOQ3F 2024-09-12T14:18:43.222 INFO:tasks.ceph:Cleaning ceph cluster... 2024-09-12T14:18:43.222 DEBUG:teuthology.orchestra.run.smithi060:> sudo rm -rf -- /etc/ceph/cluster2.conf /etc/ceph/cluster2.keyring /home/ubuntu/cephtest/cluster2.data /home/ubuntu/cephtest/cluster2.monmap /home/ubuntu/cephtest/../*.pid 2024-09-12T14:18:43.227 DEBUG:teuthology.orchestra.run.smithi130:> sudo rm -rf -- /etc/ceph/cluster2.conf /etc/ceph/cluster2.keyring /home/ubuntu/cephtest/cluster2.data /home/ubuntu/cephtest/cluster2.monmap /home/ubuntu/cephtest/../*.pid 2024-09-12T14:18:43.469 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-09-12T14:18:43.480 INFO:tasks.ceph.ceph_manager.cluster1:waiting for clean 2024-09-12T14:18:43.480 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster1 pg dump --format=json 2024-09-12T14:18:44.042 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-12T14:18:44.042 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2024-09-12T14:18:44.064 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":4428,"stamp":"2024-09-12T14:18:43.061649+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":139,"num_write_kb":1896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":100,"ondisk_log_size":100,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":3705772,"kb_used_data":15148,"kb_used_omap":4,"kb_used_meta":3690555,"kb_avail":279409748,"statfs":{"total":289910292480,"available":286115581952,"internally_reserved":0,"allocated":15511552,"data_stored":8678680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4785,"internal_metadata":3779128655},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002093"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":3887,"reported_epoch":1931,"state":"active+clean","last_fresh":"2024-09-12T14:17:58.633518+0000","last_change":"2024-09-12T12:55:32.159226+0000","last_active":"2024-09-12T14:17:58.633518+0000","last_peered":"2024-09-12T14:17:58.633518+0000","last_clean":"2024-09-12T14:17:58.633518+0000","last_became_active":"2024-09-12T12:55:30.158364+0000","last_became_peered":"2024-09-12T12:55:30.158364+0000","last_unstale":"2024-09-12T14:17:58.633518+0000","last_undegraded":"2024-09-12T14:17:58.633518+0000","last_fullsized":"2024-09-12T14:17:58.633518+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:55:29.139739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:55:29.139739+0000","last_clean_scrub_stamp":"2024-09-12T12:55:29.139739+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:47:27.317250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095102900000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":3887,"reported_epoch":1931,"state":"active+clean","last_fresh":"2024-09-12T14:17:58.636338+0000","last_change":"2024-09-12T12:55:32.158781+0000","last_active":"2024-09-12T14:17:58.636338+0000","last_peered":"2024-09-12T14:17:58.636338+0000","last_clean":"2024-09-12T14:17:58.636338+0000","last_became_active":"2024-09-12T12:55:30.154763+0000","last_became_peered":"2024-09-12T12:55:30.154763+0000","last_unstale":"2024-09-12T14:17:58.636338+0000","last_undegraded":"2024-09-12T14:17:58.636338+0000","last_fullsized":"2024-09-12T14:17:58.636338+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:55:29.139739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:55:29.139739+0000","last_clean_scrub_stamp":"2024-09-12T12:55:29.139739+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:59:24.414161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062363399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":3888,"reported_epoch":1931,"state":"active+clean","last_fresh":"2024-09-12T14:17:58.637867+0000","last_change":"2024-09-12T12:55:32.159553+0000","last_active":"2024-09-12T14:17:58.637867+0000","last_peered":"2024-09-12T14:17:58.637867+0000","last_clean":"2024-09-12T14:17:58.637867+0000","last_became_active":"2024-09-12T12:55:30.154871+0000","last_became_peered":"2024-09-12T12:55:30.154871+0000","last_unstale":"2024-09-12T14:17:58.637867+0000","last_undegraded":"2024-09-12T14:17:58.637867+0000","last_fullsized":"2024-09-12T14:17:58.637867+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:55:29.139739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:55:29.139739+0000","last_clean_scrub_stamp":"2024-09-12T12:55:29.139739+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:07:25.204569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010629509999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":3888,"reported_epoch":1931,"state":"active+clean","last_fresh":"2024-09-12T14:17:58.633203+0000","last_change":"2024-09-12T12:55:32.159226+0000","last_active":"2024-09-12T14:17:58.633203+0000","last_peered":"2024-09-12T14:17:58.633203+0000","last_clean":"2024-09-12T14:17:58.633203+0000","last_became_active":"2024-09-12T12:55:30.154781+0000","last_became_peered":"2024-09-12T12:55:30.154781+0000","last_unstale":"2024-09-12T14:17:58.633203+0000","last_undegraded":"2024-09-12T14:17:58.633203+0000","last_fullsized":"2024-09-12T14:17:58.633203+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:55:29.139739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:55:29.139739+0000","last_clean_scrub_stamp":"2024-09-12T12:55:29.139739+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:45:34.855250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010971749999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"16'2","reported_seq":3890,"reported_epoch":1931,"state":"active+clean","last_fresh":"2024-09-12T14:17:55.662088+0000","last_change":"2024-09-12T12:55:32.154984+0000","last_active":"2024-09-12T14:17:55.662088+0000","last_peered":"2024-09-12T14:17:55.662088+0000","last_clean":"2024-09-12T14:17:55.662088+0000","last_became_active":"2024-09-12T12:55:30.157400+0000","last_became_peered":"2024-09-12T12:55:30.157400+0000","last_unstale":"2024-09-12T14:17:55.662088+0000","last_undegraded":"2024-09-12T14:17:55.662088+0000","last_fullsized":"2024-09-12T14:17:55.662088+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:55:29.139739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:55:29.139739+0000","last_clean_scrub_stamp":"2024-09-12T12:55:29.139739+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:00:30.935389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051984500000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":3888,"reported_epoch":1931,"state":"active+clean","last_fresh":"2024-09-12T14:17:56.987470+0000","last_change":"2024-09-12T12:55:32.687689+0000","last_active":"2024-09-12T14:17:56.987470+0000","last_peered":"2024-09-12T14:17:56.987470+0000","last_clean":"2024-09-12T14:17:56.987470+0000","last_became_active":"2024-09-12T12:55:30.157447+0000","last_became_peered":"2024-09-12T12:55:30.157447+0000","last_unstale":"2024-09-12T14:17:56.987470+0000","last_undegraded":"2024-09-12T14:17:56.987470+0000","last_fullsized":"2024-09-12T14:17:56.987470+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:55:29.139739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:55:29.139739+0000","last_clean_scrub_stamp":"2024-09-12T12:55:29.139739+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:11:12.106599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047483699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":3890,"reported_epoch":1931,"state":"active+clean","last_fresh":"2024-09-12T14:17:56.988612+0000","last_change":"2024-09-12T12:55:32.687614+0000","last_active":"2024-09-12T14:17:56.988612+0000","last_peered":"2024-09-12T14:17:56.988612+0000","last_clean":"2024-09-12T14:17:56.988612+0000","last_became_active":"2024-09-12T12:55:30.156802+0000","last_became_peered":"2024-09-12T12:55:30.156802+0000","last_unstale":"2024-09-12T14:17:56.988612+0000","last_undegraded":"2024-09-12T14:17:56.988612+0000","last_fullsized":"2024-09-12T14:17:56.988612+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:55:29.139739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:55:29.139739+0000","last_clean_scrub_stamp":"2024-09-12T12:55:29.139739+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T00:47:04.052477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039869699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"14'1","reported_seq":3889,"reported_epoch":1931,"state":"active+clean","last_fresh":"2024-09-12T14:17:58.636368+0000","last_change":"2024-09-12T12:55:32.158817+0000","last_active":"2024-09-12T14:17:58.636368+0000","last_peered":"2024-09-12T14:17:58.636368+0000","last_clean":"2024-09-12T14:17:58.636368+0000","last_became_active":"2024-09-12T12:55:30.154063+0000","last_became_peered":"2024-09-12T12:55:30.154063+0000","last_unstale":"2024-09-12T14:17:58.636368+0000","last_undegraded":"2024-09-12T14:17:58.636368+0000","last_fullsized":"2024-09-12T14:17:58.636368+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:55:29.139739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:55:29.139739+0000","last_clean_scrub_stamp":"2024-09-12T12:55:29.139739+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:40:17.400915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062050300000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"11'97","reported_seq":4017,"reported_epoch":1931,"state":"active+clean","last_fresh":"2024-09-12T14:17:58.636778+0000","last_change":"2024-09-12T12:55:27.318143+0000","last_active":"2024-09-12T14:17:58.636778+0000","last_peered":"2024-09-12T14:17:58.636778+0000","last_clean":"2024-09-12T14:17:58.636778+0000","last_became_active":"2024-09-12T12:55:27.317743+0000","last_became_peered":"2024-09-12T12:55:27.317743+0000","last_unstale":"2024-09-12T14:17:58.636778+0000","last_undegraded":"2024-09-12T14:17:58.636778+0000","last_fullsized":"2024-09-12T14:17:58.636778+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:55:26.132558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:55:26.132558+0000","last_clean_scrub_stamp":"2024-09-12T12:55:26.132558+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:19:44.340577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359739370,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1078472,"kb_used_data":4660,"kb_used_omap":1,"kb_used_meta":1073790,"kb_avail":93293368,"statfs":{"total":96636764160,"available":95532408832,"internally_reserved":0,"allocated":4771840,"data_stored":2499302,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":1099561413},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":8,"seq":34359739370,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1295632,"kb_used_data":5244,"kb_used_omap":1,"kb_used_meta":1290366,"kb_avail":93076208,"statfs":{"total":96636764160,"available":95310036992,"internally_reserved":0,"allocated":5369856,"data_stored":3089689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":1321335237},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359739371,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1331668,"kb_used_data":5244,"kb_used_omap":1,"kb_used_meta":1326398,"kb_avail":93040172,"statfs":{"total":96636764160,"available":95273136128,"internally_reserved":0,"allocated":5369856,"data_stored":3089689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":1358232005},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T14:18:44.065 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster1 pg dump --format=json 2024-09-12T14:18:44.423 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-12T14:18:44.423 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2024-09-12T14:18:44.441 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":4428,"stamp":"2024-09-12T14:18:43.061649+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":139,"num_write_kb":1896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":100,"ondisk_log_size":100,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":3705772,"kb_used_data":15148,"kb_used_omap":4,"kb_used_meta":3690555,"kb_avail":279409748,"statfs":{"total":289910292480,"available":286115581952,"internally_reserved":0,"allocated":15511552,"data_stored":8678680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4785,"internal_metadata":3779128655},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002093"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":3887,"reported_epoch":1931,"state":"active+clean","last_fresh":"2024-09-12T14:17:58.633518+0000","last_change":"2024-09-12T12:55:32.159226+0000","last_active":"2024-09-12T14:17:58.633518+0000","last_peered":"2024-09-12T14:17:58.633518+0000","last_clean":"2024-09-12T14:17:58.633518+0000","last_became_active":"2024-09-12T12:55:30.158364+0000","last_became_peered":"2024-09-12T12:55:30.158364+0000","last_unstale":"2024-09-12T14:17:58.633518+0000","last_undegraded":"2024-09-12T14:17:58.633518+0000","last_fullsized":"2024-09-12T14:17:58.633518+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:55:29.139739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:55:29.139739+0000","last_clean_scrub_stamp":"2024-09-12T12:55:29.139739+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:47:27.317250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095102900000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":3887,"reported_epoch":1931,"state":"active+clean","last_fresh":"2024-09-12T14:17:58.636338+0000","last_change":"2024-09-12T12:55:32.158781+0000","last_active":"2024-09-12T14:17:58.636338+0000","last_peered":"2024-09-12T14:17:58.636338+0000","last_clean":"2024-09-12T14:17:58.636338+0000","last_became_active":"2024-09-12T12:55:30.154763+0000","last_became_peered":"2024-09-12T12:55:30.154763+0000","last_unstale":"2024-09-12T14:17:58.636338+0000","last_undegraded":"2024-09-12T14:17:58.636338+0000","last_fullsized":"2024-09-12T14:17:58.636338+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:55:29.139739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:55:29.139739+0000","last_clean_scrub_stamp":"2024-09-12T12:55:29.139739+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:59:24.414161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062363399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":3888,"reported_epoch":1931,"state":"active+clean","last_fresh":"2024-09-12T14:17:58.637867+0000","last_change":"2024-09-12T12:55:32.159553+0000","last_active":"2024-09-12T14:17:58.637867+0000","last_peered":"2024-09-12T14:17:58.637867+0000","last_clean":"2024-09-12T14:17:58.637867+0000","last_became_active":"2024-09-12T12:55:30.154871+0000","last_became_peered":"2024-09-12T12:55:30.154871+0000","last_unstale":"2024-09-12T14:17:58.637867+0000","last_undegraded":"2024-09-12T14:17:58.637867+0000","last_fullsized":"2024-09-12T14:17:58.637867+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:55:29.139739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:55:29.139739+0000","last_clean_scrub_stamp":"2024-09-12T12:55:29.139739+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:07:25.204569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010629509999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":3888,"reported_epoch":1931,"state":"active+clean","last_fresh":"2024-09-12T14:17:58.633203+0000","last_change":"2024-09-12T12:55:32.159226+0000","last_active":"2024-09-12T14:17:58.633203+0000","last_peered":"2024-09-12T14:17:58.633203+0000","last_clean":"2024-09-12T14:17:58.633203+0000","last_became_active":"2024-09-12T12:55:30.154781+0000","last_became_peered":"2024-09-12T12:55:30.154781+0000","last_unstale":"2024-09-12T14:17:58.633203+0000","last_undegraded":"2024-09-12T14:17:58.633203+0000","last_fullsized":"2024-09-12T14:17:58.633203+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:55:29.139739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:55:29.139739+0000","last_clean_scrub_stamp":"2024-09-12T12:55:29.139739+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:45:34.855250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010971749999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"16'2","reported_seq":3890,"reported_epoch":1931,"state":"active+clean","last_fresh":"2024-09-12T14:17:55.662088+0000","last_change":"2024-09-12T12:55:32.154984+0000","last_active":"2024-09-12T14:17:55.662088+0000","last_peered":"2024-09-12T14:17:55.662088+0000","last_clean":"2024-09-12T14:17:55.662088+0000","last_became_active":"2024-09-12T12:55:30.157400+0000","last_became_peered":"2024-09-12T12:55:30.157400+0000","last_unstale":"2024-09-12T14:17:55.662088+0000","last_undegraded":"2024-09-12T14:17:55.662088+0000","last_fullsized":"2024-09-12T14:17:55.662088+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:55:29.139739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:55:29.139739+0000","last_clean_scrub_stamp":"2024-09-12T12:55:29.139739+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:00:30.935389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051984500000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":3888,"reported_epoch":1931,"state":"active+clean","last_fresh":"2024-09-12T14:17:56.987470+0000","last_change":"2024-09-12T12:55:32.687689+0000","last_active":"2024-09-12T14:17:56.987470+0000","last_peered":"2024-09-12T14:17:56.987470+0000","last_clean":"2024-09-12T14:17:56.987470+0000","last_became_active":"2024-09-12T12:55:30.157447+0000","last_became_peered":"2024-09-12T12:55:30.157447+0000","last_unstale":"2024-09-12T14:17:56.987470+0000","last_undegraded":"2024-09-12T14:17:56.987470+0000","last_fullsized":"2024-09-12T14:17:56.987470+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:55:29.139739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:55:29.139739+0000","last_clean_scrub_stamp":"2024-09-12T12:55:29.139739+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:11:12.106599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047483699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":3890,"reported_epoch":1931,"state":"active+clean","last_fresh":"2024-09-12T14:17:56.988612+0000","last_change":"2024-09-12T12:55:32.687614+0000","last_active":"2024-09-12T14:17:56.988612+0000","last_peered":"2024-09-12T14:17:56.988612+0000","last_clean":"2024-09-12T14:17:56.988612+0000","last_became_active":"2024-09-12T12:55:30.156802+0000","last_became_peered":"2024-09-12T12:55:30.156802+0000","last_unstale":"2024-09-12T14:17:56.988612+0000","last_undegraded":"2024-09-12T14:17:56.988612+0000","last_fullsized":"2024-09-12T14:17:56.988612+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:55:29.139739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:55:29.139739+0000","last_clean_scrub_stamp":"2024-09-12T12:55:29.139739+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T00:47:04.052477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039869699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"14'1","reported_seq":3889,"reported_epoch":1931,"state":"active+clean","last_fresh":"2024-09-12T14:17:58.636368+0000","last_change":"2024-09-12T12:55:32.158817+0000","last_active":"2024-09-12T14:17:58.636368+0000","last_peered":"2024-09-12T14:17:58.636368+0000","last_clean":"2024-09-12T14:17:58.636368+0000","last_became_active":"2024-09-12T12:55:30.154063+0000","last_became_peered":"2024-09-12T12:55:30.154063+0000","last_unstale":"2024-09-12T14:17:58.636368+0000","last_undegraded":"2024-09-12T14:17:58.636368+0000","last_fullsized":"2024-09-12T14:17:58.636368+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:55:29.139739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:55:29.139739+0000","last_clean_scrub_stamp":"2024-09-12T12:55:29.139739+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:40:17.400915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062050300000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"11'97","reported_seq":4017,"reported_epoch":1931,"state":"active+clean","last_fresh":"2024-09-12T14:17:58.636778+0000","last_change":"2024-09-12T12:55:27.318143+0000","last_active":"2024-09-12T14:17:58.636778+0000","last_peered":"2024-09-12T14:17:58.636778+0000","last_clean":"2024-09-12T14:17:58.636778+0000","last_became_active":"2024-09-12T12:55:27.317743+0000","last_became_peered":"2024-09-12T12:55:27.317743+0000","last_unstale":"2024-09-12T14:17:58.636778+0000","last_undegraded":"2024-09-12T14:17:58.636778+0000","last_fullsized":"2024-09-12T14:17:58.636778+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:55:26.132558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:55:26.132558+0000","last_clean_scrub_stamp":"2024-09-12T12:55:26.132558+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:19:44.340577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359739370,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1078472,"kb_used_data":4660,"kb_used_omap":1,"kb_used_meta":1073790,"kb_avail":93293368,"statfs":{"total":96636764160,"available":95532408832,"internally_reserved":0,"allocated":4771840,"data_stored":2499302,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":1099561413},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":8,"seq":34359739370,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1295632,"kb_used_data":5244,"kb_used_omap":1,"kb_used_meta":1290366,"kb_avail":93076208,"statfs":{"total":96636764160,"available":95310036992,"internally_reserved":0,"allocated":5369856,"data_stored":3089689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":1321335237},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359739371,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1331668,"kb_used_data":5244,"kb_used_omap":1,"kb_used_meta":1326398,"kb_avail":93040172,"statfs":{"total":96636764160,"available":95273136128,"internally_reserved":0,"allocated":5369856,"data_stored":3089689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":1358232005},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T14:18:44.442 INFO:tasks.ceph.ceph_manager.cluster1:clean! 2024-09-12T14:18:44.442 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster1 pg dump --format=json 2024-09-12T14:18:44.799 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-12T14:18:44.799 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2024-09-12T14:18:44.816 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":4428,"stamp":"2024-09-12T14:18:43.061649+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":139,"num_write_kb":1896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":100,"ondisk_log_size":100,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":3705772,"kb_used_data":15148,"kb_used_omap":4,"kb_used_meta":3690555,"kb_avail":279409748,"statfs":{"total":289910292480,"available":286115581952,"internally_reserved":0,"allocated":15511552,"data_stored":8678680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4785,"internal_metadata":3779128655},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002093"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":3887,"reported_epoch":1931,"state":"active+clean","last_fresh":"2024-09-12T14:17:58.633518+0000","last_change":"2024-09-12T12:55:32.159226+0000","last_active":"2024-09-12T14:17:58.633518+0000","last_peered":"2024-09-12T14:17:58.633518+0000","last_clean":"2024-09-12T14:17:58.633518+0000","last_became_active":"2024-09-12T12:55:30.158364+0000","last_became_peered":"2024-09-12T12:55:30.158364+0000","last_unstale":"2024-09-12T14:17:58.633518+0000","last_undegraded":"2024-09-12T14:17:58.633518+0000","last_fullsized":"2024-09-12T14:17:58.633518+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:55:29.139739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:55:29.139739+0000","last_clean_scrub_stamp":"2024-09-12T12:55:29.139739+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:47:27.317250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095102900000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":3887,"reported_epoch":1931,"state":"active+clean","last_fresh":"2024-09-12T14:17:58.636338+0000","last_change":"2024-09-12T12:55:32.158781+0000","last_active":"2024-09-12T14:17:58.636338+0000","last_peered":"2024-09-12T14:17:58.636338+0000","last_clean":"2024-09-12T14:17:58.636338+0000","last_became_active":"2024-09-12T12:55:30.154763+0000","last_became_peered":"2024-09-12T12:55:30.154763+0000","last_unstale":"2024-09-12T14:17:58.636338+0000","last_undegraded":"2024-09-12T14:17:58.636338+0000","last_fullsized":"2024-09-12T14:17:58.636338+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:55:29.139739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:55:29.139739+0000","last_clean_scrub_stamp":"2024-09-12T12:55:29.139739+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:59:24.414161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062363399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":3888,"reported_epoch":1931,"state":"active+clean","last_fresh":"2024-09-12T14:17:58.637867+0000","last_change":"2024-09-12T12:55:32.159553+0000","last_active":"2024-09-12T14:17:58.637867+0000","last_peered":"2024-09-12T14:17:58.637867+0000","last_clean":"2024-09-12T14:17:58.637867+0000","last_became_active":"2024-09-12T12:55:30.154871+0000","last_became_peered":"2024-09-12T12:55:30.154871+0000","last_unstale":"2024-09-12T14:17:58.637867+0000","last_undegraded":"2024-09-12T14:17:58.637867+0000","last_fullsized":"2024-09-12T14:17:58.637867+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:55:29.139739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:55:29.139739+0000","last_clean_scrub_stamp":"2024-09-12T12:55:29.139739+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:07:25.204569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010629509999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":3888,"reported_epoch":1931,"state":"active+clean","last_fresh":"2024-09-12T14:17:58.633203+0000","last_change":"2024-09-12T12:55:32.159226+0000","last_active":"2024-09-12T14:17:58.633203+0000","last_peered":"2024-09-12T14:17:58.633203+0000","last_clean":"2024-09-12T14:17:58.633203+0000","last_became_active":"2024-09-12T12:55:30.154781+0000","last_became_peered":"2024-09-12T12:55:30.154781+0000","last_unstale":"2024-09-12T14:17:58.633203+0000","last_undegraded":"2024-09-12T14:17:58.633203+0000","last_fullsized":"2024-09-12T14:17:58.633203+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:55:29.139739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:55:29.139739+0000","last_clean_scrub_stamp":"2024-09-12T12:55:29.139739+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:45:34.855250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010971749999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"16'2","reported_seq":3890,"reported_epoch":1931,"state":"active+clean","last_fresh":"2024-09-12T14:17:55.662088+0000","last_change":"2024-09-12T12:55:32.154984+0000","last_active":"2024-09-12T14:17:55.662088+0000","last_peered":"2024-09-12T14:17:55.662088+0000","last_clean":"2024-09-12T14:17:55.662088+0000","last_became_active":"2024-09-12T12:55:30.157400+0000","last_became_peered":"2024-09-12T12:55:30.157400+0000","last_unstale":"2024-09-12T14:17:55.662088+0000","last_undegraded":"2024-09-12T14:17:55.662088+0000","last_fullsized":"2024-09-12T14:17:55.662088+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:55:29.139739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:55:29.139739+0000","last_clean_scrub_stamp":"2024-09-12T12:55:29.139739+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:00:30.935389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051984500000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":3888,"reported_epoch":1931,"state":"active+clean","last_fresh":"2024-09-12T14:17:56.987470+0000","last_change":"2024-09-12T12:55:32.687689+0000","last_active":"2024-09-12T14:17:56.987470+0000","last_peered":"2024-09-12T14:17:56.987470+0000","last_clean":"2024-09-12T14:17:56.987470+0000","last_became_active":"2024-09-12T12:55:30.157447+0000","last_became_peered":"2024-09-12T12:55:30.157447+0000","last_unstale":"2024-09-12T14:17:56.987470+0000","last_undegraded":"2024-09-12T14:17:56.987470+0000","last_fullsized":"2024-09-12T14:17:56.987470+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:55:29.139739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:55:29.139739+0000","last_clean_scrub_stamp":"2024-09-12T12:55:29.139739+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:11:12.106599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047483699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":3890,"reported_epoch":1931,"state":"active+clean","last_fresh":"2024-09-12T14:17:56.988612+0000","last_change":"2024-09-12T12:55:32.687614+0000","last_active":"2024-09-12T14:17:56.988612+0000","last_peered":"2024-09-12T14:17:56.988612+0000","last_clean":"2024-09-12T14:17:56.988612+0000","last_became_active":"2024-09-12T12:55:30.156802+0000","last_became_peered":"2024-09-12T12:55:30.156802+0000","last_unstale":"2024-09-12T14:17:56.988612+0000","last_undegraded":"2024-09-12T14:17:56.988612+0000","last_fullsized":"2024-09-12T14:17:56.988612+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:55:29.139739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:55:29.139739+0000","last_clean_scrub_stamp":"2024-09-12T12:55:29.139739+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T00:47:04.052477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039869699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"14'1","reported_seq":3889,"reported_epoch":1931,"state":"active+clean","last_fresh":"2024-09-12T14:17:58.636368+0000","last_change":"2024-09-12T12:55:32.158817+0000","last_active":"2024-09-12T14:17:58.636368+0000","last_peered":"2024-09-12T14:17:58.636368+0000","last_clean":"2024-09-12T14:17:58.636368+0000","last_became_active":"2024-09-12T12:55:30.154063+0000","last_became_peered":"2024-09-12T12:55:30.154063+0000","last_unstale":"2024-09-12T14:17:58.636368+0000","last_undegraded":"2024-09-12T14:17:58.636368+0000","last_fullsized":"2024-09-12T14:17:58.636368+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:55:29.139739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:55:29.139739+0000","last_clean_scrub_stamp":"2024-09-12T12:55:29.139739+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:40:17.400915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062050300000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"11'97","reported_seq":4017,"reported_epoch":1931,"state":"active+clean","last_fresh":"2024-09-12T14:17:58.636778+0000","last_change":"2024-09-12T12:55:27.318143+0000","last_active":"2024-09-12T14:17:58.636778+0000","last_peered":"2024-09-12T14:17:58.636778+0000","last_clean":"2024-09-12T14:17:58.636778+0000","last_became_active":"2024-09-12T12:55:27.317743+0000","last_became_peered":"2024-09-12T12:55:27.317743+0000","last_unstale":"2024-09-12T14:17:58.636778+0000","last_undegraded":"2024-09-12T14:17:58.636778+0000","last_fullsized":"2024-09-12T14:17:58.636778+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:55:26.132558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:55:26.132558+0000","last_clean_scrub_stamp":"2024-09-12T12:55:26.132558+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:19:44.340577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359739370,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1078472,"kb_used_data":4660,"kb_used_omap":1,"kb_used_meta":1073790,"kb_avail":93293368,"statfs":{"total":96636764160,"available":95532408832,"internally_reserved":0,"allocated":4771840,"data_stored":2499302,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":1099561413},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":8,"seq":34359739370,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1295632,"kb_used_data":5244,"kb_used_omap":1,"kb_used_meta":1290366,"kb_avail":93076208,"statfs":{"total":96636764160,"available":95310036992,"internally_reserved":0,"allocated":5369856,"data_stored":3089689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":1321335237},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359739371,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1331668,"kb_used_data":5244,"kb_used_omap":1,"kb_used_meta":1326398,"kb_avail":93040172,"statfs":{"total":96636764160,"available":95273136128,"internally_reserved":0,"allocated":5369856,"data_stored":3089689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":1358232005},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T14:18:44.817 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster1 osd dump --format=json 2024-09-12T14:18:45.176 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-12T14:18:45.176 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":1931,"fsid":"2a63e29a-4caf-4cc6-962c-13f8b91a9d2b","created":"2024-09-12T12:55:12.622095+0000","modified":"2024-09-12T14:17:55.652633+0000","last_up_change":"2024-09-12T12:55:24.130456+0000","last_in_change":"2024-09-12T12:55:13.728922+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T12:55:25.770242+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-12T12:55:28.339452+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":16,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8a419ad1-c999-4a27-8870-2d6e505f6398","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":156850008},{"type":"v1","addr":"172.21.15.60:6809","nonce":156850008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":156850008},{"type":"v1","addr":"172.21.15.60:6811","nonce":156850008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":156850008},{"type":"v1","addr":"172.21.15.60:6815","nonce":156850008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":156850008},{"type":"v1","addr":"172.21.15.60:6813","nonce":156850008}]},"public_addr":"172.21.15.60:6809/156850008","cluster_addr":"172.21.15.60:6811/156850008","heartbeat_back_addr":"172.21.15.60:6815/156850008","heartbeat_front_addr":"172.21.15.60:6813/156850008","state":["exists","up"]},{"osd":1,"uuid":"2b198f71-f471-4b42-9d29-f25f9f69a1e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":1750992701},{"type":"v1","addr":"172.21.15.60:6801","nonce":1750992701}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":1750992701},{"type":"v1","addr":"172.21.15.60:6803","nonce":1750992701}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":1750992701},{"type":"v1","addr":"172.21.15.60:6807","nonce":1750992701}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":1750992701},{"type":"v1","addr":"172.21.15.60:6805","nonce":1750992701}]},"public_addr":"172.21.15.60:6801/1750992701","cluster_addr":"172.21.15.60:6803/1750992701","heartbeat_back_addr":"172.21.15.60:6807/1750992701","heartbeat_front_addr":"172.21.15.60:6805/1750992701","state":["exists","up"]},{"osd":2,"uuid":"39c37b7a-038d-49f6-b67c-78a015c7966c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":37652449},{"type":"v1","addr":"172.21.15.60:6817","nonce":37652449}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":37652449},{"type":"v1","addr":"172.21.15.60:6819","nonce":37652449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":37652449},{"type":"v1","addr":"172.21.15.60:6823","nonce":37652449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":37652449},{"type":"v1","addr":"172.21.15.60:6821","nonce":37652449}]},"public_addr":"172.21.15.60:6817/37652449","cluster_addr":"172.21.15.60:6819/37652449","heartbeat_back_addr":"172.21.15.60:6823/37652449","heartbeat_front_addr":"172.21.15.60:6821/37652449","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:55:21.424833+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:55:21.274503+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:55:21.542972+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":3,"snaps":[{"begin":2979,"length":2},{"begin":2991,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T14:18:46.195 INFO:tasks.ceph:Scrubbing cluster1.osd.0 2024-09-12T14:18:46.195 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster1 tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-09-12T14:18:46.388 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-12T14:18:46.389 INFO:teuthology.orchestra.run.smithi060.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-12T14:18:46.389 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-12T14:18:46.403 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster1 osd deep-scrub 0 2024-09-12T14:18:46.514 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon cluster2.osd.2 has been restored 2024-09-12T14:18:46.755 INFO:teuthology.orchestra.run.smithi060.stderr:instructed osd(s) 0 to deep-scrub 2024-09-12T14:18:46.772 INFO:tasks.ceph:Scrubbing cluster1.osd.1 2024-09-12T14:18:46.772 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster1 tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-09-12T14:18:46.953 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-12T14:18:46.954 INFO:teuthology.orchestra.run.smithi060.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-12T14:18:46.954 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-12T14:18:46.969 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster1 osd deep-scrub 1 2024-09-12T14:18:47.327 INFO:teuthology.orchestra.run.smithi060.stderr:instructed osd(s) 1 to deep-scrub 2024-09-12T14:18:47.345 INFO:tasks.ceph:Scrubbing cluster1.osd.2 2024-09-12T14:18:47.345 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster1 tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-09-12T14:18:47.528 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-12T14:18:47.529 INFO:teuthology.orchestra.run.smithi060.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-12T14:18:47.529 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-12T14:18:47.543 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster1 osd deep-scrub 2 2024-09-12T14:18:47.899 INFO:teuthology.orchestra.run.smithi060.stderr:instructed osd(s) 2 to deep-scrub 2024-09-12T14:18:47.917 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster1 pg dump --format=json 2024-09-12T14:18:48.265 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-12T14:18:48.265 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2024-09-12T14:18:48.283 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":4430,"stamp":"2024-09-12T14:18:47.062264+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":139,"num_write_kb":1896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":100,"ondisk_log_size":100,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":3705772,"kb_used_data":15148,"kb_used_omap":4,"kb_used_meta":3690555,"kb_avail":279409748,"statfs":{"total":289910292480,"available":286115581952,"internally_reserved":0,"allocated":15511552,"data_stored":8678680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4785,"internal_metadata":3779128655},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002060"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":3887,"reported_epoch":1931,"state":"active+clean","last_fresh":"2024-09-12T14:17:58.633518+0000","last_change":"2024-09-12T12:55:32.159226+0000","last_active":"2024-09-12T14:17:58.633518+0000","last_peered":"2024-09-12T14:17:58.633518+0000","last_clean":"2024-09-12T14:17:58.633518+0000","last_became_active":"2024-09-12T12:55:30.158364+0000","last_became_peered":"2024-09-12T12:55:30.158364+0000","last_unstale":"2024-09-12T14:17:58.633518+0000","last_undegraded":"2024-09-12T14:17:58.633518+0000","last_fullsized":"2024-09-12T14:17:58.633518+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:55:29.139739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:55:29.139739+0000","last_clean_scrub_stamp":"2024-09-12T12:55:29.139739+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:47:27.317250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095102900000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":3887,"reported_epoch":1931,"state":"active+clean","last_fresh":"2024-09-12T14:17:58.636338+0000","last_change":"2024-09-12T12:55:32.158781+0000","last_active":"2024-09-12T14:17:58.636338+0000","last_peered":"2024-09-12T14:17:58.636338+0000","last_clean":"2024-09-12T14:17:58.636338+0000","last_became_active":"2024-09-12T12:55:30.154763+0000","last_became_peered":"2024-09-12T12:55:30.154763+0000","last_unstale":"2024-09-12T14:17:58.636338+0000","last_undegraded":"2024-09-12T14:17:58.636338+0000","last_fullsized":"2024-09-12T14:17:58.636338+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:55:29.139739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:55:29.139739+0000","last_clean_scrub_stamp":"2024-09-12T12:55:29.139739+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:59:24.414161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062363399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":3888,"reported_epoch":1931,"state":"active+clean","last_fresh":"2024-09-12T14:17:58.637867+0000","last_change":"2024-09-12T12:55:32.159553+0000","last_active":"2024-09-12T14:17:58.637867+0000","last_peered":"2024-09-12T14:17:58.637867+0000","last_clean":"2024-09-12T14:17:58.637867+0000","last_became_active":"2024-09-12T12:55:30.154871+0000","last_became_peered":"2024-09-12T12:55:30.154871+0000","last_unstale":"2024-09-12T14:17:58.637867+0000","last_undegraded":"2024-09-12T14:17:58.637867+0000","last_fullsized":"2024-09-12T14:17:58.637867+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:55:29.139739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:55:29.139739+0000","last_clean_scrub_stamp":"2024-09-12T12:55:29.139739+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:07:25.204569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010629509999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":3888,"reported_epoch":1931,"state":"active+clean","last_fresh":"2024-09-12T14:17:58.633203+0000","last_change":"2024-09-12T12:55:32.159226+0000","last_active":"2024-09-12T14:17:58.633203+0000","last_peered":"2024-09-12T14:17:58.633203+0000","last_clean":"2024-09-12T14:17:58.633203+0000","last_became_active":"2024-09-12T12:55:30.154781+0000","last_became_peered":"2024-09-12T12:55:30.154781+0000","last_unstale":"2024-09-12T14:17:58.633203+0000","last_undegraded":"2024-09-12T14:17:58.633203+0000","last_fullsized":"2024-09-12T14:17:58.633203+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:55:29.139739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:55:29.139739+0000","last_clean_scrub_stamp":"2024-09-12T12:55:29.139739+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:45:34.855250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010971749999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"16'2","reported_seq":3890,"reported_epoch":1931,"state":"active+clean","last_fresh":"2024-09-12T14:17:55.662088+0000","last_change":"2024-09-12T12:55:32.154984+0000","last_active":"2024-09-12T14:17:55.662088+0000","last_peered":"2024-09-12T14:17:55.662088+0000","last_clean":"2024-09-12T14:17:55.662088+0000","last_became_active":"2024-09-12T12:55:30.157400+0000","last_became_peered":"2024-09-12T12:55:30.157400+0000","last_unstale":"2024-09-12T14:17:55.662088+0000","last_undegraded":"2024-09-12T14:17:55.662088+0000","last_fullsized":"2024-09-12T14:17:55.662088+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:55:29.139739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:55:29.139739+0000","last_clean_scrub_stamp":"2024-09-12T12:55:29.139739+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:00:30.935389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051984500000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":3888,"reported_epoch":1931,"state":"active+clean","last_fresh":"2024-09-12T14:17:56.987470+0000","last_change":"2024-09-12T12:55:32.687689+0000","last_active":"2024-09-12T14:17:56.987470+0000","last_peered":"2024-09-12T14:17:56.987470+0000","last_clean":"2024-09-12T14:17:56.987470+0000","last_became_active":"2024-09-12T12:55:30.157447+0000","last_became_peered":"2024-09-12T12:55:30.157447+0000","last_unstale":"2024-09-12T14:17:56.987470+0000","last_undegraded":"2024-09-12T14:17:56.987470+0000","last_fullsized":"2024-09-12T14:17:56.987470+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:55:29.139739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:55:29.139739+0000","last_clean_scrub_stamp":"2024-09-12T12:55:29.139739+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:11:12.106599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047483699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":3890,"reported_epoch":1931,"state":"active+clean","last_fresh":"2024-09-12T14:17:56.988612+0000","last_change":"2024-09-12T12:55:32.687614+0000","last_active":"2024-09-12T14:17:56.988612+0000","last_peered":"2024-09-12T14:17:56.988612+0000","last_clean":"2024-09-12T14:17:56.988612+0000","last_became_active":"2024-09-12T12:55:30.156802+0000","last_became_peered":"2024-09-12T12:55:30.156802+0000","last_unstale":"2024-09-12T14:17:56.988612+0000","last_undegraded":"2024-09-12T14:17:56.988612+0000","last_fullsized":"2024-09-12T14:17:56.988612+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:55:29.139739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:55:29.139739+0000","last_clean_scrub_stamp":"2024-09-12T12:55:29.139739+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T00:47:04.052477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039869699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"14'1","reported_seq":3889,"reported_epoch":1931,"state":"active+clean","last_fresh":"2024-09-12T14:17:58.636368+0000","last_change":"2024-09-12T12:55:32.158817+0000","last_active":"2024-09-12T14:17:58.636368+0000","last_peered":"2024-09-12T14:17:58.636368+0000","last_clean":"2024-09-12T14:17:58.636368+0000","last_became_active":"2024-09-12T12:55:30.154063+0000","last_became_peered":"2024-09-12T12:55:30.154063+0000","last_unstale":"2024-09-12T14:17:58.636368+0000","last_undegraded":"2024-09-12T14:17:58.636368+0000","last_fullsized":"2024-09-12T14:17:58.636368+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:55:29.139739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:55:29.139739+0000","last_clean_scrub_stamp":"2024-09-12T12:55:29.139739+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:40:17.400915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062050300000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"11'97","reported_seq":4017,"reported_epoch":1931,"state":"active+clean","last_fresh":"2024-09-12T14:17:58.636778+0000","last_change":"2024-09-12T12:55:27.318143+0000","last_active":"2024-09-12T14:17:58.636778+0000","last_peered":"2024-09-12T14:17:58.636778+0000","last_clean":"2024-09-12T14:17:58.636778+0000","last_became_active":"2024-09-12T12:55:27.317743+0000","last_became_peered":"2024-09-12T12:55:27.317743+0000","last_unstale":"2024-09-12T14:17:58.636778+0000","last_undegraded":"2024-09-12T14:17:58.636778+0000","last_fullsized":"2024-09-12T14:17:58.636778+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:55:26.132558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:55:26.132558+0000","last_clean_scrub_stamp":"2024-09-12T12:55:26.132558+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:19:44.340577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359739371,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1078472,"kb_used_data":4660,"kb_used_omap":1,"kb_used_meta":1073790,"kb_avail":93293368,"statfs":{"total":96636764160,"available":95532408832,"internally_reserved":0,"allocated":4771840,"data_stored":2499302,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":1099561413},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":8,"seq":34359739371,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1295632,"kb_used_data":5244,"kb_used_omap":1,"kb_used_meta":1290366,"kb_avail":93076208,"statfs":{"total":96636764160,"available":95310036992,"internally_reserved":0,"allocated":5369856,"data_stored":3089689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":1321335237},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359739372,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1331668,"kb_used_data":5244,"kb_used_omap":1,"kb_used_meta":1326398,"kb_avail":93040172,"statfs":{"total":96636764160,"available":95273136128,"internally_reserved":0,"allocated":5369856,"data_stored":3089689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":1358232005},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T14:18:48.284 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2024-09-12T12:55:29.139739+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=12, tm_hour=12, tm_min=55, tm_sec=29, tm_wday=3, tm_yday=256, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=12, tm_hour=14, tm_min=18, tm_sec=45, tm_wday=3, tm_yday=256, tm_isdst=0) 2024-09-12T14:18:48.284 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2024-09-12T12:55:29.139739+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=12, tm_hour=12, tm_min=55, tm_sec=29, tm_wday=3, tm_yday=256, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=12, tm_hour=14, tm_min=18, tm_sec=45, tm_wday=3, tm_yday=256, tm_isdst=0) 2024-09-12T14:18:48.284 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2024-09-12T12:55:29.139739+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=12, tm_hour=12, tm_min=55, tm_sec=29, tm_wday=3, tm_yday=256, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=12, tm_hour=14, tm_min=18, tm_sec=45, tm_wday=3, tm_yday=256, tm_isdst=0) 2024-09-12T14:18:48.284 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2024-09-12T12:55:29.139739+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=12, tm_hour=12, tm_min=55, tm_sec=29, tm_wday=3, tm_yday=256, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=12, tm_hour=14, tm_min=18, tm_sec=45, tm_wday=3, tm_yday=256, tm_isdst=0) 2024-09-12T14:18:48.284 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2024-09-12T12:55:29.139739+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=12, tm_hour=12, tm_min=55, tm_sec=29, tm_wday=3, tm_yday=256, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=12, tm_hour=14, tm_min=18, tm_sec=45, tm_wday=3, tm_yday=256, tm_isdst=0) 2024-09-12T14:18:48.284 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2024-09-12T12:55:29.139739+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=12, tm_hour=12, tm_min=55, tm_sec=29, tm_wday=3, tm_yday=256, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=12, tm_hour=14, tm_min=18, tm_sec=45, tm_wday=3, tm_yday=256, tm_isdst=0) 2024-09-12T14:18:48.284 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2024-09-12T12:55:29.139739+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=12, tm_hour=12, tm_min=55, tm_sec=29, tm_wday=3, tm_yday=256, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=12, tm_hour=14, tm_min=18, tm_sec=45, tm_wday=3, tm_yday=256, tm_isdst=0) 2024-09-12T14:18:48.285 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2024-09-12T12:55:29.139739+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=12, tm_hour=12, tm_min=55, tm_sec=29, tm_wday=3, tm_yday=256, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=12, tm_hour=14, tm_min=18, tm_sec=45, tm_wday=3, tm_yday=256, tm_isdst=0) 2024-09-12T14:18:48.285 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-09-12T12:55:26.132558+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=12, tm_hour=12, tm_min=55, tm_sec=26, tm_wday=3, tm_yday=256, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=12, tm_hour=14, tm_min=18, tm_sec=45, tm_wday=3, tm_yday=256, tm_isdst=0) 2024-09-12T14:18:48.285 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-09-12T14:19:08.286 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster1 pg dump --format=json 2024-09-12T14:19:08.649 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-12T14:19:08.649 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2024-09-12T14:19:08.666 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":4440,"stamp":"2024-09-12T14:19:07.066421+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":139,"num_write_kb":1896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":104,"ondisk_log_size":104,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":3705756,"kb_used_data":15148,"kb_used_omap":4,"kb_used_meta":3690555,"kb_avail":279409764,"statfs":{"total":289910292480,"available":286115598336,"internally_reserved":0,"allocated":15511552,"data_stored":8678680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4785,"internal_metadata":3779128655},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002477"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":3896,"reported_epoch":1931,"state":"active+clean","last_fresh":"2024-09-12T14:18:52.530150+0000","last_change":"2024-09-12T14:18:52.530180+0000","last_active":"2024-09-12T14:18:52.530150+0000","last_peered":"2024-09-12T14:18:52.530150+0000","last_clean":"2024-09-12T14:18:52.530150+0000","last_became_active":"2024-09-12T12:55:30.158364+0000","last_became_peered":"2024-09-12T12:55:30.158364+0000","last_unstale":"2024-09-12T14:18:52.530150+0000","last_undegraded":"2024-09-12T14:18:52.530150+0000","last_fullsized":"2024-09-12T14:18:52.530150+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T14:18:52.530035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T14:18:52.530035+0000","last_clean_scrub_stamp":"2024-09-12T14:18:52.530035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:18:14.736183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095102900000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":3896,"reported_epoch":1931,"state":"active+clean","last_fresh":"2024-09-12T14:18:49.460386+0000","last_change":"2024-09-12T14:18:49.460408+0000","last_active":"2024-09-12T14:18:49.460386+0000","last_peered":"2024-09-12T14:18:49.460386+0000","last_clean":"2024-09-12T14:18:49.460386+0000","last_became_active":"2024-09-12T12:55:30.154763+0000","last_became_peered":"2024-09-12T12:55:30.154763+0000","last_unstale":"2024-09-12T14:18:49.460386+0000","last_undegraded":"2024-09-12T14:18:49.460386+0000","last_fullsized":"2024-09-12T14:18:49.460386+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T14:18:49.460293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T14:18:49.460293+0000","last_clean_scrub_stamp":"2024-09-12T14:18:49.460293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:26:00.776472+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.00062363399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":3898,"reported_epoch":1931,"state":"active+clean","last_fresh":"2024-09-12T14:18:51.511067+0000","last_change":"2024-09-12T14:18:51.511067+0000","last_active":"2024-09-12T14:18:51.511067+0000","last_peered":"2024-09-12T14:18:51.511067+0000","last_clean":"2024-09-12T14:18:51.511067+0000","last_became_active":"2024-09-12T12:55:30.154871+0000","last_became_peered":"2024-09-12T12:55:30.154871+0000","last_unstale":"2024-09-12T14:18:51.511067+0000","last_undegraded":"2024-09-12T14:18:51.511067+0000","last_fullsized":"2024-09-12T14:18:51.511067+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T14:18:51.510949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T14:18:51.510949+0000","last_clean_scrub_stamp":"2024-09-12T14:18:51.510949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T00:03:22.764652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010629509999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":3898,"reported_epoch":1931,"state":"active+clean","last_fresh":"2024-09-12T14:18:50.461184+0000","last_change":"2024-09-12T14:18:50.461207+0000","last_active":"2024-09-12T14:18:50.461184+0000","last_peered":"2024-09-12T14:18:50.461184+0000","last_clean":"2024-09-12T14:18:50.461184+0000","last_became_active":"2024-09-12T12:55:30.154781+0000","last_became_peered":"2024-09-12T12:55:30.154781+0000","last_unstale":"2024-09-12T14:18:50.461184+0000","last_undegraded":"2024-09-12T14:18:50.461184+0000","last_fullsized":"2024-09-12T14:18:50.461184+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T14:18:50.461074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T14:18:50.461074+0000","last_clean_scrub_stamp":"2024-09-12T14:18:50.461074+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:00:47.198216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010971749999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"1931'3","reported_seq":3901,"reported_epoch":1931,"state":"active+clean","last_fresh":"2024-09-12T14:18:47.032605+0000","last_change":"2024-09-12T14:18:47.032605+0000","last_active":"2024-09-12T14:18:47.032605+0000","last_peered":"2024-09-12T14:18:47.032605+0000","last_clean":"2024-09-12T14:18:47.032605+0000","last_became_active":"2024-09-12T12:55:30.157400+0000","last_became_peered":"2024-09-12T12:55:30.157400+0000","last_unstale":"2024-09-12T14:18:47.032605+0000","last_undegraded":"2024-09-12T14:18:47.032605+0000","last_fullsized":"2024-09-12T14:18:47.032605+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"1931'3","last_scrub_stamp":"2024-09-12T14:18:47.032510+0000","last_deep_scrub":"1931'3","last_deep_scrub_stamp":"2024-09-12T14:18:47.032510+0000","last_clean_scrub_stamp":"2024-09-12T14:18:47.032510+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:48:13.905698+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0.00051984500000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":3898,"reported_epoch":1931,"state":"active+clean","last_fresh":"2024-09-12T14:18:49.067640+0000","last_change":"2024-09-12T14:18:49.067640+0000","last_active":"2024-09-12T14:18:49.067640+0000","last_peered":"2024-09-12T14:18:49.067640+0000","last_clean":"2024-09-12T14:18:49.067640+0000","last_became_active":"2024-09-12T12:55:30.157447+0000","last_became_peered":"2024-09-12T12:55:30.157447+0000","last_unstale":"2024-09-12T14:18:49.067640+0000","last_undegraded":"2024-09-12T14:18:49.067640+0000","last_fullsized":"2024-09-12T14:18:49.067640+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T14:18:49.067529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T14:18:49.067529+0000","last_clean_scrub_stamp":"2024-09-12T14:18:49.067529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T00:53:35.322482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047483699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":3899,"reported_epoch":1931,"state":"active+clean","last_fresh":"2024-09-12T14:18:48.022058+0000","last_change":"2024-09-12T14:18:48.022078+0000","last_active":"2024-09-12T14:18:48.022058+0000","last_peered":"2024-09-12T14:18:48.022058+0000","last_clean":"2024-09-12T14:18:48.022058+0000","last_became_active":"2024-09-12T12:55:30.156802+0000","last_became_peered":"2024-09-12T12:55:30.156802+0000","last_unstale":"2024-09-12T14:18:48.022058+0000","last_undegraded":"2024-09-12T14:18:48.022058+0000","last_fullsized":"2024-09-12T14:18:48.022058+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T14:18:48.021982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T14:18:48.021982+0000","last_clean_scrub_stamp":"2024-09-12T14:18:48.021982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:50:15.418867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039869699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"1931'2","reported_seq":3900,"reported_epoch":1931,"state":"active+clean","last_fresh":"2024-09-12T14:18:48.421954+0000","last_change":"2024-09-12T14:18:48.421954+0000","last_active":"2024-09-12T14:18:48.421954+0000","last_peered":"2024-09-12T14:18:48.421954+0000","last_clean":"2024-09-12T14:18:48.421954+0000","last_became_active":"2024-09-12T12:55:30.154063+0000","last_became_peered":"2024-09-12T12:55:30.154063+0000","last_unstale":"2024-09-12T14:18:48.421954+0000","last_undegraded":"2024-09-12T14:18:48.421954+0000","last_fullsized":"2024-09-12T14:18:48.421954+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"1931'2","last_scrub_stamp":"2024-09-12T14:18:48.421847+0000","last_deep_scrub":"1931'2","last_deep_scrub_stamp":"2024-09-12T14:18:48.421847+0000","last_clean_scrub_stamp":"2024-09-12T14:18:48.421847+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T01:09:27.456324+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.00062050300000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"1931'99","reported_seq":4028,"reported_epoch":1931,"state":"active+clean","last_fresh":"2024-09-12T14:18:47.440253+0000","last_change":"2024-09-12T14:18:47.440287+0000","last_active":"2024-09-12T14:18:47.440253+0000","last_peered":"2024-09-12T14:18:47.440253+0000","last_clean":"2024-09-12T14:18:47.440253+0000","last_became_active":"2024-09-12T12:55:27.317743+0000","last_became_peered":"2024-09-12T12:55:27.317743+0000","last_unstale":"2024-09-12T14:18:47.440253+0000","last_undegraded":"2024-09-12T14:18:47.440253+0000","last_fullsized":"2024-09-12T14:18:47.440253+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"1931'99","last_scrub_stamp":"2024-09-12T14:18:47.440184+0000","last_deep_scrub":"1931'99","last_deep_scrub_stamp":"2024-09-12T14:18:47.440184+0000","last_clean_scrub_stamp":"2024-09-12T14:18:47.440184+0000","objects_scrubbed":2,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:40:15.748987+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":99,"ondisk_log_size":99,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359739375,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1078468,"kb_used_data":4660,"kb_used_omap":1,"kb_used_meta":1073790,"kb_avail":93293372,"statfs":{"total":96636764160,"available":95532412928,"internally_reserved":0,"allocated":4771840,"data_stored":2499302,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":1099561413},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":8,"seq":34359739375,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1295628,"kb_used_data":5244,"kb_used_omap":1,"kb_used_meta":1290366,"kb_avail":93076212,"statfs":{"total":96636764160,"available":95310041088,"internally_reserved":0,"allocated":5369856,"data_stored":3089689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":1321335237},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359739376,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1331660,"kb_used_data":5244,"kb_used_omap":1,"kb_used_meta":1326398,"kb_avail":93040180,"statfs":{"total":96636764160,"available":95273144320,"internally_reserved":0,"allocated":5369856,"data_stored":3089689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":1358232005},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T14:19:08.667 DEBUG:teuthology.orchestra.run.smithi060:> sudo ceph --cluster cluster1 config set global mon_health_to_clog false 2024-09-12T14:19:09.042 INFO:teuthology.misc:Shutting down mds daemons... 2024-09-12T14:19:09.042 INFO:teuthology.misc:Shutting down osd daemons... 2024-09-12T14:19:09.042 DEBUG:tasks.ceph.cluster1.osd.0:waiting for process to exit 2024-09-12T14:19:09.042 INFO:teuthology.orchestra.run:waiting for 300 2024-09-12T14:19:12.588 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon cluster1.osd.0 is failed for ~0s 2024-09-12T14:19:15.144 INFO:tasks.ceph.cluster1.osd.0:Stopped 2024-09-12T14:19:15.145 DEBUG:tasks.ceph.cluster1.osd.1:waiting for process to exit 2024-09-12T14:19:15.145 INFO:teuthology.orchestra.run:waiting for 300 2024-09-12T14:19:17.891 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon cluster1.osd.1 is failed for ~0s 2024-09-12T14:19:17.892 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon cluster1.osd.0 has been restored 2024-09-12T14:19:21.247 INFO:tasks.ceph.cluster1.osd.1:Stopped 2024-09-12T14:19:21.247 DEBUG:tasks.ceph.cluster1.osd.2:waiting for process to exit 2024-09-12T14:19:21.247 INFO:teuthology.orchestra.run:waiting for 300 2024-09-12T14:19:23.094 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon cluster1.osd.2 is failed for ~0s 2024-09-12T14:19:23.094 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon cluster1.osd.1 has been restored 2024-09-12T14:19:27.350 INFO:tasks.ceph.cluster1.osd.2:Stopped 2024-09-12T14:19:27.350 INFO:teuthology.misc:Shutting down mgr daemons... 2024-09-12T14:19:27.351 DEBUG:tasks.ceph.cluster1.mgr.x:waiting for process to exit 2024-09-12T14:19:27.351 INFO:teuthology.orchestra.run:waiting for 300 2024-09-12T14:19:27.429 INFO:tasks.ceph.cluster1.mgr.x:Stopped 2024-09-12T14:19:27.429 INFO:teuthology.misc:Shutting down mon daemons... 2024-09-12T14:19:27.430 DEBUG:tasks.ceph.cluster1.mon.a:waiting for process to exit 2024-09-12T14:19:27.430 INFO:teuthology.orchestra.run:waiting for 300 2024-09-12T14:19:27.464 INFO:tasks.ceph.cluster1.mon.a:Stopped 2024-09-12T14:19:27.464 INFO:tasks.ceph:Checking cluster log for badness... 2024-09-12T14:19:27.464 DEBUG:teuthology.orchestra.run.smithi060:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/cluster1.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | head -n 1 2024-09-12T14:19:27.637 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/cluster1-0 on ubuntu@smithi060.front.sepia.ceph.com 2024-09-12T14:19:27.637 DEBUG:teuthology.orchestra.run.smithi060:> sync && sudo umount -f /var/lib/ceph/osd/cluster1-0 2024-09-12T14:19:27.996 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/cluster1-1 on ubuntu@smithi060.front.sepia.ceph.com 2024-09-12T14:19:27.997 DEBUG:teuthology.orchestra.run.smithi060:> sync && sudo umount -f /var/lib/ceph/osd/cluster1-1 2024-09-12T14:19:28.095 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon cluster1.osd.2 has been restored 2024-09-12T14:19:28.231 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/cluster1-2 on ubuntu@smithi060.front.sepia.ceph.com 2024-09-12T14:19:28.232 DEBUG:teuthology.orchestra.run.smithi060:> sync && sudo umount -f /var/lib/ceph/osd/cluster1-2 2024-09-12T14:19:28.464 INFO:tasks.ceph:Archiving mon data... 2024-09-12T14:19:28.464 DEBUG:teuthology.misc:Transferring archived files from smithi060:/var/lib/ceph/mon/cluster1-a to /home/teuthworker/archive/teuthology-2024-09-11_21:48:03-krbd-squid-testing-default-smithi/7901765/data/cluster1.mon.a.tgz 2024-09-12T14:19:28.464 DEBUG:teuthology.orchestra.run.smithi060:> mktemp 2024-09-12T14:19:28.504 INFO:teuthology.orchestra.run.smithi060.stdout:/tmp/tmp.ANsd7BR3Wi 2024-09-12T14:19:28.505 DEBUG:teuthology.orchestra.run.smithi060:> sudo tar cz -f - -C /var/lib/ceph/mon/cluster1-a -- . > /tmp/tmp.ANsd7BR3Wi 2024-09-12T14:19:28.720 DEBUG:teuthology.orchestra.run.smithi060:> sudo chmod 0666 /tmp/tmp.ANsd7BR3Wi 2024-09-12T14:19:28.829 DEBUG:teuthology.orchestra.remote:smithi060:/tmp/tmp.ANsd7BR3Wi is 530KB 2024-09-12T14:19:28.921 DEBUG:teuthology.orchestra.run.smithi060:> rm -fr /tmp/tmp.ANsd7BR3Wi 2024-09-12T14:19:28.956 INFO:tasks.ceph:Cleaning ceph cluster... 2024-09-12T14:19:28.956 DEBUG:teuthology.orchestra.run.smithi060:> sudo rm -rf -- /etc/ceph/cluster1.conf /etc/ceph/cluster1.keyring /home/ubuntu/cephtest/cluster1.data /home/ubuntu/cephtest/cluster1.monmap /home/ubuntu/cephtest/../*.pid 2024-09-12T14:19:29.001 DEBUG:teuthology.orchestra.run.smithi130:> sudo rm -rf -- /etc/ceph/cluster1.conf /etc/ceph/cluster1.keyring /home/ubuntu/cephtest/cluster1.data /home/ubuntu/cephtest/cluster1.monmap /home/ubuntu/cephtest/../*.pid 2024-09-12T14:19:29.098 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-12T14:19:29.154 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-12T14:19:29.155 INFO:tasks.ceph:Archiving crash dumps... 2024-09-12T14:19:29.156 DEBUG:teuthology.misc:Transferring archived files from smithi060:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-09-11_21:48:03-krbd-squid-testing-default-smithi/7901765/remote/smithi060/crash 2024-09-12T14:19:29.157 DEBUG:teuthology.orchestra.run.smithi060:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-12T14:19:29.212 DEBUG:teuthology.misc:Transferring archived files from smithi130:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-09-11_21:48:03-krbd-squid-testing-default-smithi/7901765/remote/smithi130/crash 2024-09-12T14:19:29.212 DEBUG:teuthology.orchestra.run.smithi130:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-12T14:19:29.268 INFO:tasks.ceph:Compressing logs... 2024-09-12T14:19:29.268 DEBUG:teuthology.orchestra.run.smithi060:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-12T14:19:29.273 DEBUG:teuthology.orchestra.run.smithi130:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-12T14:19:29.329 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.32984.log 2024-09-12T14:19:29.330 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.34516.log 2024-09-12T14:19:29.331 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/cluster1-client.admin.32984.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.32984.log.gz 2024-09-12T14:19:29.331 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-mon.a.log 2024-09-12T14:19:29.332 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/cluster1-client.admin.34516.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.34516.log.gz 2024-09-12T14:19:29.332 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.32438.log 2024-09-12T14:19:29.332 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/cluster1-mon.a.log: gzip -5 --verbose -- /var/log/ceph/cluster1-osd.1.log 2024-09-12T14:19:29.333 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/cluster1-client.admin.32438.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.32438.log.gz 2024-09-12T14:19:29.333 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.32730.log 2024-09-12T14:19:29.334 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/cluster1-osd.1.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.29039.log 2024-09-12T14:19:29.335 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/cluster1-client.admin.32730.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.32730.log.gz 2024-09-12T14:19:29.335 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.32816.log 2024-09-12T14:19:29.335 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/cluster1-client.admin.29039.log: 0.0%gzip -- replaced with /var/log/ceph/cluster1-client.admin.29039.log.gz 2024-09-12T14:19:29.336 INFO:teuthology.orchestra.run.smithi060.stderr: -5 --verbose -- /var/log/ceph/cluster1-client.admin.32522.log 2024-09-12T14:19:29.336 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/cluster1-client.admin.32816.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.32816.log.gz 2024-09-12T14:19:29.336 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.34603.log 2024-09-12T14:19:29.337 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/cluster1-client.admin.32522.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.32522.log.gz 2024-09-12T14:19:29.337 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.29138.log 2024-09-12T14:19:29.338 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/cluster1-client.admin.34603.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.34603.log.gz 2024-09-12T14:19:29.338 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.34687.log 2024-09-12T14:19:29.339 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/cluster1-client.admin.29138.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.29138.log.gz 2024-09-12T14:19:29.339 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.32603.log 2024-09-12T14:19:29.339 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/cluster1-client.admin.34687.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.34687.log.gz 2024-09-12T14:19:29.340 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.32604.log 2024-09-12T14:19:29.340 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/cluster1-client.admin.32603.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.32603.log.gz 2024-09-12T14:19:29.340 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.33026.log 2024-09-12T14:19:29.341 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/cluster1-client.admin.32604.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.32604.log.gz 2024-09-12T14:19:29.341 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.28800.log 2024-09-12T14:19:29.342 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/cluster1-client.admin.33026.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.33026.log.gz 2024-09-12T14:19:29.342 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.29206.log 2024-09-12T14:19:29.343 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/cluster1-client.admin.28800.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.28800.log.gz 2024-09-12T14:19:29.343 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.32855.log 2024-09-12T14:19:29.343 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/cluster1-client.admin.29206.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.29206.log.gz 2024-09-12T14:19:29.344 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.32602.log 2024-09-12T14:19:29.344 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/cluster1-client.admin.32855.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.32855.log.gz 2024-09-12T14:19:29.344 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.32729.log 2024-09-12T14:19:29.345 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/cluster1-client.admin.32602.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.32602.log.gz 2024-09-12T14:19:29.345 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.31844.log 2024-09-12T14:19:29.346 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/cluster1-client.admin.32729.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.32729.log.gz 2024-09-12T14:19:29.346 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.34771.log 2024-09-12T14:19:29.347 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/cluster1-client.admin.31844.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.31844.log.gz 2024-09-12T14:19:29.347 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.34466.log 2024-09-12T14:19:29.347 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/cluster1-client.admin.34771.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.34771.log.gz 2024-09-12T14:19:29.348 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.34937.log 2024-09-12T14:19:29.348 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/cluster1-client.admin.34466.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.34466.log.gz 2024-09-12T14:19:29.348 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-osd.2.log 2024-09-12T14:19:29.349 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/cluster1-client.admin.34937.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.34937.log.gz 2024-09-12T14:19:29.349 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.32856.log 2024-09-12T14:19:29.350 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/cluster1-osd.2.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.34560.log 2024-09-12T14:19:29.351 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/cluster1-client.admin.32856.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.32856.log.gz 2024-09-12T14:19:29.351 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.32344.log 2024-09-12T14:19:29.351 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/cluster1-client.admin.34560.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.34560.log.gz 2024-09-12T14:19:29.351 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-mgr.x.log 2024-09-12T14:19:29.352 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/cluster1-client.admin.32344.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.32344.log.gz 2024-09-12T14:19:29.352 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.32690.log 2024-09-12T14:19:29.353 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/cluster1-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.34813.log 2024-09-12T14:19:29.354 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/cluster1-client.admin.32690.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.32690.log.gz 2024-09-12T14:19:29.354 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1.audit.log 2024-09-12T14:19:29.354 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/cluster1-client.admin.34813.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.34813.log.gz 2024-09-12T14:19:29.354 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.34897.log 2024-09-12T14:19:29.355 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/cluster1.audit.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.34729.log 2024-09-12T14:19:29.356 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/cluster1-client.admin.34897.log: gzip -5 --verbose -- /var/log/ceph/cluster1-osd.0.log 2024-09-12T14:19:29.356 INFO:teuthology.orchestra.run.smithi060.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.34897.log.gz 2024-09-12T14:19:29.357 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/cluster1-client.admin.34729.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.34729.log.gz 2024-09-12T14:19:29.357 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.32480.log 2024-09-12T14:19:29.358 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/cluster1-osd.0.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.32942.log 2024-09-12T14:19:29.359 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/cluster1-client.admin.32480.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.32480.log.gz 2024-09-12T14:19:29.359 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.32267.log 2024-09-12T14:19:29.360 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/cluster1-client.admin.32942.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.32942.log.gz 2024-09-12T14:19:29.360 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.34424.log 2024-09-12T14:19:29.361 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/cluster1-client.admin.32267.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.32267.log.gz 2024-09-12T14:19:29.361 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.32386.log 2024-09-12T14:19:29.362 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/cluster1-client.admin.34424.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.28999.log 2024-09-12T14:19:29.362 INFO:teuthology.orchestra.run.smithi060.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.34424.log.gz 2024-09-12T14:19:29.362 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/cluster1-client.admin.32386.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.32386.log.gz 2024-09-12T14:19:29.362 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.34645.log 2024-09-12T14:19:29.363 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.92558.log 2024-09-12T14:19:29.364 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/cluster1-client.admin.28999.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.28999.log.gz 2024-09-12T14:19:29.364 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.34855.log 2024-09-12T14:19:29.365 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/cluster1-client.admin.34645.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.34645.log.gz 2024-09-12T14:19:29.365 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.24463.log 2024-09-12T14:19:29.365 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/cluster1-client.admin.34855.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.34855.log.gz 2024-09-12T14:19:29.366 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1.log 2024-09-12T14:19:29.366 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.137306.log 2024-09-12T14:19:29.366 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.92558.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.92558.log.gz 2024-09-12T14:19:29.367 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.61850.log 2024-09-12T14:19:29.367 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.137306.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.137306.log.gz 2024-09-12T14:19:29.367 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.77236.log 2024-09-12T14:19:29.367 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.61850.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.61850.log.gz 2024-09-12T14:19:29.367 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.129181.log 2024-09-12T14:19:29.368 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/ceph.tmp-client.admin.24463.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.28959.log 2024-09-12T14:19:29.368 INFO:teuthology.orchestra.run.smithi060.stderr: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.24463.log.gz 2024-09-12T14:19:29.369 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/cluster1.log: /var/log/ceph/cluster1-client.admin.28959.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.28959.log.gz 2024-09-12T14:19:29.370 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.77236.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.77236.log.gz 2024-09-12T14:19:29.370 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.134932.log 2024-09-12T14:19:29.370 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.129181.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.129181.log.gz 2024-09-12T14:19:29.370 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.46676.log 2024-09-12T14:19:29.370 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.134932.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.134932.log.gz 2024-09-12T14:19:29.370 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.40389.log 2024-09-12T14:19:29.370 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.46676.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.46676.log.gz 2024-09-12T14:19:29.370 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.74530.log 2024-09-12T14:19:29.371 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.40389.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.44655.log 2024-09-12T14:19:29.371 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.74530.log: gzip -5 --verbose 0.0% -- -- replaced with /var/log/ceph/cluster1-client.mirror.74530.log.gz 2024-09-12T14:19:29.371 INFO:teuthology.orchestra.run.smithi130.stderr: /var/log/ceph/cluster1-client.mirror.122309.log 2024-09-12T14:19:29.371 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.40389.log.gz 2024-09-12T14:19:29.372 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.44655.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.140776.log 2024-09-12T14:19:29.373 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.122309.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.122309.log.gz 2024-09-12T14:19:29.373 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.44655.log.gz 2024-09-12T14:19:29.373 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.134195.log 2024-09-12T14:19:29.374 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.140776.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.140776.log.gz 2024-09-12T14:19:29.374 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.56775.log 2024-09-12T14:19:29.374 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.134195.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.134195.log.gz 2024-09-12T14:19:29.374 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.37823.log 2024-09-12T14:19:29.375 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.56775.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.56775.log.gz 2024-09-12T14:19:29.375 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.129034.log 2024-09-12T14:19:29.376 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.37823.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.37823.log.gz 2024-09-12T14:19:29.376 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.142665.log 2024-09-12T14:19:29.376 INFO:teuthology.orchestra.run.smithi060.stderr: 87.6% -- replaced with /var/log/ceph/cluster1.log.gz 2024-09-12T14:19:29.377 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.129034.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.129034.log.gz 2024-09-12T14:19:29.377 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.45667.log 2024-09-12T14:19:29.378 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.142665.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.142665.log.gz 2024-09-12T14:19:29.378 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.89264.log 2024-09-12T14:19:29.379 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.45667.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.45667.log.gz 2024-09-12T14:19:29.379 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.39565.log 2024-09-12T14:19:29.379 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.89264.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.89264.log.gz 2024-09-12T14:19:29.379 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.130491.log 2024-09-12T14:19:29.380 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.39565.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.39565.log.gz 2024-09-12T14:19:29.380 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.100518.log 2024-09-12T14:19:29.381 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.130491.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.130491.log.gz 2024-09-12T14:19:29.381 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.128483.log 2024-09-12T14:19:29.382 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.100518.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.100518.log.gz 2024-09-12T14:19:29.382 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.47798.log 2024-09-12T14:19:29.382 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.128483.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.128483.log.gz 2024-09-12T14:19:29.382 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.137032.log 2024-09-12T14:19:29.383 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.47798.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.47798.log.gz 2024-09-12T14:19:29.383 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.45643.log 2024-09-12T14:19:29.384 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.137032.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.137032.log.gz 2024-09-12T14:19:29.384 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.75773.log 2024-09-12T14:19:29.385 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.45643.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.45643.log.gz 2024-09-12T14:19:29.385 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.154341.log 2024-09-12T14:19:29.385 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.75773.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.135871.log 2024-09-12T14:19:29.386 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.154341.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.75773.log.gz 2024-09-12T14:19:29.386 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.154341.log.gz 2024-09-12T14:19:29.386 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.54899.log 2024-09-12T14:19:29.387 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.135871.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.135871.log.gz 2024-09-12T14:19:29.387 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.63569.log 2024-09-12T14:19:29.388 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.54899.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.54899.log.gz 2024-09-12T14:19:29.388 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.60301.log 2024-09-12T14:19:29.389 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.63569.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.63569.log.gz 2024-09-12T14:19:29.389 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.39276.log 2024-09-12T14:19:29.389 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.60301.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.87374.log 2024-09-12T14:19:29.390 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.39276.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.39276.log.gz 2024-09-12T14:19:29.390 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.60301.log.gz 2024-09-12T14:19:29.390 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.68989.log 2024-09-12T14:19:29.391 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.87374.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.87374.log.gz 2024-09-12T14:19:29.391 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.52603.log 2024-09-12T14:19:29.392 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.68989.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.68989.log.gz 2024-09-12T14:19:29.392 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.56697.log 2024-09-12T14:19:29.393 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.52603.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.52603.log.gz 2024-09-12T14:19:29.393 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.69822.log 2024-09-12T14:19:29.393 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.56697.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.56697.log.gz 2024-09-12T14:19:29.394 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.74990.log 2024-09-12T14:19:29.394 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.69822.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.69822.log.gz 2024-09-12T14:19:29.394 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.139590.log 2024-09-12T14:19:29.395 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.74990.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.74990.log.gz 2024-09-12T14:19:29.395 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.60101.log 2024-09-12T14:19:29.397 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.139590.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.139590.log.gz 2024-09-12T14:19:29.397 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.61402.log 2024-09-12T14:19:29.398 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.60101.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.60101.log.gz 2024-09-12T14:19:29.398 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.142333.log 2024-09-12T14:19:29.398 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.61402.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.61402.log.gz 2024-09-12T14:19:29.398 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.121636.log 2024-09-12T14:19:29.399 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.142333.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.142333.log.gz 2024-09-12T14:19:29.399 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.146169.log 2024-09-12T14:19:29.399 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.121636.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.121636.log.gz 2024-09-12T14:19:29.399 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.39591.log 2024-09-12T14:19:29.400 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.146169.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.146169.log.gz 2024-09-12T14:19:29.400 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.80863.log 2024-09-12T14:19:29.401 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.39591.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.39591.log.gz 2024-09-12T14:19:29.401 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.150564.log 2024-09-12T14:19:29.401 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.80863.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.80863.log.gz 2024-09-12T14:19:29.401 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.132431.log 2024-09-12T14:19:29.402 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.150564.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.150564.log.gz 2024-09-12T14:19:29.402 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.134301.log 2024-09-12T14:19:29.403 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.132431.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.132431.log.gz 2024-09-12T14:19:29.403 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.111947.log 2024-09-12T14:19:29.404 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.134301.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.134301.log.gz 2024-09-12T14:19:29.404 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.55966.log 2024-09-12T14:19:29.405 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.111947.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.111947.log.gz 2024-09-12T14:19:29.405 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.90989.log 2024-09-12T14:19:29.405 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.55966.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.55966.log.gz 2024-09-12T14:19:29.406 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.61761.log 2024-09-12T14:19:29.406 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.90989.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.90989.log.gz 2024-09-12T14:19:29.406 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.114107.log 2024-09-12T14:19:29.407 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.61761.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.61761.log.gz 2024-09-12T14:19:29.407 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.132222.log 2024-09-12T14:19:29.408 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.114107.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.114107.log.gz 2024-09-12T14:19:29.408 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.107107.log 2024-09-12T14:19:29.409 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.132222.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.132222.log.gz 2024-09-12T14:19:29.409 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.43970.log 2024-09-12T14:19:29.409 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.107107.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.107107.log.gz 2024-09-12T14:19:29.410 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.112025.log 2024-09-12T14:19:29.410 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.43970.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.43970.log.gz 2024-09-12T14:19:29.410 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.88257.log 2024-09-12T14:19:29.411 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.112025.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.112025.log.gz 2024-09-12T14:19:29.411 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.118659.log 2024-09-12T14:19:29.412 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.88257.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.88257.log.gz 2024-09-12T14:19:29.412 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.83065.log 2024-09-12T14:19:29.413 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.118659.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.118659.log.gz 2024-09-12T14:19:29.413 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.151084.log 2024-09-12T14:19:29.414 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.83065.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.83065.log.gz 2024-09-12T14:19:29.414 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.147388.log 2024-09-12T14:19:29.414 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.151084.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.151084.log.gz 2024-09-12T14:19:29.414 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.41691.log 2024-09-12T14:19:29.415 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.147388.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.147388.log.gz 2024-09-12T14:19:29.415 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.128674.log 2024-09-12T14:19:29.416 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.41691.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.41691.log.gz 2024-09-12T14:19:29.416 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.112954.log 2024-09-12T14:19:29.417 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.128674.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.128674.log.gz 2024-09-12T14:19:29.417 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.97579.log 2024-09-12T14:19:29.418 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.112954.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.112954.log.gz 2024-09-12T14:19:29.418 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.64971.log 2024-09-12T14:19:29.418 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.97579.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.97579.log.gz 2024-09-12T14:19:29.419 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.72108.log 2024-09-12T14:19:29.419 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.64971.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.64971.log.gz 2024-09-12T14:19:29.419 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.101087.log 2024-09-12T14:19:29.420 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.72108.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.72108.log.gz 2024-09-12T14:19:29.420 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.103596.log 2024-09-12T14:19:29.421 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.101087.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.101087.log.gz 2024-09-12T14:19:29.421 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.49997.log 2024-09-12T14:19:29.422 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.103596.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.103596.log.gz 2024-09-12T14:19:29.422 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.71950.log 2024-09-12T14:19:29.422 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.49997.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.49997.log.gz 2024-09-12T14:19:29.422 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.98922.log 2024-09-12T14:19:29.423 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.71950.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.71950.log.gz 2024-09-12T14:19:29.423 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.134025.log 2024-09-12T14:19:29.424 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.98922.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.98922.log.gz 2024-09-12T14:19:29.424 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.136359.log 2024-09-12T14:19:29.425 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.134025.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.134025.log.gz 2024-09-12T14:19:29.425 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.102636.log 2024-09-12T14:19:29.426 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.136359.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.136359.log.gz 2024-09-12T14:19:29.426 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.76023.log 2024-09-12T14:19:29.426 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.102636.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.102636.log.gz 2024-09-12T14:19:29.427 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.123459.log 2024-09-12T14:19:29.427 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.76023.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.76023.log.gz 2024-09-12T14:19:29.427 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.44312.log 2024-09-12T14:19:29.428 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.123459.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.123459.log.gz 2024-09-12T14:19:29.428 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.55060.log 2024-09-12T14:19:29.429 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.44312.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.44312.log.gz 2024-09-12T14:19:29.429 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.117521.log 2024-09-12T14:19:29.430 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.55060.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.55060.log.gz 2024-09-12T14:19:29.430 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.76799.log 2024-09-12T14:19:29.430 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.117521.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.117521.log.gz 2024-09-12T14:19:29.431 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.112929.log 2024-09-12T14:19:29.431 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.76799.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.137202.log 2024-09-12T14:19:29.432 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.112929.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.112929.log.gz 2024-09-12T14:19:29.432 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.53819.log 2024-09-12T14:19:29.433 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.137202.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.137202.log.gz 2024-09-12T14:19:29.433 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.120713.log 2024-09-12T14:19:29.433 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.53819.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.53819.log.gz 2024-09-12T14:19:29.434 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.145502.log 2024-09-12T14:19:29.434 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.120713.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.120713.log.gz 2024-09-12T14:19:29.434 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.72539.log 2024-09-12T14:19:29.435 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.145502.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.145502.log.gz 2024-09-12T14:19:29.435 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.99449.log 2024-09-12T14:19:29.435 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.76799.log.gz 2024-09-12T14:19:29.436 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.72539.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.72539.log.gz 2024-09-12T14:19:29.436 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.117582.log 2024-09-12T14:19:29.437 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.99449.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.99449.log.gz 2024-09-12T14:19:29.437 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.124672.log 2024-09-12T14:19:29.438 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.117582.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.117582.log.gz 2024-09-12T14:19:29.438 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.147918.log 2024-09-12T14:19:29.438 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.124672.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.124672.log.gz 2024-09-12T14:19:29.438 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.131211.log 2024-09-12T14:19:29.439 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.147918.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.147918.log.gz 2024-09-12T14:19:29.439 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.41439.log 2024-09-12T14:19:29.440 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.131211.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.131211.log.gz 2024-09-12T14:19:29.440 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.106600.log 2024-09-12T14:19:29.441 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.41439.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.41439.log.gz 2024-09-12T14:19:29.441 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.60983.log 2024-09-12T14:19:29.442 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.106600.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.106600.log.gz 2024-09-12T14:19:29.442 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.67275.log 2024-09-12T14:19:29.442 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.60983.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.60983.log.gz 2024-09-12T14:19:29.442 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.143856.log 2024-09-12T14:19:29.443 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.67275.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.67275.log.gz 2024-09-12T14:19:29.443 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.86430.log 2024-09-12T14:19:29.444 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.143856.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.143856.log.gz 2024-09-12T14:19:29.444 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.67473.log 2024-09-12T14:19:29.445 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.86430.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.86430.log.gz 2024-09-12T14:19:29.445 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.119441.log 2024-09-12T14:19:29.445 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.67473.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.67473.log.gz 2024-09-12T14:19:29.446 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.101295.log 2024-09-12T14:19:29.446 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.119441.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.119441.log.gz 2024-09-12T14:19:29.446 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.117039.log 2024-09-12T14:19:29.447 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.101295.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.101295.log.gz 2024-09-12T14:19:29.447 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.150474.log 2024-09-12T14:19:29.448 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.117039.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.117039.log.gz 2024-09-12T14:19:29.448 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.133106.log 2024-09-12T14:19:29.449 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.150474.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.150474.log.gz 2024-09-12T14:19:29.449 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.70485.log 2024-09-12T14:19:29.449 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.133106.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.133106.log.gz 2024-09-12T14:19:29.450 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.37344.log 2024-09-12T14:19:29.451 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.70485.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.153712.log 2024-09-12T14:19:29.451 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.70485.log.gz 2024-09-12T14:19:29.451 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.37344.log: gzip 0.0% -5 -- replaced with /var/log/ceph/cluster1-client.mirror.37344.log.gz --verbose 2024-09-12T14:19:29.451 INFO:teuthology.orchestra.run.smithi130.stderr: -- /var/log/ceph/cluster1-client.admin.46630.log 2024-09-12T14:19:29.452 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.153712.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.136523.log 0.0% 2024-09-12T14:19:29.452 INFO:teuthology.orchestra.run.smithi130.stderr: -- replaced with /var/log/ceph/cluster1-client.mirror.153712.log.gz 2024-09-12T14:19:29.453 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.46630.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.46630.log.gz 2024-09-12T14:19:29.453 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.118720.log 2024-09-12T14:19:29.453 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.136523.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.82268.log 2024-09-12T14:19:29.453 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.136523.log.gz 2024-09-12T14:19:29.454 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.118720.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.118720.log.gz 2024-09-12T14:19:29.454 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.151391.log 2024-09-12T14:19:29.455 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.82268.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.82268.log.gz 2024-09-12T14:19:29.455 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.146738.log 2024-09-12T14:19:29.455 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.151391.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.151391.log.gz 2024-09-12T14:19:29.456 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.73920.log 2024-09-12T14:19:29.456 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.146738.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.146738.log.gzgzip 2024-09-12T14:19:29.456 INFO:teuthology.orchestra.run.smithi130.stderr: -5 --verbose -- /var/log/ceph/cluster1-client.mirror.64091.log 2024-09-12T14:19:29.457 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.73920.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.73920.log.gz 2024-09-12T14:19:29.457 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.38434.log 2024-09-12T14:19:29.458 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.64091.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.64091.log.gz 2024-09-12T14:19:29.458 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.73756.log 2024-09-12T14:19:29.459 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.38434.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.38434.log.gz 2024-09-12T14:19:29.459 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.139385.log 2024-09-12T14:19:29.460 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.73756.log: 0.0%gzip -- replaced with /var/log/ceph/cluster1-client.mirror.73756.log.gz -5 2024-09-12T14:19:29.460 INFO:teuthology.orchestra.run.smithi130.stderr: --verbose -- /var/log/ceph/cluster1-client.mirror.84540.log 2024-09-12T14:19:29.460 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.139385.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.139385.log.gzgzip 2024-09-12T14:19:29.460 INFO:teuthology.orchestra.run.smithi130.stderr: -5 --verbose -- /var/log/ceph/cluster1-client.admin.138697.log 2024-09-12T14:19:29.461 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.84540.log: gzip -5 --verbose 0.0% -- -- replaced with /var/log/ceph/cluster1-client.mirror.84540.log.gz /var/log/ceph/cluster1-client.admin.68547.log 2024-09-12T14:19:29.461 INFO:teuthology.orchestra.run.smithi130.stderr: 2024-09-12T14:19:29.463 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.138697.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.98738.log 2024-09-12T14:19:29.463 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.138697.log.gz 2024-09-12T14:19:29.463 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.68547.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.68547.log.gz 2024-09-12T14:19:29.463 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.103011.log 2024-09-12T14:19:29.466 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.98738.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.98738.log.gz 2024-09-12T14:19:29.466 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.43736.log 2024-09-12T14:19:29.466 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.103011.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.103011.log.gz 2024-09-12T14:19:29.466 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.113059.log 2024-09-12T14:19:29.466 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.43736.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.43736.log.gz 2024-09-12T14:19:29.467 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.78433.log 2024-09-12T14:19:29.467 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.113059.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.113059.log.gz 2024-09-12T14:19:29.467 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.69225.log 2024-09-12T14:19:29.471 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.78433.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.78433.log.gz 2024-09-12T14:19:29.471 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.134368.log 2024-09-12T14:19:29.471 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.69225.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.69225.log.gz 2024-09-12T14:19:29.471 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.35092.log 2024-09-12T14:19:29.471 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.134368.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.134368.log.gz 2024-09-12T14:19:29.472 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.140893.log 2024-09-12T14:19:29.472 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.35092.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.35092.log.gz 2024-09-12T14:19:29.472 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.89936.log 2024-09-12T14:19:29.472 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.140893.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.140893.log.gz 2024-09-12T14:19:29.472 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.45559.log 2024-09-12T14:19:29.472 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.89936.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.89936.log.gz 2024-09-12T14:19:29.472 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.97873.log 2024-09-12T14:19:29.475 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.45559.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.45559.log.gz 2024-09-12T14:19:29.475 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.120461.log 2024-09-12T14:19:29.475 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.97873.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.97873.log.gz 2024-09-12T14:19:29.476 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.112849.log 2024-09-12T14:19:29.476 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.120461.log: 0.0%gzip -- replaced with /var/log/ceph/cluster1-client.mirror.120461.log.gz -5 2024-09-12T14:19:29.476 INFO:teuthology.orchestra.run.smithi130.stderr: --verbose -- /var/log/ceph/cluster1-client.admin.79304.log 2024-09-12T14:19:29.476 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.112849.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.112849.log.gz 2024-09-12T14:19:29.476 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.54831.log 2024-09-12T14:19:29.476 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.79304.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.79806.log 2024-09-12T14:19:29.476 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.79304.log.gz 2024-09-12T14:19:29.476 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.54831.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.54831.log.gz 2024-09-12T14:19:29.476 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.99507.log 2024-09-12T14:19:29.478 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.79806.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.151246.log 2024-09-12T14:19:29.478 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.79806.log.gz 2024-09-12T14:19:29.478 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.99507.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.99507.log.gz 2024-09-12T14:19:29.478 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.48591.log 2024-09-12T14:19:29.479 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.151246.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.151246.log.gz 2024-09-12T14:19:29.479 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.88318.log 2024-09-12T14:19:29.479 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.48591.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.48591.log.gz 2024-09-12T14:19:29.479 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.134577.log 2024-09-12T14:19:29.480 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.88318.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.88318.log.gz 2024-09-12T14:19:29.480 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.89331.log 2024-09-12T14:19:29.480 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.134577.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.134577.log.gz 2024-09-12T14:19:29.480 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.147390.log 2024-09-12T14:19:29.481 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.89331.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.89331.log.gz 2024-09-12T14:19:29.482 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.49263.log 2024-09-12T14:19:29.482 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.147390.log: gzip 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.147390.log.gz -5 2024-09-12T14:19:29.482 INFO:teuthology.orchestra.run.smithi130.stderr: --verbose -- /var/log/ceph/cluster1-client.mirror.84093.log 2024-09-12T14:19:29.483 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.49263.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.49263.log.gz 2024-09-12T14:19:29.483 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.130047.log 2024-09-12T14:19:29.483 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.84093.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.84093.log.gz 2024-09-12T14:19:29.483 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.93383.log 2024-09-12T14:19:29.484 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.130047.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.130047.log.gz 2024-09-12T14:19:29.484 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.82871.log 2024-09-12T14:19:29.485 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.93383.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.93383.log.gz 2024-09-12T14:19:29.485 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.122142.log 2024-09-12T14:19:29.486 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.82871.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.82871.log.gz 2024-09-12T14:19:29.486 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.90420.log 2024-09-12T14:19:29.486 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.122142.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.122142.log.gz 2024-09-12T14:19:29.487 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.137622.log 2024-09-12T14:19:29.487 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.90420.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.90420.log.gz 2024-09-12T14:19:29.487 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.144870.log 2024-09-12T14:19:29.488 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.137622.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.137622.log.gz 2024-09-12T14:19:29.488 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.106120.log 2024-09-12T14:19:29.489 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.144870.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.144870.log.gz 2024-09-12T14:19:29.489 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.55856.log 2024-09-12T14:19:29.490 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.106120.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.106120.log.gz 2024-09-12T14:19:29.490 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.116043.log 2024-09-12T14:19:29.491 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.55856.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.55856.log.gz 2024-09-12T14:19:29.491 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.95148.log 2024-09-12T14:19:29.492 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.116043.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.116043.log.gz 2024-09-12T14:19:29.492 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.55350.log 2024-09-12T14:19:29.493 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.95148.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.95148.log.gz 2024-09-12T14:19:29.493 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.95719.log 2024-09-12T14:19:29.493 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.55350.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.55350.log.gz 2024-09-12T14:19:29.493 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.96547.log 2024-09-12T14:19:29.495 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.95719.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.95719.log.gz 2024-09-12T14:19:29.495 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.54029.log 2024-09-12T14:19:29.495 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.96547.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.96547.log.gz 2024-09-12T14:19:29.495 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.59120.log 2024-09-12T14:19:29.498 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.54029.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.54029.log.gz 2024-09-12T14:19:29.498 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.44998.log 2024-09-12T14:19:29.498 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.59120.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.59120.log.gz 2024-09-12T14:19:29.498 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.117769.log 2024-09-12T14:19:29.498 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.44998.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.44998.log.gz 2024-09-12T14:19:29.498 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.131646.log 2024-09-12T14:19:29.499 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.117769.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.117769.log.gz 2024-09-12T14:19:29.499 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.87417.log 2024-09-12T14:19:29.502 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.131646.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.131646.log.gz 2024-09-12T14:19:29.503 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.57055.log 2024-09-12T14:19:29.503 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.87417.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.87417.log.gz 2024-09-12T14:19:29.503 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.124878.log 2024-09-12T14:19:29.503 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.57055.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.57055.log.gz 2024-09-12T14:19:29.503 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.103230.log 2024-09-12T14:19:29.504 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.124878.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.124878.log.gz 2024-09-12T14:19:29.504 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.115477.log 2024-09-12T14:19:29.504 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.103230.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.103230.log.gz 2024-09-12T14:19:29.504 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.50667.log 2024-09-12T14:19:29.504 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.115477.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.115477.log.gz 2024-09-12T14:19:29.504 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.146447.log 2024-09-12T14:19:29.508 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.50667.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.50667.log.gz 2024-09-12T14:19:29.508 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.81154.log 2024-09-12T14:19:29.509 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.146447.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.146447.log.gz 2024-09-12T14:19:29.509 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.42555.log 2024-09-12T14:19:29.509 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.81154.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.81154.log.gz 2024-09-12T14:19:29.509 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.131407.log 2024-09-12T14:19:29.509 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.42555.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.42555.log.gz 2024-09-12T14:19:29.509 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.59604.log 2024-09-12T14:19:29.509 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.131407.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.131407.log.gz 2024-09-12T14:19:29.510 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.43852.log 2024-09-12T14:19:29.510 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.59604.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.59604.log.gz 2024-09-12T14:19:29.510 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.88131.log 2024-09-12T14:19:29.510 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.43852.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.43852.log.gz 2024-09-12T14:19:29.510 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.69052.log 2024-09-12T14:19:29.514 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.88131.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.88131.log.gz 2024-09-12T14:19:29.514 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.144168.log 2024-09-12T14:19:29.514 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.69052.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.69052.log.gz 2024-09-12T14:19:29.514 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.61193.log 2024-09-12T14:19:29.514 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.144168.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.144168.log.gz 2024-09-12T14:19:29.514 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.61529.log 2024-09-12T14:19:29.514 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.61193.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.61193.log.gz 2024-09-12T14:19:29.514 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.36744.log 2024-09-12T14:19:29.514 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.61529.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.61529.log.gz 2024-09-12T14:19:29.515 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.96426.log 2024-09-12T14:19:29.515 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.36744.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.36744.log.gz 2024-09-12T14:19:29.515 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.91165.log 2024-09-12T14:19:29.515 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.96426.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.96426.log.gz 2024-09-12T14:19:29.515 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.99215.log 2024-09-12T14:19:29.518 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.91165.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.91165.log.gz 2024-09-12T14:19:29.518 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.92059.log 2024-09-12T14:19:29.518 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.99215.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.99215.log.gz 2024-09-12T14:19:29.518 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.111269.log 2024-09-12T14:19:29.518 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.92059.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.92059.log.gz 2024-09-12T14:19:29.518 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.136465.log 2024-09-12T14:19:29.518 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.111269.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.111269.log.gz 2024-09-12T14:19:29.518 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.109361.log 2024-09-12T14:19:29.519 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.136465.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.136465.log.gz 2024-09-12T14:19:29.519 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.59006.log 2024-09-12T14:19:29.521 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.109361.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.109361.log.gz 2024-09-12T14:19:29.521 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.73672.log 2024-09-12T14:19:29.521 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.59006.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.59006.log.gz 2024-09-12T14:19:29.521 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.74089.log 2024-09-12T14:19:29.521 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.73672.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.73672.log.gz 2024-09-12T14:19:29.521 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.122354.log 2024-09-12T14:19:29.521 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.74089.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.74089.log.gz 2024-09-12T14:19:29.522 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.41099.log 2024-09-12T14:19:29.522 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.122354.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.122354.log.gz 2024-09-12T14:19:29.523 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.138102.log 2024-09-12T14:19:29.523 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.41099.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.126589.log 2024-09-12T14:19:29.523 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.41099.log.gz 2024-09-12T14:19:29.523 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.138102.log: gzip 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.138102.log.gz -5 2024-09-12T14:19:29.523 INFO:teuthology.orchestra.run.smithi130.stderr: --verbose -- /var/log/ceph/cluster1-client.mirror.63691.log 2024-09-12T14:19:29.524 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.126589.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.126589.log.gz 2024-09-12T14:19:29.524 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.128504.log 2024-09-12T14:19:29.525 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.63691.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.63691.log.gz 2024-09-12T14:19:29.525 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.42093.log 2024-09-12T14:19:29.526 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.128504.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.128504.log.gz 2024-09-12T14:19:29.526 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.66298.log 2024-09-12T14:19:29.527 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.42093.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.42093.log.gz 2024-09-12T14:19:29.527 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.138699.log 2024-09-12T14:19:29.528 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.66298.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.66298.log.gz 2024-09-12T14:19:29.529 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.137867.log 2024-09-12T14:19:29.529 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.138699.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.138699.log.gz 2024-09-12T14:19:29.529 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.117883.log 2024-09-12T14:19:29.531 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.137867.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.137867.log.gz 2024-09-12T14:19:29.531 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.35109.log 2024-09-12T14:19:29.531 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.117883.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.117883.log.gz 2024-09-12T14:19:29.531 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.153759.log 2024-09-12T14:19:29.531 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.35109.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.35109.log.gz 2024-09-12T14:19:29.531 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.140327.log 2024-09-12T14:19:29.533 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.153759.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.153759.log.gz 2024-09-12T14:19:29.533 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.136148.log 2024-09-12T14:19:29.533 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.140327.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.140327.log.gz 2024-09-12T14:19:29.533 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.52853.log 2024-09-12T14:19:29.533 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.136148.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.136148.log.gz 2024-09-12T14:19:29.534 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.133019.log 2024-09-12T14:19:29.535 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.52853.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.52853.log.gz 2024-09-12T14:19:29.536 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.35764.log 2024-09-12T14:19:29.536 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.133019.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.85192.log 2024-09-12T14:19:29.536 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.133019.log.gz 2024-09-12T14:19:29.536 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.35764.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.35764.log.gz 2024-09-12T14:19:29.536 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.37300.log 2024-09-12T14:19:29.539 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.85192.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.85192.log.gz 2024-09-12T14:19:29.539 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.64782.log 2024-09-12T14:19:29.539 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.37300.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.37300.log.gz 2024-09-12T14:19:29.539 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.49409.log 2024-09-12T14:19:29.539 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.64782.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.64782.log.gz 2024-09-12T14:19:29.540 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.42345.log 2024-09-12T14:19:29.540 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.49409.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.49409.log.gz 2024-09-12T14:19:29.540 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.93251.log 2024-09-12T14:19:29.543 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.42345.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.42345.log.gz 2024-09-12T14:19:29.543 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.144766.log 2024-09-12T14:19:29.543 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.93251.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.93251.log.gz 2024-09-12T14:19:29.543 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.144982.log 2024-09-12T14:19:29.543 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.144766.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.144766.log.gz 2024-09-12T14:19:29.543 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.36880.log 2024-09-12T14:19:29.543 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.144982.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.144982.log.gz 2024-09-12T14:19:29.544 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.101654.log 2024-09-12T14:19:29.544 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.36880.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.36880.log.gz 2024-09-12T14:19:29.544 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.57902.log 2024-09-12T14:19:29.546 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.101654.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.101654.log.gz 2024-09-12T14:19:29.546 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.44877.log 2024-09-12T14:19:29.546 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.57902.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.57902.log.gz 2024-09-12T14:19:29.546 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.57922.log 2024-09-12T14:19:29.546 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.44877.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.44877.log.gz 2024-09-12T14:19:29.547 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.57995.log 2024-09-12T14:19:29.547 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.57922.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.57922.log.gz 2024-09-12T14:19:29.547 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.85570.log 2024-09-12T14:19:29.548 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.57995.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.57995.log.gz 2024-09-12T14:19:29.548 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.49911.log 2024-09-12T14:19:29.548 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.85570.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.85570.log.gz 2024-09-12T14:19:29.549 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.122055.log 2024-09-12T14:19:29.549 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.49911.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.49911.log.gz 2024-09-12T14:19:29.549 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.90810.log 2024-09-12T14:19:29.550 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.122055.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.122055.log.gz 2024-09-12T14:19:29.550 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.39530.log 2024-09-12T14:19:29.550 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.90810.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.90810.log.gz 2024-09-12T14:19:29.550 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.104681.log 2024-09-12T14:19:29.551 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.39530.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.39530.log.gz 2024-09-12T14:19:29.551 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.36049.log 2024-09-12T14:19:29.552 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.104681.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.104681.log.gz 2024-09-12T14:19:29.552 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.120441.log 2024-09-12T14:19:29.553 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.36049.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.36049.log.gz 2024-09-12T14:19:29.553 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.64515.log 2024-09-12T14:19:29.553 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.120441.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.120441.log.gz 2024-09-12T14:19:29.553 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.59517.log 2024-09-12T14:19:29.554 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.64515.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.64515.log.gz 2024-09-12T14:19:29.554 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.76567.log 2024-09-12T14:19:29.555 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.59517.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.59517.log.gz 2024-09-12T14:19:29.555 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.109358.log 2024-09-12T14:19:29.556 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.76567.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.76567.log.gz 2024-09-12T14:19:29.556 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.89330.log 2024-09-12T14:19:29.557 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.109358.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.109358.log.gz 2024-09-12T14:19:29.557 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.110829.log 2024-09-12T14:19:29.558 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.89330.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.89330.log.gz 2024-09-12T14:19:29.558 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.52707.log 2024-09-12T14:19:29.558 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.110829.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.110829.log.gz 2024-09-12T14:19:29.558 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.78599.log 2024-09-12T14:19:29.559 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.52707.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.52707.log.gz 2024-09-12T14:19:29.559 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.114190.log 2024-09-12T14:19:29.560 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.78599.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.78599.log.gz 2024-09-12T14:19:29.560 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.123889.log 2024-09-12T14:19:29.561 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.114190.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.114190.log.gz 2024-09-12T14:19:29.561 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.73141.log 2024-09-12T14:19:29.563 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.123889.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.123889.log.gz 2024-09-12T14:19:29.563 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.82666.log 2024-09-12T14:19:29.563 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.73141.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.73141.log.gz 2024-09-12T14:19:29.563 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.70143.log 2024-09-12T14:19:29.563 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.82666.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.82666.log.gz 2024-09-12T14:19:29.563 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.62223.log 2024-09-12T14:19:29.565 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.70143.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.70143.log.gz 2024-09-12T14:19:29.565 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.72599.log 2024-09-12T14:19:29.565 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.62223.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.62223.log.gz 2024-09-12T14:19:29.565 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.90144.log 2024-09-12T14:19:29.567 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.72599.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.72599.log.gz 2024-09-12T14:19:29.567 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.40894.log 2024-09-12T14:19:29.567 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.90144.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.90144.log.gz 2024-09-12T14:19:29.567 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.129751.log 2024-09-12T14:19:29.567 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.40894.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.40894.log.gz 2024-09-12T14:19:29.567 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.95544.log 2024-09-12T14:19:29.569 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.129751.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.84477.log 2024-09-12T14:19:29.569 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.129751.log.gz 2024-09-12T14:19:29.569 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.95544.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.95544.log.gz 2024-09-12T14:19:29.569 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.64343.log 2024-09-12T14:19:29.571 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.84477.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.84477.log.gz 2024-09-12T14:19:29.571 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.47627.log 2024-09-12T14:19:29.571 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.64343.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.64343.log.gz 2024-09-12T14:19:29.572 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.105182.log 2024-09-12T14:19:29.572 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.47627.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.47627.log.gz 2024-09-12T14:19:29.572 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.141147.log 2024-09-12T14:19:29.574 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.105182.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.105182.log.gz 2024-09-12T14:19:29.574 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.66026.log 2024-09-12T14:19:29.574 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.141147.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.141147.log.gz 2024-09-12T14:19:29.574 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.147432.log 2024-09-12T14:19:29.575 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.66026.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.66026.log.gz 2024-09-12T14:19:29.575 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.116137.log 2024-09-12T14:19:29.575 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.147432.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.147432.log.gz 2024-09-12T14:19:29.575 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.120795.log 2024-09-12T14:19:29.576 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.116137.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.116137.log.gz 2024-09-12T14:19:29.576 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.115196.log 2024-09-12T14:19:29.577 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.120795.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.120795.log.gz 2024-09-12T14:19:29.577 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.100854.log 2024-09-12T14:19:29.577 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.115196.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.115196.log.gz 2024-09-12T14:19:29.577 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.76917.log 2024-09-12T14:19:29.578 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.100854.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.100854.log.gz 2024-09-12T14:19:29.578 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.49010.log 2024-09-12T14:19:29.578 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.76917.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.76917.log.gz 2024-09-12T14:19:29.578 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.121007.log 2024-09-12T14:19:29.579 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.49010.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.49010.log.gz 2024-09-12T14:19:29.580 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.146547.log 2024-09-12T14:19:29.580 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.121007.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.121007.log.gz 2024-09-12T14:19:29.580 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.92371.log 2024-09-12T14:19:29.580 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.146547.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.146547.log.gz 2024-09-12T14:19:29.581 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.68936.log 2024-09-12T14:19:29.581 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.92371.log: gzip 0.0% -5 -- replaced with /var/log/ceph/cluster2-client.admin.92371.log.gz --verbose 2024-09-12T14:19:29.581 INFO:teuthology.orchestra.run.smithi130.stderr: -- /var/log/ceph/cluster1-client.mirror.122919.log 2024-09-12T14:19:29.582 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.68936.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.68936.log.gz 2024-09-12T14:19:29.582 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.134676.log 2024-09-12T14:19:29.583 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.122919.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.122919.log.gz 2024-09-12T14:19:29.583 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.147310.log 2024-09-12T14:19:29.584 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.134676.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.134676.log.gz 2024-09-12T14:19:29.584 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.121549.log 2024-09-12T14:19:29.584 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.147310.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.147310.log.gz 2024-09-12T14:19:29.585 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.100267.log 2024-09-12T14:19:29.585 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.121549.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.121549.log.gz 2024-09-12T14:19:29.585 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.85590.log 2024-09-12T14:19:29.586 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.100267.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.100267.log.gz 2024-09-12T14:19:29.586 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.121279.log 2024-09-12T14:19:29.587 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.85590.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.85590.log.gz 2024-09-12T14:19:29.587 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.138829.log 2024-09-12T14:19:29.588 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.121279.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.121279.log.gz 2024-09-12T14:19:29.588 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.72600.log 2024-09-12T14:19:29.589 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.138829.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.138829.log.gz 2024-09-12T14:19:29.589 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.142417.log 2024-09-12T14:19:29.589 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.72600.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.72600.log.gz 2024-09-12T14:19:29.589 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.54765.log 2024-09-12T14:19:29.590 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.142417.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.142417.log.gz 2024-09-12T14:19:29.590 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.140789.log 2024-09-12T14:19:29.591 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.54765.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.54765.log.gz 2024-09-12T14:19:29.591 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.143960.log 2024-09-12T14:19:29.592 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.140789.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.71933.log 0.0% 2024-09-12T14:19:29.592 INFO:teuthology.orchestra.run.smithi130.stderr: -- replaced with /var/log/ceph/cluster2-client.admin.140789.log.gz 2024-09-12T14:19:29.594 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.143960.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.143960.log.gz 2024-09-12T14:19:29.594 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.129161.log 2024-09-12T14:19:29.594 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.71933.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.71933.log.gz 2024-09-12T14:19:29.594 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.90470.log 2024-09-12T14:19:29.595 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.129161.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.129161.log.gz 2024-09-12T14:19:29.595 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.146109.log 2024-09-12T14:19:29.595 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.90470.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.90470.log.gz 2024-09-12T14:19:29.595 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.135689.log 2024-09-12T14:19:29.596 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.146109.log: 0.0%gzip -- replaced with /var/log/ceph/cluster1-client.mirror.146109.log.gz 2024-09-12T14:19:29.596 INFO:teuthology.orchestra.run.smithi130.stderr: -5 --verbose -- /var/log/ceph/cluster1-client.mirror.131121.log 2024-09-12T14:19:29.597 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.135689.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.135689.log.gz 2024-09-12T14:19:29.597 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.151218.log 2024-09-12T14:19:29.597 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.131121.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.131121.log.gz 2024-09-12T14:19:29.597 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.97054.log 2024-09-12T14:19:29.599 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.151218.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.151218.log.gz 2024-09-12T14:19:29.599 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.99447.log 2024-09-12T14:19:29.599 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.97054.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.97054.log.gz 2024-09-12T14:19:29.599 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.38371.log 2024-09-12T14:19:29.601 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.99447.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.99447.log.gz 2024-09-12T14:19:29.601 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.98188.log 2024-09-12T14:19:29.601 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.38371.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.38371.log.gz 2024-09-12T14:19:29.601 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.142549.log 2024-09-12T14:19:29.602 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.98188.log: gzip -5 --verbose 0.0% -- -- replaced with /var/log/ceph/cluster1-client.mirror.98188.log.gz 2024-09-12T14:19:29.602 INFO:teuthology.orchestra.run.smithi130.stderr: /var/log/ceph/cluster1-client.mirror.49306.log 2024-09-12T14:19:29.604 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.142549.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.142549.log.gz 2024-09-12T14:19:29.604 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.94026.log 2024-09-12T14:19:29.604 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.49306.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.49306.log.gz 2024-09-12T14:19:29.604 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.147958.log 2024-09-12T14:19:29.604 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.94026.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.94026.log.gz 2024-09-12T14:19:29.604 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.108813.log 2024-09-12T14:19:29.607 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.147958.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.147958.log.gz 2024-09-12T14:19:29.607 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.53610.log 2024-09-12T14:19:29.607 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.108813.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.108813.log.gz 2024-09-12T14:19:29.607 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.141892.log 2024-09-12T14:19:29.607 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.53610.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.53610.log.gz 2024-09-12T14:19:29.608 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.137174.log 2024-09-12T14:19:29.608 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.141892.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.141892.log.gz 2024-09-12T14:19:29.608 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.121829.log 2024-09-12T14:19:29.609 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.137174.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.137174.log.gz 2024-09-12T14:19:29.609 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.53106.log 2024-09-12T14:19:29.609 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.121829.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.121829.log.gz 2024-09-12T14:19:29.610 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.101183.log 2024-09-12T14:19:29.610 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.53106.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.53106.log.gz 2024-09-12T14:19:29.610 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.105752.log 2024-09-12T14:19:29.611 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.101183.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.101183.log.gz 2024-09-12T14:19:29.611 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.36746.log 2024-09-12T14:19:29.611 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.105752.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.105752.log.gz 2024-09-12T14:19:29.611 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.74865.log 2024-09-12T14:19:29.612 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.36746.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.36746.log.gz 2024-09-12T14:19:29.612 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.144210.log 2024-09-12T14:19:29.613 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.74865.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.74865.log.gz 2024-09-12T14:19:29.613 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.105143.log 2024-09-12T14:19:29.614 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.144210.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.144210.log.gz 2024-09-12T14:19:29.614 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.58887.log 2024-09-12T14:19:29.614 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.105143.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.105143.log.gz 2024-09-12T14:19:29.614 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.121429.log 2024-09-12T14:19:29.615 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.58887.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.58887.log.gz 2024-09-12T14:19:29.615 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.138410.log 2024-09-12T14:19:29.616 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.121429.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.121429.log.gz 2024-09-12T14:19:29.616 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.50561.log 2024-09-12T14:19:29.617 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.138410.log: gzip -5 --verbose -- 0.0% /var/log/ceph/cluster1-client.mirror.87068.log -- replaced with /var/log/ceph/cluster1-client.admin.138410.log.gz 2024-09-12T14:19:29.617 INFO:teuthology.orchestra.run.smithi130.stderr: 2024-09-12T14:19:29.617 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.50561.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.50561.log.gz 2024-09-12T14:19:29.618 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.98546.log 2024-09-12T14:19:29.618 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.87068.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.87068.log.gz 2024-09-12T14:19:29.618 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.97708.log 2024-09-12T14:19:29.619 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.98546.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.98546.log.gz 2024-09-12T14:19:29.619 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.62096.log 2024-09-12T14:19:29.620 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.97708.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.97708.log.gz 2024-09-12T14:19:29.620 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.112847.log 2024-09-12T14:19:29.621 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.62096.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.62096.log.gz 2024-09-12T14:19:29.621 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.135770.log 2024-09-12T14:19:29.621 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.112847.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.112847.log.gz 2024-09-12T14:19:29.622 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.112047.log 2024-09-12T14:19:29.622 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.135770.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.135770.log.gz 2024-09-12T14:19:29.622 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.145861.log 2024-09-12T14:19:29.623 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.112047.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.112047.log.gz 2024-09-12T14:19:29.623 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.84539.log 2024-09-12T14:19:29.624 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.145861.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.145861.log.gz 2024-09-12T14:19:29.624 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.99094.log 2024-09-12T14:19:29.625 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.84539.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.84539.log.gz 2024-09-12T14:19:29.625 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.102856.log 2024-09-12T14:19:29.626 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.99094.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.99094.log.gz 2024-09-12T14:19:29.626 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.149661.log 2024-09-12T14:19:29.627 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.102856.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.102856.log.gz 2024-09-12T14:19:29.627 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.147245.log 2024-09-12T14:19:29.628 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.149661.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.149661.log.gz 2024-09-12T14:19:29.628 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.125049.log 2024-09-12T14:19:29.628 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.147245.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.147245.log.gz 2024-09-12T14:19:29.628 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.120522.log 2024-09-12T14:19:29.630 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.125049.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.125049.log.gz 2024-09-12T14:19:29.630 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.69576.log 2024-09-12T14:19:29.630 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.120522.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.120522.log.gz 2024-09-12T14:19:29.630 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.54031.log 2024-09-12T14:19:29.632 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.69576.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.69576.log.gz 2024-09-12T14:19:29.632 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.52095.log 2024-09-12T14:19:29.632 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.54031.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.54031.log.gz 2024-09-12T14:19:29.632 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.129503.log 2024-09-12T14:19:29.632 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.52095.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.52095.log.gz 2024-09-12T14:19:29.633 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.108292.log 2024-09-12T14:19:29.634 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.129503.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.129503.log.gz 2024-09-12T14:19:29.634 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.122688.log 2024-09-12T14:19:29.634 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.108292.log: 0.0%gzip -- replaced with /var/log/ceph/cluster1-client.mirror.108292.log.gz -5 2024-09-12T14:19:29.634 INFO:teuthology.orchestra.run.smithi130.stderr: --verbose -- /var/log/ceph/cluster1-client.mirror.96883.log 2024-09-12T14:19:29.635 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.122688.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.122688.log.gz 2024-09-12T14:19:29.635 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.61762.log 2024-09-12T14:19:29.635 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.96883.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.96883.log.gz 2024-09-12T14:19:29.636 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.48694.log 2024-09-12T14:19:29.636 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.61762.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.61762.log.gz 2024-09-12T14:19:29.637 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.64531.log 2024-09-12T14:19:29.639 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.48694.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.120354.log 2024-09-12T14:19:29.639 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.48694.log.gz 2024-09-12T14:19:29.639 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.64531.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.64531.log.gz 2024-09-12T14:19:29.639 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.73229.log 2024-09-12T14:19:29.639 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.120354.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.120354.log.gz 2024-09-12T14:19:29.639 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.38057.log 2024-09-12T14:19:29.640 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.73229.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.151955.log 2024-09-12T14:19:29.641 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.73229.log.gz 2024-09-12T14:19:29.641 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.38057.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.38057.log.gz 2024-09-12T14:19:29.641 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.102011.log 2024-09-12T14:19:29.642 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.151955.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.71164.log 2024-09-12T14:19:29.642 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.151955.log.gz 2024-09-12T14:19:29.642 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.102011.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.102011.log.gz 2024-09-12T14:19:29.642 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.93544.log 2024-09-12T14:19:29.643 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.71164.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.71164.log.gz 2024-09-12T14:19:29.643 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.149284.log 2024-09-12T14:19:29.644 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.93544.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.93544.log.gz 2024-09-12T14:19:29.644 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.116151.log 2024-09-12T14:19:29.645 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.149284.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.149284.log.gz 2024-09-12T14:19:29.645 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.96279.log 2024-09-12T14:19:29.645 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.116151.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.116151.log.gz 2024-09-12T14:19:29.645 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.97096.log 2024-09-12T14:19:29.646 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.96279.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.96279.log.gz 2024-09-12T14:19:29.646 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.114866.log 2024-09-12T14:19:29.647 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.97096.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.97096.log.gz 2024-09-12T14:19:29.647 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.117468.log 2024-09-12T14:19:29.647 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.114866.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.134342.log 2024-09-12T14:19:29.647 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.114866.log.gz 2024-09-12T14:19:29.648 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.117468.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.117468.log.gz 2024-09-12T14:19:29.648 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.154130.log 2024-09-12T14:19:29.649 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.134342.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.134342.log.gz 2024-09-12T14:19:29.649 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.128585.log 2024-09-12T14:19:29.650 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.154130.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.154130.log.gz 2024-09-12T14:19:29.650 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.130161.log 2024-09-12T14:19:29.650 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.128585.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.128585.log.gz 2024-09-12T14:19:29.651 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.138477.log 2024-09-12T14:19:29.651 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.130161.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.145963.log 2024-09-12T14:19:29.651 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.130161.log.gz 2024-09-12T14:19:29.652 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.138477.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.138477.log.gz 2024-09-12T14:19:29.652 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.97408.log 2024-09-12T14:19:29.653 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.145963.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.145963.log.gz 2024-09-12T14:19:29.653 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.90224.log 2024-09-12T14:19:29.654 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.97408.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.97408.log.gz 2024-09-12T14:19:29.654 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.139701.log 2024-09-12T14:19:29.654 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.90224.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.90224.log.gz 2024-09-12T14:19:29.655 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.84120.log 2024-09-12T14:19:29.655 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.139701.log: gzip 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.139701.log.gz -5 2024-09-12T14:19:29.655 INFO:teuthology.orchestra.run.smithi130.stderr: --verbose -- /var/log/ceph/cluster1-client.mirror.89980.log 2024-09-12T14:19:29.656 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.84120.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.84120.log.gz 2024-09-12T14:19:29.656 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.121592.log 2024-09-12T14:19:29.657 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.89980.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.44875.log 2024-09-12T14:19:29.657 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.89980.log.gz 2024-09-12T14:19:29.657 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.121592.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.121592.log.gz 2024-09-12T14:19:29.658 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.79370.log 2024-09-12T14:19:29.659 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.44875.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.44875.log.gz 2024-09-12T14:19:29.659 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.111038.log 2024-09-12T14:19:29.660 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.79370.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.79370.log.gz 2024-09-12T14:19:29.660 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.63187.log 2024-09-12T14:19:29.660 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.111038.log: gzip -5 --verbose -- 0.0% /var/log/ceph/cluster1-client.mirror.62610.log 2024-09-12T14:19:29.660 INFO:teuthology.orchestra.run.smithi130.stderr: -- replaced with /var/log/ceph/cluster1-client.mirror.111038.log.gz 2024-09-12T14:19:29.661 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.63187.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.63187.log.gz 2024-09-12T14:19:29.661 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.109779.log 2024-09-12T14:19:29.662 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.62610.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.62610.log.gz 2024-09-12T14:19:29.662 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.102857.log 2024-09-12T14:19:29.662 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.109779.log: 0.0%gzip -- replaced with /var/log/ceph/cluster1-client.mirror.109779.log.gz 2024-09-12T14:19:29.663 INFO:teuthology.orchestra.run.smithi130.stderr: -5 --verbose -- /var/log/ceph/cluster1-client.mirror.133062.log 2024-09-12T14:19:29.663 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.102857.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.102857.log.gz 2024-09-12T14:19:29.663 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.141737.log 2024-09-12T14:19:29.664 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.133062.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.96631.log 2024-09-12T14:19:29.664 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.133062.log.gz 2024-09-12T14:19:29.665 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.141737.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.79977.log 0.0% 2024-09-12T14:19:29.665 INFO:teuthology.orchestra.run.smithi130.stderr: -- replaced with /var/log/ceph/cluster2-client.mirror.141737.log.gz 2024-09-12T14:19:29.665 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.96631.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.96631.log.gz 2024-09-12T14:19:29.665 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.114954.log 2024-09-12T14:19:29.666 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.79977.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.79977.log.gz 2024-09-12T14:19:29.666 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.34316.log 2024-09-12T14:19:29.667 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.114954.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.114954.log.gz 2024-09-12T14:19:29.667 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.82174.log 2024-09-12T14:19:29.668 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.34316.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.34316.log.gz 2024-09-12T14:19:29.668 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.132576.log 2024-09-12T14:19:29.670 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.82174.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.139339.log 2024-09-12T14:19:29.670 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.82174.log.gz 2024-09-12T14:19:29.670 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.132576.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.132576.log.gz 2024-09-12T14:19:29.670 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.56807.log 2024-09-12T14:19:29.672 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.139339.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.139339.log.gz 2024-09-12T14:19:29.673 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.117334.log 2024-09-12T14:19:29.673 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.56807.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.56807.log.gz 2024-09-12T14:19:29.673 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.63354.log 2024-09-12T14:19:29.673 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.117334.log: gzip 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.117334.log.gz 2024-09-12T14:19:29.673 INFO:teuthology.orchestra.run.smithi130.stderr: -5 --verbose -- /var/log/ceph/cluster1-client.mirror.64356.log 2024-09-12T14:19:29.675 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.63354.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.63354.log.gz 2024-09-12T14:19:29.675 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.146343.log 2024-09-12T14:19:29.675 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.64356.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.64356.log.gz 2024-09-12T14:19:29.676 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.42428.log 2024-09-12T14:19:29.676 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.146343.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.146343.log.gz 2024-09-12T14:19:29.676 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.122586.log 2024-09-12T14:19:29.676 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.42428.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.42428.log.gz 2024-09-12T14:19:29.676 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.86114.log 2024-09-12T14:19:29.678 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.122586.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.122586.log.gz 2024-09-12T14:19:29.678 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.107584.log 2024-09-12T14:19:29.678 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.86114.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.86114.log.gz 2024-09-12T14:19:29.678 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.79347.log 2024-09-12T14:19:29.678 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.107584.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.107584.log.gz 2024-09-12T14:19:29.678 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.42557.log 2024-09-12T14:19:29.679 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.79347.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.79347.log.gz 2024-09-12T14:19:29.679 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.130359.log 2024-09-12T14:19:29.680 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.42557.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.42557.log.gz 2024-09-12T14:19:29.680 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.152759.log 2024-09-12T14:19:29.681 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.130359.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.130359.log.gz 2024-09-12T14:19:29.681 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.55907.log 2024-09-12T14:19:29.681 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.152759.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.152759.log.gz 2024-09-12T14:19:29.681 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.59274.log 2024-09-12T14:19:29.682 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.55907.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.55907.log.gz 2024-09-12T14:19:29.682 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.105436.log 2024-09-12T14:19:29.683 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.59274.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.59274.log.gz 2024-09-12T14:19:29.683 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.49638.log 2024-09-12T14:19:29.683 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.105436.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.105436.log.gz 2024-09-12T14:19:29.683 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.126586.log 2024-09-12T14:19:29.685 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.49638.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.47753.log 2024-09-12T14:19:29.685 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.49638.log.gz 2024-09-12T14:19:29.685 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.126586.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.126586.log.gz 2024-09-12T14:19:29.685 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.124098.log 2024-09-12T14:19:29.686 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.47753.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.47753.log.gz 2024-09-12T14:19:29.686 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.116694.log 2024-09-12T14:19:29.686 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.124098.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.124098.log.gz 2024-09-12T14:19:29.687 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.116923.log 2024-09-12T14:19:29.687 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.116694.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.116694.log.gz 2024-09-12T14:19:29.687 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.150563.log 2024-09-12T14:19:29.688 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.116923.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.116923.log.gz 2024-09-12T14:19:29.688 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.134091.log 2024-09-12T14:19:29.689 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.150563.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.150563.log.gz 2024-09-12T14:19:29.689 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.60525.log 2024-09-12T14:19:29.690 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.134091.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.134091.log.gz 2024-09-12T14:19:29.690 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.59558.log 2024-09-12T14:19:29.691 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.60525.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.60525.log.gz 2024-09-12T14:19:29.691 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.134511.log 2024-09-12T14:19:29.692 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.59558.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.59558.log.gz 2024-09-12T14:19:29.692 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.46173.log 2024-09-12T14:19:29.692 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.134511.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.134511.log.gz 2024-09-12T14:19:29.692 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.55000.log 2024-09-12T14:19:29.693 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.46173.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.46173.log.gz 2024-09-12T14:19:29.693 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.127115.log 2024-09-12T14:19:29.694 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.55000.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.55000.log.gz 2024-09-12T14:19:29.694 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.70753.log 2024-09-12T14:19:29.695 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.127115.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.127115.log.gz 2024-09-12T14:19:29.696 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.132473.log 2024-09-12T14:19:29.696 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.70753.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.70753.log.gz 2024-09-12T14:19:29.696 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.60238.log 2024-09-12T14:19:29.696 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.132473.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.132473.log.gz 2024-09-12T14:19:29.697 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.63821.log 2024-09-12T14:19:29.699 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.60238.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.60238.log.gz 2024-09-12T14:19:29.699 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.66770.log 2024-09-12T14:19:29.699 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.63821.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.63821.log.gz 2024-09-12T14:19:29.699 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.136128.log 2024-09-12T14:19:29.699 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.66770.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.66770.log.gz 2024-09-12T14:19:29.699 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.66469.log 2024-09-12T14:19:29.701 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.136128.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.136128.log.gz 2024-09-12T14:19:29.701 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.113860.log 2024-09-12T14:19:29.701 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.66469.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.66469.log.gz 2024-09-12T14:19:29.701 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.87373.log 2024-09-12T14:19:29.702 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.113860.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.113860.log.gz 2024-09-12T14:19:29.702 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.73268.log 2024-09-12T14:19:29.704 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.87373.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.87373.log.gz 2024-09-12T14:19:29.704 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.133921.log 2024-09-12T14:19:29.704 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.73268.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.73268.log.gz 2024-09-12T14:19:29.705 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.44116.log 2024-09-12T14:19:29.705 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.133921.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.133921.log.gz 2024-09-12T14:19:29.705 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.55592.log 2024-09-12T14:19:29.705 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.44116.log: gzip -5 --verbose -- 0.0% /var/log/ceph/cluster1-client.admin.142064.log -- replaced with /var/log/ceph/cluster1-client.admin.44116.log.gz 2024-09-12T14:19:29.705 INFO:teuthology.orchestra.run.smithi130.stderr: 2024-09-12T14:19:29.708 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.55592.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.55592.log.gz 2024-09-12T14:19:29.708 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.51653.log 2024-09-12T14:19:29.708 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.142064.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.142064.log.gz 2024-09-12T14:19:29.708 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.115828.log 2024-09-12T14:19:29.708 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.51653.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.51653.log.gz 2024-09-12T14:19:29.708 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.133735.log 2024-09-12T14:19:29.709 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.115828.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.115828.log.gz 2024-09-12T14:19:29.709 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.130395.log 2024-09-12T14:19:29.709 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.133735.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.133735.log.gz 2024-09-12T14:19:29.709 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.48094.log 2024-09-12T14:19:29.711 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.130395.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.130395.log.gz 2024-09-12T14:19:29.711 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.81022.log 2024-09-12T14:19:29.711 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.48094.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.48094.log.gz 2024-09-12T14:19:29.711 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.44094.log 2024-09-12T14:19:29.711 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.81022.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.81022.log.gz 2024-09-12T14:19:29.711 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.106536.log 2024-09-12T14:19:29.711 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.44094.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.44094.log.gz 2024-09-12T14:19:29.712 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.98103.log 2024-09-12T14:19:29.713 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.106536.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.106536.log.gz 2024-09-12T14:19:29.713 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.66993.log 2024-09-12T14:19:29.713 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.98103.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.98103.log.gz 2024-09-12T14:19:29.713 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.130093.log 2024-09-12T14:19:29.714 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.66993.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.66993.log.gz 2024-09-12T14:19:29.714 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.46945.log 2024-09-12T14:19:29.714 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.130093.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.130093.log.gz 2024-09-12T14:19:29.714 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.85232.log 2024-09-12T14:19:29.716 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.46945.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.46945.log.gz 2024-09-12T14:19:29.716 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.67605.log 2024-09-12T14:19:29.716 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.85232.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.85232.log.gz 2024-09-12T14:19:29.716 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.148108.log 2024-09-12T14:19:29.717 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.67605.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.67605.log.gz 2024-09-12T14:19:29.718 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.88857.log 2024-09-12T14:19:29.718 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.148108.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.148108.log.gz 2024-09-12T14:19:29.718 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.139653.log 2024-09-12T14:19:29.719 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.88857.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.88857.log.gz 2024-09-12T14:19:29.719 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.145964.log 2024-09-12T14:19:29.719 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.139653.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.139653.log.gz 2024-09-12T14:19:29.719 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.136780.log 2024-09-12T14:19:29.720 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.145964.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.145964.log.gz 2024-09-12T14:19:29.720 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.85950.log 2024-09-12T14:19:29.721 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.136780.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.136780.log.gz 2024-09-12T14:19:29.722 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.42010.log 2024-09-12T14:19:29.722 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.85950.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.85950.log.gz 2024-09-12T14:19:29.722 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.106838.log 2024-09-12T14:19:29.722 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.42010.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.42010.log.gz 2024-09-12T14:19:29.723 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.55885.log 2024-09-12T14:19:29.723 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.106838.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.106838.log.gz 2024-09-12T14:19:29.723 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.114010.log 2024-09-12T14:19:29.724 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.55885.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.55885.log.gz 2024-09-12T14:19:29.724 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.74298.log 2024-09-12T14:19:29.725 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.114010.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.114010.log.gz 2024-09-12T14:19:29.726 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.34190.log 2024-09-12T14:19:29.726 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.74298.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.74298.log.gz 2024-09-12T14:19:29.726 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.89624.log 2024-09-12T14:19:29.727 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.34190.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.34190.log.gz 2024-09-12T14:19:29.727 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.107484.log 2024-09-12T14:19:29.727 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.89624.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.89624.log.gz 2024-09-12T14:19:29.728 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.135351.log 2024-09-12T14:19:29.729 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.107484.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.107484.log.gz 2024-09-12T14:19:29.729 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.46925.log 2024-09-12T14:19:29.729 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.135351.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.135351.log.gz 2024-09-12T14:19:29.729 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.119026.log 2024-09-12T14:19:29.730 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.46925.log: gzip 0.0% -5 -- replaced with /var/log/ceph/cluster1-client.mirror.46925.log.gz 2024-09-12T14:19:29.730 INFO:teuthology.orchestra.run.smithi130.stderr: --verbose -- /var/log/ceph/cluster1-client.mirror.51509.log 2024-09-12T14:19:29.731 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.119026.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.119026.log.gz 2024-09-12T14:19:29.731 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.136614.log 2024-09-12T14:19:29.732 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.51509.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.51509.log.gzgzip 2024-09-12T14:19:29.732 INFO:teuthology.orchestra.run.smithi130.stderr: -5 --verbose -- /var/log/ceph/cluster1-client.mirror.106047.log 2024-09-12T14:19:29.732 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.136614.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.136614.log.gz 2024-09-12T14:19:29.733 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.145334.log 2024-09-12T14:19:29.734 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.106047.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.106047.log.gz 2024-09-12T14:19:29.734 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.68916.log 2024-09-12T14:19:29.734 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.145334.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.145334.log.gz 2024-09-12T14:19:29.734 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.85170.log 2024-09-12T14:19:29.735 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.68916.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.68916.log.gz 2024-09-12T14:19:29.735 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.70351.log 2024-09-12T14:19:29.736 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.85170.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.85170.log.gz 2024-09-12T14:19:29.736 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.141188.log 2024-09-12T14:19:29.737 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.70351.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.70351.log.gz 2024-09-12T14:19:29.737 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.61465.log 2024-09-12T14:19:29.737 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.141188.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.141188.log.gz 2024-09-12T14:19:29.738 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.105130.log 2024-09-12T14:19:29.739 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.61465.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.61465.log.gz 2024-09-12T14:19:29.739 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.146067.log 2024-09-12T14:19:29.739 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.105130.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.105130.log.gz 2024-09-12T14:19:29.739 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.64007.log 2024-09-12T14:19:29.740 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.146067.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.146067.log.gz 2024-09-12T14:19:29.740 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.133859.log 2024-09-12T14:19:29.740 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.64007.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.64007.log.gz 2024-09-12T14:19:29.740 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.110595.log 2024-09-12T14:19:29.741 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.133859.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.133859.log.gz 2024-09-12T14:19:29.741 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.75831.log 2024-09-12T14:19:29.742 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.110595.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.110595.log.gz 2024-09-12T14:19:29.742 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.35075.log 2024-09-12T14:19:29.742 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.75831.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.75831.log.gz 2024-09-12T14:19:29.742 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.143647.log 2024-09-12T14:19:29.743 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.35075.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.35075.log.gz 2024-09-12T14:19:29.743 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.39842.log 2024-09-12T14:19:29.744 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.143647.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.143647.log.gz 2024-09-12T14:19:29.744 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.144767.log 2024-09-12T14:19:29.751 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.39842.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.39842.log.gz 2024-09-12T14:19:29.751 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.96005.log 2024-09-12T14:19:29.752 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.144767.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.144767.log.gz 2024-09-12T14:19:29.752 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.108247.log 2024-09-12T14:19:29.752 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.96005.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.96005.log.gz 2024-09-12T14:19:29.752 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.79567.log 2024-09-12T14:19:29.752 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.108247.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.108247.log.gz 2024-09-12T14:19:29.752 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.131489.log 2024-09-12T14:19:29.752 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.79567.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.79567.log.gz 2024-09-12T14:19:29.752 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.56064.log 2024-09-12T14:19:29.752 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.131489.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.131489.log.gz 2024-09-12T14:19:29.752 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.119442.log 2024-09-12T14:19:29.752 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.56064.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.56064.log.gz 2024-09-12T14:19:29.752 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.133626.log 2024-09-12T14:19:29.753 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.119442.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.119442.log.gz 2024-09-12T14:19:29.753 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.110114.log 2024-09-12T14:19:29.753 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.133626.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.40474.log 2024-09-12T14:19:29.753 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.133626.log.gz 2024-09-12T14:19:29.754 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.110114.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.110114.log.gz 2024-09-12T14:19:29.754 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.85045.log 2024-09-12T14:19:29.754 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.40474.log: gzip 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.40474.log.gz -5 2024-09-12T14:19:29.754 INFO:teuthology.orchestra.run.smithi130.stderr: --verbose -- /var/log/ceph/cluster1-client.mirror.145711.log 2024-09-12T14:19:29.754 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.85045.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.85045.log.gz 2024-09-12T14:19:29.754 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.135185.log 2024-09-12T14:19:29.755 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.145711.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.145711.log.gz 2024-09-12T14:19:29.755 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.79455.log 2024-09-12T14:19:29.755 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.135185.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.135185.log.gz 2024-09-12T14:19:29.756 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.145122.log 2024-09-12T14:19:29.756 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.79455.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.79455.log.gz 2024-09-12T14:19:29.756 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.113308.log 2024-09-12T14:19:29.757 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.145122.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.145122.log.gz 2024-09-12T14:19:29.757 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.146760.log 2024-09-12T14:19:29.758 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.113308.log: 0.0%gzip -- replaced with /var/log/ceph/cluster1-client.mirror.113308.log.gz 2024-09-12T14:19:29.758 INFO:teuthology.orchestra.run.smithi130.stderr: -5 --verbose -- /var/log/ceph/cluster2-client.mirror.58337.log 2024-09-12T14:19:29.758 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.146760.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.146760.log.gz 2024-09-12T14:19:29.758 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.151138.log 2024-09-12T14:19:29.759 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.58337.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.58337.log.gz 2024-09-12T14:19:29.760 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.151299.log 2024-09-12T14:19:29.761 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.151138.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.151138.log.gz 2024-09-12T14:19:29.761 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.98478.log 2024-09-12T14:19:29.761 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.151299.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.151299.log.gz 2024-09-12T14:19:29.761 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.34955.log 2024-09-12T14:19:29.762 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.98478.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.98478.log.gz 2024-09-12T14:19:29.762 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.66628.log 2024-09-12T14:19:29.763 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.34955.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.34955.log.gz 2024-09-12T14:19:29.763 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.45892.log 2024-09-12T14:19:29.763 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.66628.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.66628.log.gz 2024-09-12T14:19:29.764 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.141550.log 2024-09-12T14:19:29.764 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.45892.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.45892.log.gz 2024-09-12T14:19:29.764 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.71907.log 2024-09-12T14:19:29.765 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.141550.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.141550.log.gz 2024-09-12T14:19:29.765 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.50919.log 2024-09-12T14:19:29.766 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.71907.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.71907.log.gz 2024-09-12T14:19:29.766 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.50960.log 2024-09-12T14:19:29.766 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.50919.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.50919.log.gz 2024-09-12T14:19:29.767 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.137474.log 2024-09-12T14:19:29.767 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.50960.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.50960.log.gz 2024-09-12T14:19:29.767 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.52039.log 2024-09-12T14:19:29.768 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.137474.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.55812.log 2024-09-12T14:19:29.768 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.137474.log.gz 2024-09-12T14:19:29.769 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.52039.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.52039.log.gz 2024-09-12T14:19:29.769 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.75332.log 2024-09-12T14:19:29.770 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.55812.log: gzip -5 0.0% --verbose -- replaced with /var/log/ceph/cluster1-client.admin.55812.log.gz -- 2024-09-12T14:19:29.770 INFO:teuthology.orchestra.run.smithi130.stderr: /var/log/ceph/cluster1-client.mirror.146240.log 2024-09-12T14:19:29.771 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.75332.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.75332.log.gz 2024-09-12T14:19:29.771 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.149726.log 2024-09-12T14:19:29.771 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.146240.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.146240.log.gz 2024-09-12T14:19:29.771 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.146197.log 2024-09-12T14:19:29.772 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.149726.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.149726.log.gz 2024-09-12T14:19:29.772 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.88806.log 2024-09-12T14:19:29.773 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.146197.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.146197.log.gz 2024-09-12T14:19:29.773 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.88551.log 2024-09-12T14:19:29.774 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.88806.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.88806.log.gz 2024-09-12T14:19:29.774 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.78379.log 2024-09-12T14:19:29.774 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.88551.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.88551.log.gz 2024-09-12T14:19:29.774 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.131804.log 2024-09-12T14:19:29.775 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.78379.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.78379.log.gz 2024-09-12T14:19:29.775 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.74890.log 2024-09-12T14:19:29.776 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.131804.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.131804.log.gz 2024-09-12T14:19:29.776 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.74911.log 2024-09-12T14:19:29.777 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.74890.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.74890.log.gz 2024-09-12T14:19:29.777 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.51931.log 2024-09-12T14:19:29.778 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.74911.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.105081.log 2024-09-12T14:19:29.778 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.74911.log.gz 2024-09-12T14:19:29.778 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.51931.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.51931.log.gz 2024-09-12T14:19:29.778 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.64701.log 2024-09-12T14:19:29.779 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.105081.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.105081.log.gz 2024-09-12T14:19:29.779 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.75940.log 2024-09-12T14:19:29.780 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.64701.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.64701.log.gz 2024-09-12T14:19:29.780 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.54030.log 2024-09-12T14:19:29.781 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.75940.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.75940.log.gz 2024-09-12T14:19:29.781 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.57148.log 2024-09-12T14:19:29.782 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.54030.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.54030.log.gz 2024-09-12T14:19:29.782 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.121405.log 2024-09-12T14:19:29.782 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.57148.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.57148.log.gz 2024-09-12T14:19:29.782 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.129658.log 2024-09-12T14:19:29.783 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.121405.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.121405.log.gz 2024-09-12T14:19:29.783 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.82776.log 2024-09-12T14:19:29.784 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.129658.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.129658.log.gz 2024-09-12T14:19:29.784 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.35211.log 2024-09-12T14:19:29.785 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.82776.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.82776.log.gz 2024-09-12T14:19:29.785 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.150716.log 2024-09-12T14:19:29.786 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.35211.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.35211.log.gz 2024-09-12T14:19:29.786 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.61802.log 2024-09-12T14:19:29.786 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.150716.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.150716.log.gz 2024-09-12T14:19:29.786 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.75054.log 2024-09-12T14:19:29.787 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.61802.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.61802.log.gz 2024-09-12T14:19:29.787 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.97161.log 2024-09-12T14:19:29.788 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.75054.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.75054.log.gz 2024-09-12T14:19:29.788 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.64661.log 2024-09-12T14:19:29.789 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.97161.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.97161.log.gz 2024-09-12T14:19:29.789 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.143643.log 2024-09-12T14:19:29.790 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.64661.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.153531.log 2024-09-12T14:19:29.790 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.64661.log.gz 2024-09-12T14:19:29.790 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.143643.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.143643.log.gz 2024-09-12T14:19:29.791 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.146319.log 2024-09-12T14:19:29.791 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.153531.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.153531.log.gz 2024-09-12T14:19:29.791 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.129819.log 2024-09-12T14:19:29.792 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.146319.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.146319.log.gz 2024-09-12T14:19:29.792 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.73859.log 2024-09-12T14:19:29.793 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.129819.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.129819.log.gz 2024-09-12T14:19:29.793 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.132409.log 2024-09-12T14:19:29.794 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.73859.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.82824.log 2024-09-12T14:19:29.794 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.73859.log.gz 2024-09-12T14:19:29.795 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.132409.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.132409.log.gz 2024-09-12T14:19:29.795 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.50710.log 2024-09-12T14:19:29.796 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.82824.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.82824.log.gz 2024-09-12T14:19:29.796 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.29301.log 2024-09-12T14:19:29.797 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.50710.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.50710.log.gz 2024-09-12T14:19:29.797 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.124190.log 2024-09-12T14:19:29.798 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.29301.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.29301.log.gz 2024-09-12T14:19:29.798 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.149784.log 2024-09-12T14:19:29.798 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.124190.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.124190.log.gz 2024-09-12T14:19:29.799 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.72495.log 2024-09-12T14:19:29.799 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.149784.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.149784.log.gz 2024-09-12T14:19:29.799 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.125767.log 2024-09-12T14:19:29.800 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.72495.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.45499.log 2024-09-12T14:19:29.800 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.72495.log.gz 2024-09-12T14:19:29.800 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.125767.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.125767.log.gz 2024-09-12T14:19:29.800 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.141416.log 2024-09-12T14:19:29.801 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.45499.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.45499.log.gz 2024-09-12T14:19:29.801 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.114868.log 2024-09-12T14:19:29.802 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.141416.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.141416.log.gz 2024-09-12T14:19:29.802 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.148315.log 2024-09-12T14:19:29.803 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.114868.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.114868.log.gz 2024-09-12T14:19:29.803 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.57423.log 2024-09-12T14:19:29.803 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.148315.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.148315.log.gz 2024-09-12T14:19:29.803 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.153845.log 2024-09-12T14:19:29.804 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.57423.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.57423.log.gz 2024-09-12T14:19:29.804 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.78296.log 2024-09-12T14:19:29.805 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.153845.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.153845.log.gz 2024-09-12T14:19:29.805 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.63400.log 2024-09-12T14:19:29.806 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.78296.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.78296.log.gz 2024-09-12T14:19:29.806 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.74547.log 2024-09-12T14:19:29.806 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.63400.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.63400.log.gz 2024-09-12T14:19:29.807 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.44051.log 2024-09-12T14:19:29.807 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.74547.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.74547.log.gz 2024-09-12T14:19:29.807 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.83287.log 2024-09-12T14:19:29.808 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.44051.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.44051.log.gzgzip 2024-09-12T14:19:29.808 INFO:teuthology.orchestra.run.smithi130.stderr: -5 --verbose -- /var/log/ceph/cluster1-client.mirror.110536.log 2024-09-12T14:19:29.810 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.83287.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.83287.log.gz 2024-09-12T14:19:29.810 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.55878.log 2024-09-12T14:19:29.810 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.110536.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.110536.log.gz 2024-09-12T14:19:29.810 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.104683.log 2024-09-12T14:19:29.811 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.55878.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.55878.log.gz 2024-09-12T14:19:29.811 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.125834.log 2024-09-12T14:19:29.811 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.104683.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.104683.log.gz 2024-09-12T14:19:29.811 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.99656.log 2024-09-12T14:19:29.812 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.125834.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.125834.log.gz 2024-09-12T14:19:29.812 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.73502.log 2024-09-12T14:19:29.813 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.99656.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.99656.log.gz 2024-09-12T14:19:29.813 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.84876.log 2024-09-12T14:19:29.814 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.73502.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.73502.log.gz 2024-09-12T14:19:29.814 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.58492.log 2024-09-12T14:19:29.814 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.84876.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.84876.log.gz 2024-09-12T14:19:29.814 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.59383.log 2024-09-12T14:19:29.815 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.58492.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.58492.log.gz 2024-09-12T14:19:29.815 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.101234.log 2024-09-12T14:19:29.816 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.59383.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.59383.log.gz 2024-09-12T14:19:29.816 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.66802.log 2024-09-12T14:19:29.817 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.101234.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.101234.log.gz 2024-09-12T14:19:29.817 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.124496.log 2024-09-12T14:19:29.818 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.66802.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.66802.log.gz 2024-09-12T14:19:29.818 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.66714.log 2024-09-12T14:19:29.819 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.124496.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.124496.log.gz 2024-09-12T14:19:29.819 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.73966.log 2024-09-12T14:19:29.819 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.66714.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.66714.log.gz 2024-09-12T14:19:29.819 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.95755.log 2024-09-12T14:19:29.820 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.73966.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.73966.log.gz 2024-09-12T14:19:29.820 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.51822.log 2024-09-12T14:19:29.821 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.95755.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.95755.log.gz 2024-09-12T14:19:29.821 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.128070.log 2024-09-12T14:19:29.822 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.51822.log: gzip -5 --verbose -- 0.0% /var/log/ceph/cluster2-client.mirror.153463.log -- replaced with /var/log/ceph/cluster1-client.mirror.51822.log.gz 2024-09-12T14:19:29.822 INFO:teuthology.orchestra.run.smithi130.stderr: 2024-09-12T14:19:29.822 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.128070.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.128070.log.gz 2024-09-12T14:19:29.823 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.146110.log 2024-09-12T14:19:29.823 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.153463.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.153463.log.gz 2024-09-12T14:19:29.823 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.83594.log 2024-09-12T14:19:29.824 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.146110.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.146110.log.gz 2024-09-12T14:19:29.824 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.49741.log 2024-09-12T14:19:29.825 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.83594.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.83594.log.gz 2024-09-12T14:19:29.825 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.104723.log 2024-09-12T14:19:29.826 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.49741.log: gzip 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.49741.log.gz -5 2024-09-12T14:19:29.826 INFO:teuthology.orchestra.run.smithi130.stderr: --verbose -- /var/log/ceph/cluster1-client.mirror.132141.log 2024-09-12T14:19:29.826 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.104723.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.104723.log.gz 2024-09-12T14:19:29.827 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.35670.log 2024-09-12T14:19:29.827 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.132141.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.132141.log.gz 2024-09-12T14:19:29.827 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.81973.log 2024-09-12T14:19:29.829 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.35670.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.35670.log.gz 2024-09-12T14:19:29.829 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.85044.log 2024-09-12T14:19:29.829 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.81973.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.81973.log.gz 2024-09-12T14:19:29.829 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.132013.log 2024-09-12T14:19:29.830 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.85044.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.85044.log.gz 2024-09-12T14:19:29.830 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.85380.log 2024-09-12T14:19:29.831 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.132013.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.132013.log.gz 2024-09-12T14:19:29.831 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.114091.log 2024-09-12T14:19:29.831 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.85380.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.85380.log.gz 2024-09-12T14:19:29.831 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.49806.log 2024-09-12T14:19:29.832 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.114091.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.114091.log.gz 2024-09-12T14:19:29.832 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.148821.log 2024-09-12T14:19:29.833 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.49806.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.49806.log.gz 2024-09-12T14:19:29.833 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.116216.log 2024-09-12T14:19:29.834 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.148821.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.148821.log.gz 2024-09-12T14:19:29.834 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.62504.log 2024-09-12T14:19:29.835 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.116216.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.116216.log.gz 2024-09-12T14:19:29.835 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.113816.log 2024-09-12T14:19:29.836 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.62504.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.62504.log.gz 2024-09-12T14:19:29.836 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.51235.log 2024-09-12T14:19:29.836 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.113816.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.113816.log.gz 2024-09-12T14:19:29.837 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.82802.log 2024-09-12T14:19:29.837 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.51235.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.51235.log.gz 2024-09-12T14:19:29.837 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.129523.log 2024-09-12T14:19:29.838 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.82802.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.82802.log.gz 2024-09-12T14:19:29.838 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.77030.log 2024-09-12T14:19:29.839 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.129523.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.129523.log.gz 2024-09-12T14:19:29.839 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.66614.log 2024-09-12T14:19:29.840 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.77030.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.77030.log.gz 2024-09-12T14:19:29.840 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.51405.log 2024-09-12T14:19:29.840 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.66614.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.66614.log.gz 2024-09-12T14:19:29.841 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.67541.log 2024-09-12T14:19:29.842 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.51405.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.51405.log.gz 2024-09-12T14:19:29.842 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.151485.log 2024-09-12T14:19:29.842 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.67541.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.67541.log.gz 2024-09-12T14:19:29.842 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.125111.log 2024-09-12T14:19:29.843 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.151485.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.41229.log 2024-09-12T14:19:29.843 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.151485.log.gz 2024-09-12T14:19:29.843 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.125111.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.125111.log.gz 2024-09-12T14:19:29.844 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.135738.log 2024-09-12T14:19:29.845 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.41229.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.41229.log.gz 2024-09-12T14:19:29.845 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.93497.log 2024-09-12T14:19:29.845 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.135738.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.135738.log.gz 2024-09-12T14:19:29.846 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.71212.log 2024-09-12T14:19:29.846 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.93497.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.93497.log.gz 2024-09-12T14:19:29.846 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.99935.log 2024-09-12T14:19:29.847 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.71212.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.71212.log.gz 2024-09-12T14:19:29.847 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.104556.log 2024-09-12T14:19:29.848 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.99935.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.99935.log.gz 2024-09-12T14:19:29.848 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.79171.log 2024-09-12T14:19:29.848 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.104556.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.104556.log.gz 2024-09-12T14:19:29.849 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.99577.log 2024-09-12T14:19:29.849 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.79171.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.48967.log 0.0% 2024-09-12T14:19:29.850 INFO:teuthology.orchestra.run.smithi130.stderr: -- replaced with /var/log/ceph/cluster1-client.admin.79171.log.gz 2024-09-12T14:19:29.850 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.99577.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.99577.log.gz 2024-09-12T14:19:29.850 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.69956.log 2024-09-12T14:19:29.851 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.48967.log: gzip -5 --verbose -- 0.0% /var/log/ceph/cluster1-client.mirror.52117.log 2024-09-12T14:19:29.851 INFO:teuthology.orchestra.run.smithi130.stderr: -- replaced with /var/log/ceph/cluster1-client.mirror.48967.log.gz 2024-09-12T14:19:29.852 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.69956.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.69956.log.gz 2024-09-12T14:19:29.852 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.43853.log 2024-09-12T14:19:29.852 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.52117.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.52117.log.gz 2024-09-12T14:19:29.853 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.52710.log 2024-09-12T14:19:29.853 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.43853.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.145543.log 2024-09-12T14:19:29.853 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.43853.log.gz 2024-09-12T14:19:29.854 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.52710.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.52710.log.gz 2024-09-12T14:19:29.854 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.47385.log 2024-09-12T14:19:29.855 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.145543.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.62367.log 2024-09-12T14:19:29.855 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.145543.log.gz 2024-09-12T14:19:29.855 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.47385.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.47385.log.gz 2024-09-12T14:19:29.856 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.89773.log 2024-09-12T14:19:29.856 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.62367.log: 0.0%gzip -- replaced with /var/log/ceph/cluster1-client.mirror.62367.log.gz 2024-09-12T14:19:29.856 INFO:teuthology.orchestra.run.smithi130.stderr: -5 --verbose -- /var/log/ceph/cluster1-client.mirror.40433.log 2024-09-12T14:19:29.857 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.89773.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.89773.log.gz 2024-09-12T14:19:29.857 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.93185.log 2024-09-12T14:19:29.858 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.40433.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.40433.log.gz 2024-09-12T14:19:29.858 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.116480.log 2024-09-12T14:19:29.859 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.93185.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.93185.log.gz 2024-09-12T14:19:29.859 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.52281.log 2024-09-12T14:19:29.859 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.116480.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.116480.log.gz 2024-09-12T14:19:29.860 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.116265.log 2024-09-12T14:19:29.860 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.52281.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.52281.log.gz 2024-09-12T14:19:29.860 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.114999.log 2024-09-12T14:19:29.861 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.116265.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.116265.log.gz 2024-09-12T14:19:29.861 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.95429.log 2024-09-12T14:19:29.862 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.114999.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.114999.log.gz 2024-09-12T14:19:29.862 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.107449.log 2024-09-12T14:19:29.862 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.95429.log: 0.0%gzip -- replaced with /var/log/ceph/cluster1-client.mirror.95429.log.gz 2024-09-12T14:19:29.862 INFO:teuthology.orchestra.run.smithi130.stderr: -5 --verbose -- /var/log/ceph/cluster1-client.mirror.75771.log 2024-09-12T14:19:29.863 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.107449.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.107449.log.gz 2024-09-12T14:19:29.863 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.41961.log 2024-09-12T14:19:29.864 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.75771.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.149201.log 2024-09-12T14:19:29.864 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.75771.log.gz 2024-09-12T14:19:29.865 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.41961.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.41961.log.gz 2024-09-12T14:19:29.865 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.50793.log 2024-09-12T14:19:29.865 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.149201.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.104949.log 2024-09-12T14:19:29.866 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.149201.log.gz 2024-09-12T14:19:29.866 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.50793.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.50793.log.gz 2024-09-12T14:19:29.866 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.56454.log 2024-09-12T14:19:29.867 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.104949.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.104949.log.gz 2024-09-12T14:19:29.867 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.45945.log 2024-09-12T14:19:29.868 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.56454.log: gzip 0.0% -5 -- replaced with /var/log/ceph/cluster1-client.admin.56454.log.gz --verbose 2024-09-12T14:19:29.868 INFO:teuthology.orchestra.run.smithi130.stderr: -- /var/log/ceph/cluster1-client.admin.55351.log 2024-09-12T14:19:29.868 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.45945.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.53987.log 2024-09-12T14:19:29.868 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.45945.log.gz 2024-09-12T14:19:29.869 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.55351.log: 0.0%gzip -- replaced with /var/log/ceph/cluster1-client.admin.55351.log.gz -5 2024-09-12T14:19:29.869 INFO:teuthology.orchestra.run.smithi130.stderr: --verbose -- /var/log/ceph/cluster1-client.mirror.143920.log 2024-09-12T14:19:29.870 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.53987.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.53987.log.gz 2024-09-12T14:19:29.870 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.66503.log 2024-09-12T14:19:29.870 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.143920.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.143920.log.gz 2024-09-12T14:19:29.871 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.116609.log 2024-09-12T14:19:29.871 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.66503.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.66170.log 2024-09-12T14:19:29.871 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.66503.log.gz 2024-09-12T14:19:29.872 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.116609.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.116609.log.gz 2024-09-12T14:19:29.872 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.152053.log 2024-09-12T14:19:29.873 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.66170.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.66170.log.gz 2024-09-12T14:19:29.873 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.42699.log 2024-09-12T14:19:29.874 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.152053.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.152053.log.gz 2024-09-12T14:19:29.874 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.141322.log 2024-09-12T14:19:29.874 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.42699.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.42699.log.gz 2024-09-12T14:19:29.874 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.33940.log 2024-09-12T14:19:29.875 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.141322.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.40350.log 2024-09-12T14:19:29.875 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.141322.log.gz 2024-09-12T14:19:29.876 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.33940.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.33940.log.gz 2024-09-12T14:19:29.876 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.121257.log 2024-09-12T14:19:29.877 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.40350.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.59976.log 2024-09-12T14:19:29.877 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.40350.log.gz 2024-09-12T14:19:29.878 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.121257.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.121257.log.gz 2024-09-12T14:19:29.878 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.74702.log 2024-09-12T14:19:29.878 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.59976.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.59976.log.gz 2024-09-12T14:19:29.879 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.106800.log 2024-09-12T14:19:29.879 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.74702.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.74702.log.gz 2024-09-12T14:19:29.879 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.71258.log 2024-09-12T14:19:29.880 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.106800.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.53020.log 2024-09-12T14:19:29.880 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.106800.log.gz 2024-09-12T14:19:29.881 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.71258.log: gzip -5 --verbose 0.0% -- -- replaced with /var/log/ceph/cluster1-client.mirror.71258.log.gz /var/log/ceph/cluster1-client.mirror.100690.log 2024-09-12T14:19:29.881 INFO:teuthology.orchestra.run.smithi130.stderr: 2024-09-12T14:19:29.881 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.53020.log: gzip 0.0% -5 -- replaced with /var/log/ceph/cluster1-client.mirror.53020.log.gz --verbose 2024-09-12T14:19:29.882 INFO:teuthology.orchestra.run.smithi130.stderr: -- /var/log/ceph/cluster1-client.mirror.136250.log 2024-09-12T14:19:29.882 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.100690.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.100690.log.gz 2024-09-12T14:19:29.882 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.103319.log 2024-09-12T14:19:29.883 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.136250.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.136250.log.gz 2024-09-12T14:19:29.883 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.125540.log 2024-09-12T14:19:29.884 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.103319.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.77915.log 2024-09-12T14:19:29.884 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.103319.log.gz 2024-09-12T14:19:29.885 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.125540.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.125540.log.gz 2024-09-12T14:19:29.885 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.148020.log 2024-09-12T14:19:29.885 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.77915.log: 0.0%gzip -- replaced with /var/log/ceph/cluster1-client.mirror.77915.log.gz 2024-09-12T14:19:29.885 INFO:teuthology.orchestra.run.smithi130.stderr: -5 --verbose -- /var/log/ceph/cluster1-client.mirror.97641.log 2024-09-12T14:19:29.886 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.148020.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.148020.log.gz 2024-09-12T14:19:29.886 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.80636.log 2024-09-12T14:19:29.887 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.97641.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.97641.log.gz 2024-09-12T14:19:29.887 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.58861.log 2024-09-12T14:19:29.888 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.80636.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.80636.log.gz 2024-09-12T14:19:29.888 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.65789.log 2024-09-12T14:19:29.888 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.58861.log: 0.0%gzip -- replaced with /var/log/ceph/cluster1-client.admin.58861.log.gz -5 2024-09-12T14:19:29.889 INFO:teuthology.orchestra.run.smithi130.stderr: --verbose -- /var/log/ceph/cluster1-client.mirror.54071.log 2024-09-12T14:19:29.889 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.65789.log: gzip 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.65789.log.gz -5 2024-09-12T14:19:29.889 INFO:teuthology.orchestra.run.smithi130.stderr: --verbose -- /var/log/ceph/cluster2-client.mirror.68707.log 2024-09-12T14:19:29.890 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.54071.log: gzip -5 --verbose 0.0% -- /var/log/ceph/cluster1-client.mirror.142379.log -- replaced with /var/log/ceph/cluster1-client.mirror.54071.log.gz 2024-09-12T14:19:29.890 INFO:teuthology.orchestra.run.smithi130.stderr: 2024-09-12T14:19:29.890 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.68707.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.68707.log.gz 2024-09-12T14:19:29.891 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.113056.log 2024-09-12T14:19:29.891 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.142379.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.99879.log 2024-09-12T14:19:29.891 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.142379.log.gz 2024-09-12T14:19:29.892 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.113056.log: 0.0%gzip -- replaced with /var/log/ceph/cluster1-client.mirror.113056.log.gz 2024-09-12T14:19:29.892 INFO:teuthology.orchestra.run.smithi130.stderr: -5 --verbose -- /var/log/ceph/cluster1-client.mirror.71564.log 2024-09-12T14:19:29.893 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.99879.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.99879.log.gz 2024-09-12T14:19:29.893 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.63528.log 2024-09-12T14:19:29.894 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.71564.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.71564.log.gz 2024-09-12T14:19:29.894 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.55415.log 2024-09-12T14:19:29.894 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.63528.log: 0.0%gzip -- replaced with /var/log/ceph/cluster1-client.mirror.63528.log.gz -5 2024-09-12T14:19:29.894 INFO:teuthology.orchestra.run.smithi130.stderr: --verbose -- /var/log/ceph/cluster1-client.mirror.85525.log 2024-09-12T14:19:29.895 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.55415.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.55415.log.gz 2024-09-12T14:19:29.895 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.82629.log 2024-09-12T14:19:29.896 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.85525.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.85525.log.gz 2024-09-12T14:19:29.896 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.124250.log 2024-09-12T14:19:29.896 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.82629.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.82629.log.gz 2024-09-12T14:19:29.897 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.79655.log 2024-09-12T14:19:29.898 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.124250.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.124250.log.gz 2024-09-12T14:19:29.898 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.82475.log 2024-09-12T14:19:29.898 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.79655.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.79655.log.gz 2024-09-12T14:19:29.899 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.126588.log 2024-09-12T14:19:29.900 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.82475.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.82475.log.gz 2024-09-12T14:19:29.900 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.63987.log 2024-09-12T14:19:29.900 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.126588.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.126588.log.gz 2024-09-12T14:19:29.901 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.76987.log 2024-09-12T14:19:29.901 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.63987.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.63987.log.gz 2024-09-12T14:19:29.901 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.129083.log 2024-09-12T14:19:29.902 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.76987.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.76987.log.gz 2024-09-12T14:19:29.902 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.116961.log 2024-09-12T14:19:29.903 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.129083.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.129083.log.gz 2024-09-12T14:19:29.903 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.108562.log 2024-09-12T14:19:29.904 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.116961.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.126895.log 2024-09-12T14:19:29.904 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.116961.log.gz 2024-09-12T14:19:29.905 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.108562.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.108562.log.gz 2024-09-12T14:19:29.905 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.64616.log 2024-09-12T14:19:29.905 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.126895.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.126895.log.gz 2024-09-12T14:19:29.905 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.112762.log 2024-09-12T14:19:29.906 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.64616.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.64616.log.gz 2024-09-12T14:19:29.906 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.52685.log 2024-09-12T14:19:29.907 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.112762.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.112762.log.gz 2024-09-12T14:19:29.907 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.99095.log 2024-09-12T14:19:29.907 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.52685.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.52685.log.gz 2024-09-12T14:19:29.908 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.110242.log 2024-09-12T14:19:29.908 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.99095.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.99095.log.gz 2024-09-12T14:19:29.908 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.150023.log 2024-09-12T14:19:29.909 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.110242.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.110242.log.gz 2024-09-12T14:19:29.909 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.103913.log 2024-09-12T14:19:29.910 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.150023.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.150023.log.gz 2024-09-12T14:19:29.910 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.111187.log 2024-09-12T14:19:29.911 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.103913.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.103913.log.gz 2024-09-12T14:19:29.911 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.72105.log 2024-09-12T14:19:29.912 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.111187.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.111187.log.gz 2024-09-12T14:19:29.912 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.131617.log 2024-09-12T14:19:29.913 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.72105.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.72105.log.gz 2024-09-12T14:19:29.913 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.95085.log 2024-09-12T14:19:29.913 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.131617.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.131617.log.gz 2024-09-12T14:19:29.914 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.43808.log 2024-09-12T14:19:29.914 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.95085.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.95085.log.gz 2024-09-12T14:19:29.914 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.122748.log 2024-09-12T14:19:29.915 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.43808.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.43808.log.gz 2024-09-12T14:19:29.915 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.45668.log 2024-09-12T14:19:29.916 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.122748.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.122748.log.gz 2024-09-12T14:19:29.916 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.89097.log 2024-09-12T14:19:29.916 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.45668.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.45668.log.gz 2024-09-12T14:19:29.916 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.145292.log 2024-09-12T14:19:29.917 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.89097.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.89097.log.gz 2024-09-12T14:19:29.917 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.144378.log 2024-09-12T14:19:29.918 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.145292.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.145292.log.gz 2024-09-12T14:19:29.918 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.36681.log 2024-09-12T14:19:29.919 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.144378.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.144378.log.gz 2024-09-12T14:19:29.919 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.81563.log 2024-09-12T14:19:29.919 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.36681.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.36681.log.gz 2024-09-12T14:19:29.919 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.81153.log 2024-09-12T14:19:29.920 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.81563.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.81563.log.gz 2024-09-12T14:19:29.920 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.44170.log 2024-09-12T14:19:29.921 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.81153.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.81153.log.gz 2024-09-12T14:19:29.921 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.43226.log 2024-09-12T14:19:29.922 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.44170.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.44170.log.gz 2024-09-12T14:19:29.922 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.86576.log 2024-09-12T14:19:29.923 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.43226.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.43226.log.gz 2024-09-12T14:19:29.923 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.58947.log 2024-09-12T14:19:29.923 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.86576.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.86576.log.gz 2024-09-12T14:19:29.924 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.120150.log 2024-09-12T14:19:29.924 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.58947.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.58947.log.gz 2024-09-12T14:19:29.924 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.77812.log 2024-09-12T14:19:29.925 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.120150.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.120150.log.gz 2024-09-12T14:19:29.925 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.148085.log 2024-09-12T14:19:29.926 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.77812.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.77812.log.gz 2024-09-12T14:19:29.926 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.115936.log 2024-09-12T14:19:29.927 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.148085.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.148085.log.gz 2024-09-12T14:19:29.927 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.89390.log 2024-09-12T14:19:29.927 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.115936.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.115936.log.gz 2024-09-12T14:19:29.927 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.112616.log 2024-09-12T14:19:29.928 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.89390.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.89390.log.gz 2024-09-12T14:19:29.928 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.65499.log 2024-09-12T14:19:29.929 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.112616.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.112616.log.gz 2024-09-12T14:19:29.929 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.86894.log 2024-09-12T14:19:29.930 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.65499.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.65499.log.gz 2024-09-12T14:19:29.930 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.128399.log 2024-09-12T14:19:29.931 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.86894.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.86894.log.gz 2024-09-12T14:19:29.931 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.77873.log 2024-09-12T14:19:29.931 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.128399.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.128399.log.gz 2024-09-12T14:19:29.931 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.135773.log 2024-09-12T14:19:29.932 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.77873.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.77873.log.gz 2024-09-12T14:19:29.932 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.50083.log 2024-09-12T14:19:29.933 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.135773.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.135773.log.gz 2024-09-12T14:19:29.933 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.153390.log 2024-09-12T14:19:29.934 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.50083.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.50083.log.gz 2024-09-12T14:19:29.934 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.39279.log 2024-09-12T14:19:29.935 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.153390.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.153390.log.gz 2024-09-12T14:19:29.935 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.56118.log 2024-09-12T14:19:29.935 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.39279.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.39279.log.gz 2024-09-12T14:19:29.936 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.64617.log 2024-09-12T14:19:29.937 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.56118.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.56118.log.gz 2024-09-12T14:19:29.937 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.34904.log 2024-09-12T14:19:29.937 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.64617.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.64617.log.gz 2024-09-12T14:19:29.938 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.45195.log 2024-09-12T14:19:29.939 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.34904.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.34904.log.gz 2024-09-12T14:19:29.939 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.93704.log 2024-09-12T14:19:29.939 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.45195.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.45195.log.gz 2024-09-12T14:19:29.939 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.146241.log 2024-09-12T14:19:29.939 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.93704.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.93704.log.gz 2024-09-12T14:19:29.940 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.128069.log 2024-09-12T14:19:29.940 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.146241.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.146241.log.gz 2024-09-12T14:19:29.940 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.68546.log 2024-09-12T14:19:29.942 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.128069.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.128069.log.gz 2024-09-12T14:19:29.942 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.147074.log 2024-09-12T14:19:29.942 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.68546.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.68546.log.gz 2024-09-12T14:19:29.942 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.58317.log 2024-09-12T14:19:29.943 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.147074.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.147074.log.gz 2024-09-12T14:19:29.943 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.45866.log 2024-09-12T14:19:29.943 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.58317.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.58317.log.gz 2024-09-12T14:19:29.943 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.154425.log 2024-09-12T14:19:29.944 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.45866.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.45866.log.gz 2024-09-12T14:19:29.944 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.130020.log 2024-09-12T14:19:29.945 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.154425.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.154425.log.gz 2024-09-12T14:19:29.945 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.59320.log 2024-09-12T14:19:29.946 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.130020.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.130020.log.gz 2024-09-12T14:19:29.946 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.76124.log 2024-09-12T14:19:29.946 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.59320.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.59320.log.gz 2024-09-12T14:19:29.947 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.62392.log 2024-09-12T14:19:29.947 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.76124.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.76124.log.gz 2024-09-12T14:19:29.947 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.137315.log 2024-09-12T14:19:29.948 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.62392.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.62392.log.gz 2024-09-12T14:19:29.948 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.54766.log 2024-09-12T14:19:29.949 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.137315.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.137315.log.gz 2024-09-12T14:19:29.949 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.115380.log 2024-09-12T14:19:29.950 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.54766.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.54766.log.gz 2024-09-12T14:19:29.950 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.83808.log 2024-09-12T14:19:29.951 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.115380.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.117635.log 2024-09-12T14:19:29.951 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.115380.log.gz 2024-09-12T14:19:29.952 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.83808.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.92925.log 2024-09-12T14:19:29.952 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.83808.log.gz 2024-09-12T14:19:29.952 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.117635.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.117635.log.gz 2024-09-12T14:19:29.953 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.97160.log 2024-09-12T14:19:29.953 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.92925.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.40679.log 2024-09-12T14:19:29.953 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.92925.log.gz 2024-09-12T14:19:29.954 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.97160.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.97160.log.gz 2024-09-12T14:19:29.954 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.144211.log 2024-09-12T14:19:29.955 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.40679.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.40679.log.gz 2024-09-12T14:19:29.955 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.79598.log 2024-09-12T14:19:29.955 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.144211.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.75331.log 2024-09-12T14:19:29.955 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.144211.log.gz 2024-09-12T14:19:29.956 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.79598.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.79598.log.gz 2024-09-12T14:19:29.956 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.86154.log 2024-09-12T14:19:29.957 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.75331.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.75331.log.gz 2024-09-12T14:19:29.957 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.95611.log 2024-09-12T14:19:29.958 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.86154.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.92362.log 2024-09-12T14:19:29.958 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.86154.log.gz 2024-09-12T14:19:29.959 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.95611.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.95611.log.gz 2024-09-12T14:19:29.959 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.136106.log 2024-09-12T14:19:29.959 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.92362.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.92362.log.gz 2024-09-12T14:19:29.960 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.56272.log 2024-09-12T14:19:29.960 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.136106.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.136106.log.gz 2024-09-12T14:19:29.960 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.70009.log 2024-09-12T14:19:29.961 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.56272.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.56272.log.gz 2024-09-12T14:19:29.961 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.43676.log 2024-09-12T14:19:29.962 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.70009.log: gzip 0.0% -5 --verbose -- replaced with /var/log/ceph/cluster2-client.mirror.70009.log.gz -- 2024-09-12T14:19:29.962 INFO:teuthology.orchestra.run.smithi130.stderr: /var/log/ceph/cluster1-client.mirror.136461.log 2024-09-12T14:19:29.962 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.43676.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.43676.log.gz 2024-09-12T14:19:29.963 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.79780.log 2024-09-12T14:19:29.963 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.136461.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.136461.log.gz 2024-09-12T14:19:29.963 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.50962.log 2024-09-12T14:19:29.964 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.79780.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.79780.log.gz 2024-09-12T14:19:29.964 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.48240.log 2024-09-12T14:19:29.965 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.50962.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.50962.log.gz 2024-09-12T14:19:29.965 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.106490.log 2024-09-12T14:19:29.966 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.48240.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.48240.log.gz 2024-09-12T14:19:29.966 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.48548.log 2024-09-12T14:19:29.966 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.106490.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.106490.log.gz 2024-09-12T14:19:29.966 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.101294.log 2024-09-12T14:19:29.968 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.48548.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.35765.log 2024-09-12T14:19:29.968 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.48548.log.gz 2024-09-12T14:19:29.968 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.101294.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.101294.log.gz 2024-09-12T14:19:29.969 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.105725.log 2024-09-12T14:19:29.969 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.35765.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.35765.log.gz 2024-09-12T14:19:29.969 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.154549.log 2024-09-12T14:19:29.970 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.105725.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.105725.log.gz 2024-09-12T14:19:29.970 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.150995.log 2024-09-12T14:19:29.972 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.154549.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.154549.log.gz 2024-09-12T14:19:29.972 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.139383.log 2024-09-12T14:19:29.972 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.150995.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.150995.log.gz 2024-09-12T14:19:29.972 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.60299.log 2024-09-12T14:19:29.976 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.139383.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.51991.log 2024-09-12T14:19:29.976 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.139383.log.gz 2024-09-12T14:19:29.976 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.60299.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.96277.log 0.0% 2024-09-12T14:19:29.976 INFO:teuthology.orchestra.run.smithi130.stderr: -- replaced with /var/log/ceph/cluster1-client.mirror.60299.log.gz 2024-09-12T14:19:29.976 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.51991.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.51991.log.gz 2024-09-12T14:19:29.977 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.134677.log 2024-09-12T14:19:29.977 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.96277.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.96277.log.gz 2024-09-12T14:19:29.977 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.64784.log 2024-09-12T14:19:29.977 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.134677.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.134677.log.gz 2024-09-12T14:19:29.977 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.123451.log 2024-09-12T14:19:29.980 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.64784.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.51549.log 2024-09-12T14:19:29.980 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.64784.log.gz 2024-09-12T14:19:29.980 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.123451.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.123451.log.gz 2024-09-12T14:19:29.981 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.57716.log 2024-09-12T14:19:29.981 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.51549.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.51549.log.gz 2024-09-12T14:19:29.982 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.132847.log 2024-09-12T14:19:29.982 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.57716.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.57716.log.gz 2024-09-12T14:19:29.982 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.85778.log 2024-09-12T14:19:29.982 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.132847.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.132847.log.gz 2024-09-12T14:19:29.982 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.144268.log 2024-09-12T14:19:29.982 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.85778.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.64491.log 2024-09-12T14:19:29.982 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.85778.log.gz 2024-09-12T14:19:29.983 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.144268.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.144268.log.gz 2024-09-12T14:19:29.984 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.150672.log 2024-09-12T14:19:29.984 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.64491.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.64491.log.gz 2024-09-12T14:19:29.984 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.94623.log 2024-09-12T14:19:29.984 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.150672.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.150672.log.gz 2024-09-12T14:19:29.984 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.34189.log 2024-09-12T14:19:29.984 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.94623.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.108563.log 2024-09-12T14:19:29.984 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.94623.log.gz 2024-09-12T14:19:29.985 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.34189.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.34189.log.gz 2024-09-12T14:19:29.985 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.127967.log 2024-09-12T14:19:29.985 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.108563.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.108563.log.gz 2024-09-12T14:19:29.985 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.119334.log 2024-09-12T14:19:29.986 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.127967.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.127967.log.gz 2024-09-12T14:19:29.986 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.121488.log 2024-09-12T14:19:29.986 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.119334.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.119334.log.gz 2024-09-12T14:19:29.987 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.78512.log 2024-09-12T14:19:29.987 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.121488.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.142970.log 2024-09-12T14:19:29.987 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.121488.log.gz 2024-09-12T14:19:29.988 INFO:teuthology.orchestra.run.smithi060.stderr: 92.5% -- replaced with /var/log/ceph/cluster1.audit.log.gz 2024-09-12T14:19:29.989 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.78512.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.78512.log.gz 2024-09-12T14:19:29.989 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.110622.log 2024-09-12T14:19:29.989 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.142970.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.142970.log.gz 2024-09-12T14:19:29.989 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.85360.log 2024-09-12T14:19:29.990 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.110622.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.110622.log.gz 2024-09-12T14:19:29.990 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.139287.log 2024-09-12T14:19:29.991 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.85360.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.62262.log 2024-09-12T14:19:29.991 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.85360.log.gz 2024-09-12T14:19:29.991 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.139287.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.139287.log.gz 2024-09-12T14:19:29.991 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.64321.log 2024-09-12T14:19:29.992 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.62262.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.62262.log.gz 2024-09-12T14:19:29.992 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.119270.log 2024-09-12T14:19:29.993 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.64321.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.64321.log.gz 2024-09-12T14:19:29.993 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.57017.log 2024-09-12T14:19:29.993 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.119270.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.119270.log.gz 2024-09-12T14:19:29.994 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.75583.log 2024-09-12T14:19:29.994 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.57017.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.148259.log 2024-09-12T14:19:29.994 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.57017.log.gz 2024-09-12T14:19:29.995 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.75583.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.75583.log.gz 2024-09-12T14:19:29.995 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.50846.log 2024-09-12T14:19:29.996 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.148259.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.148259.log.gz 2024-09-12T14:19:29.996 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.118210.log 2024-09-12T14:19:29.997 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.50846.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.50846.log.gz 2024-09-12T14:19:29.997 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.40680.log 2024-09-12T14:19:29.998 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.118210.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.118210.log.gz 2024-09-12T14:19:29.998 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.119851.log 2024-09-12T14:19:29.998 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.40680.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.130000.log 2024-09-12T14:19:29.998 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.40680.log.gz 2024-09-12T14:19:29.999 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.119851.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.119851.log.gz 2024-09-12T14:19:29.999 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.127205.log 2024-09-12T14:19:30.000 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.130000.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.130000.log.gz 2024-09-12T14:19:30.000 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.87287.log 2024-09-12T14:19:30.001 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.127205.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.127205.log.gz 2024-09-12T14:19:30.001 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.73649.log 2024-09-12T14:19:30.001 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.87287.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.87287.log.gz 2024-09-12T14:19:30.001 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.62222.log 2024-09-12T14:19:30.002 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.73649.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.73649.log.gz 2024-09-12T14:19:30.002 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.72915.log 2024-09-12T14:19:30.003 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.62222.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.99448.log 2024-09-12T14:19:30.003 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.62222.log.gz 2024-09-12T14:19:30.004 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.72915.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.72915.log.gz 2024-09-12T14:19:30.004 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.39697.log 2024-09-12T14:19:30.004 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.99448.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.99448.log.gz 2024-09-12T14:19:30.004 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.123576.log 2024-09-12T14:19:30.005 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.39697.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.41336.log 2024-09-12T14:19:30.005 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.39697.log.gz 2024-09-12T14:19:30.006 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.123576.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.123576.log.gz 2024-09-12T14:19:30.006 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.120859.log 2024-09-12T14:19:30.007 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.41336.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.41336.log.gz 2024-09-12T14:19:30.007 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.71517.log 2024-09-12T14:19:30.008 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.120859.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.120859.log.gz 2024-09-12T14:19:30.008 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.103099.log 2024-09-12T14:19:30.008 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.71517.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.83987.log 2024-09-12T14:19:30.008 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.71517.log.gz 2024-09-12T14:19:30.009 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.103099.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.103099.log.gz 2024-09-12T14:19:30.009 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.140815.log 2024-09-12T14:19:30.010 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.83987.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.83987.log.gz 2024-09-12T14:19:30.010 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.91231.log 2024-09-12T14:19:30.011 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.140815.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.140815.log.gz 2024-09-12T14:19:30.011 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.138135.log 2024-09-12T14:19:30.012 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.91231.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.91231.log.gz 2024-09-12T14:19:30.012 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.59142.log 2024-09-12T14:19:30.012 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.138135.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.138135.log.gz 2024-09-12T14:19:30.013 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.41650.log 2024-09-12T14:19:30.013 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.59142.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.59142.log.gz 2024-09-12T14:19:30.013 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.37449.log 2024-09-12T14:19:30.014 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.41650.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.95301.log 2024-09-12T14:19:30.014 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.41650.log.gz 2024-09-12T14:19:30.015 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.37449.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.37449.log.gz 2024-09-12T14:19:30.015 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.81946.log 2024-09-12T14:19:30.016 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.95301.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.117837.log 2024-09-12T14:19:30.016 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.95301.log.gz 2024-09-12T14:19:30.017 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.81946.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.81946.log.gz 2024-09-12T14:19:30.017 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.101403.log 2024-09-12T14:19:30.017 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.117837.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.117837.log.gz 2024-09-12T14:19:30.017 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.123219.log 2024-09-12T14:19:30.018 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.101403.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.70948.log 2024-09-12T14:19:30.018 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.101403.log.gz 2024-09-12T14:19:30.019 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.123219.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.123219.log.gz 2024-09-12T14:19:30.019 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.109715.log 2024-09-12T14:19:30.020 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.70948.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.70948.log.gz 2024-09-12T14:19:30.020 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.119153.log 2024-09-12T14:19:30.020 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.109715.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.109715.log.gz 2024-09-12T14:19:30.020 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.146448.log 2024-09-12T14:19:30.021 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.119153.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.63145.log 2024-09-12T14:19:30.021 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.119153.log.gz 2024-09-12T14:19:30.022 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.146448.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.146448.log.gz 2024-09-12T14:19:30.022 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.95169.log 2024-09-12T14:19:30.023 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.63145.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.63145.log.gz 2024-09-12T14:19:30.023 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.140380.log 2024-09-12T14:19:30.023 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.95169.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.95169.log.gz 2024-09-12T14:19:30.024 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.129057.log 2024-09-12T14:19:30.025 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.140380.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.136151.log 2024-09-12T14:19:30.025 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.140380.log.gz 2024-09-12T14:19:30.025 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.129057.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.129057.log.gz 2024-09-12T14:19:30.025 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.41801.log 2024-09-12T14:19:30.026 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.136151.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.136151.log.gz 2024-09-12T14:19:30.026 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.43412.log 2024-09-12T14:19:30.027 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.41801.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.62807.log 0.0% 2024-09-12T14:19:30.027 INFO:teuthology.orchestra.run.smithi130.stderr: -- replaced with /var/log/ceph/cluster1-client.mirror.41801.log.gz 2024-09-12T14:19:30.028 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.43412.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.43412.log.gz 2024-09-12T14:19:30.028 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.37997.log 2024-09-12T14:19:30.028 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.62807.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.62807.log.gz 2024-09-12T14:19:30.028 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.152049.log 2024-09-12T14:19:30.029 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.37997.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.68473.log 2024-09-12T14:19:30.029 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.37997.log.gz 2024-09-12T14:19:30.030 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.152049.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.152049.log.gz 2024-09-12T14:19:30.030 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.144065.log 2024-09-12T14:19:30.031 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.68473.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.68473.log.gz 2024-09-12T14:19:30.031 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.109257.log 2024-09-12T14:19:30.031 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.144065.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.144065.log.gz 2024-09-12T14:19:30.032 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.84163.log 2024-09-12T14:19:30.032 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.109257.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.129316.log 2024-09-12T14:19:30.032 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.109257.log.gz 2024-09-12T14:19:30.033 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.84163.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.84163.log.gz 2024-09-12T14:19:30.033 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.45801.log 2024-09-12T14:19:30.034 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.129316.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.129316.log.gz 2024-09-12T14:19:30.034 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.148653.log 2024-09-12T14:19:30.035 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.45801.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.45801.log.gz 2024-09-12T14:19:30.035 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.135350.log 2024-09-12T14:19:30.035 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.148653.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.148653.log.gz 2024-09-12T14:19:30.036 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.71563.log 2024-09-12T14:19:30.037 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.135350.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.111861.log 2024-09-12T14:19:30.037 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.135350.log.gz 2024-09-12T14:19:30.037 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.71563.log: gzip 0.0% -5 --verbose -- replaced with /var/log/ceph/cluster1-client.mirror.71563.log.gz -- 2024-09-12T14:19:30.037 INFO:teuthology.orchestra.run.smithi130.stderr: /var/log/ceph/cluster1-client.admin.67365.log 2024-09-12T14:19:30.038 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.111861.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.111861.log.gz 2024-09-12T14:19:30.038 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.125452.log 2024-09-12T14:19:30.039 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.67365.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.67365.log.gz 2024-09-12T14:19:30.039 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.43758.log 2024-09-12T14:19:30.039 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.125452.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.125452.log.gz 2024-09-12T14:19:30.039 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.127426.log 2024-09-12T14:19:30.040 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.43758.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.93342.log 2024-09-12T14:19:30.040 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.43758.log.gz 2024-09-12T14:19:30.041 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.127426.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.127426.log.gz 2024-09-12T14:19:30.041 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.63649.log 2024-09-12T14:19:30.042 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.93342.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.93342.log.gz 2024-09-12T14:19:30.042 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.89013.log 2024-09-12T14:19:30.043 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.63649.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.63649.log.gz 2024-09-12T14:19:30.043 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.67961.log 2024-09-12T14:19:30.043 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.89013.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.89013.log.gz 2024-09-12T14:19:30.043 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.141691.log 2024-09-12T14:19:30.044 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.67961.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.67961.log.gz 2024-09-12T14:19:30.044 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.148291.log 2024-09-12T14:19:30.045 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.141691.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.142882.log 2024-09-12T14:19:30.045 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.141691.log.gz 2024-09-12T14:19:30.046 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.148291.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.148291.log.gz 2024-09-12T14:19:30.046 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.50186.log 2024-09-12T14:19:30.046 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.142882.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.142882.log.gz 2024-09-12T14:19:30.047 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.105477.log 2024-09-12T14:19:30.047 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.50186.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.50186.log.gz 2024-09-12T14:19:30.047 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.59820.log 2024-09-12T14:19:30.048 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.105477.log: gzip -5 --verbose 0.0% -- /var/log/ceph/cluster1-client.mirror.114108.log -- replaced with /var/log/ceph/cluster1-client.mirror.105477.log.gz 2024-09-12T14:19:30.048 INFO:teuthology.orchestra.run.smithi130.stderr: 2024-09-12T14:19:30.049 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.59820.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.59820.log.gz 2024-09-12T14:19:30.049 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.64522.log 2024-09-12T14:19:30.050 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.114108.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.114108.log.gz 2024-09-12T14:19:30.050 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.63188.log 2024-09-12T14:19:30.050 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.64522.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.54155.log 2024-09-12T14:19:30.051 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.64522.log.gz 2024-09-12T14:19:30.051 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.63188.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.63188.log.gz 2024-09-12T14:19:30.051 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.147473.log 2024-09-12T14:19:30.052 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.54155.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.54155.log.gz 2024-09-12T14:19:30.052 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.52774.log 2024-09-12T14:19:30.053 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.147473.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.68810.log 2024-09-12T14:19:30.053 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.147473.log.gz 2024-09-12T14:19:30.054 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.52774.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.52774.log.gz 2024-09-12T14:19:30.054 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.59490.log 2024-09-12T14:19:30.054 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.68810.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.68810.log.gz 2024-09-12T14:19:30.055 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.129249.log 2024-09-12T14:19:30.055 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.59490.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.59490.log.gz 2024-09-12T14:19:30.055 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.57603.log 2024-09-12T14:19:30.056 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.129249.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.151877.log 2024-09-12T14:19:30.056 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.129249.log.gz 2024-09-12T14:19:30.057 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.57603.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.57603.log.gz 2024-09-12T14:19:30.057 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.104320.log 2024-09-12T14:19:30.057 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.151877.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.151877.log.gz 2024-09-12T14:19:30.058 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.99007.log 2024-09-12T14:19:30.058 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.104320.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.104320.log.gz 2024-09-12T14:19:30.058 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.132199.log 2024-09-12T14:19:30.059 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.99007.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.99007.log.gz 2024-09-12T14:19:30.059 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.98818.log 2024-09-12T14:19:30.060 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.132199.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.132199.log.gz 2024-09-12T14:19:30.060 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.97116.log 2024-09-12T14:19:30.061 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.98818.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.83747.log 2024-09-12T14:19:30.061 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.98818.log.gz 2024-09-12T14:19:30.061 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.97116.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.97116.log.gz 2024-09-12T14:19:30.062 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.128680.log 2024-09-12T14:19:30.062 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.83747.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.83747.log.gz 2024-09-12T14:19:30.062 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.93172.log 2024-09-12T14:19:30.063 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.128680.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.128680.log.gz 2024-09-12T14:19:30.063 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.45971.log 2024-09-12T14:19:30.064 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.93172.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.118091.log 2024-09-12T14:19:30.064 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.93172.log.gz 2024-09-12T14:19:30.065 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.45971.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.45971.log.gz 2024-09-12T14:19:30.065 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.139126.log 2024-09-12T14:19:30.066 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.118091.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.118091.log.gz 2024-09-12T14:19:30.066 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.102747.log 2024-09-12T14:19:30.066 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.139126.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.45010.log 2024-09-12T14:19:30.066 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.139126.log.gz 2024-09-12T14:19:30.067 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.102747.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.102747.log.gz 2024-09-12T14:19:30.067 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.86703.log 2024-09-12T14:19:30.068 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.45010.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.45010.log.gz 2024-09-12T14:19:30.068 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.56091.log 2024-09-12T14:19:30.068 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.86703.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.86703.log.gz 2024-09-12T14:19:30.069 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.47433.log 2024-09-12T14:19:30.069 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.56091.log: gzip 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.56091.log.gz -5 --verbose 2024-09-12T14:19:30.070 INFO:teuthology.orchestra.run.smithi130.stderr: -- /var/log/ceph/cluster1-client.mirror.130754.log 2024-09-12T14:19:30.070 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.47433.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.47433.log.gz 2024-09-12T14:19:30.071 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.137898.log 2024-09-12T14:19:30.071 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.130754.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.130754.log.gz 2024-09-12T14:19:30.071 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.70412.log 2024-09-12T14:19:30.072 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.137898.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.52518.log 2024-09-12T14:19:30.072 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.137898.log.gz 2024-09-12T14:19:30.073 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.70412.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.70412.log.gz 2024-09-12T14:19:30.073 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.69602.log 2024-09-12T14:19:30.073 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.52518.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.52518.log.gz 2024-09-12T14:19:30.073 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.120397.log 2024-09-12T14:19:30.074 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.69602.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.134785.log 2024-09-12T14:19:30.074 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.69602.log.gz 2024-09-12T14:19:30.075 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.120397.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.120397.log.gz 2024-09-12T14:19:30.075 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.47716.log 2024-09-12T14:19:30.076 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.134785.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.134785.log.gz 2024-09-12T14:19:30.076 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.86324.log 2024-09-12T14:19:30.077 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.47716.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.47716.log.gz 2024-09-12T14:19:30.077 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.101466.log 2024-09-12T14:19:30.077 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.86324.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.86324.log.gz 2024-09-12T14:19:30.077 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.76678.log 2024-09-12T14:19:30.078 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.101466.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.135774.log 2024-09-12T14:19:30.078 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.101466.log.gz 2024-09-12T14:19:30.079 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.76678.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.76678.log.gz 2024-09-12T14:19:30.079 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.60902.log 2024-09-12T14:19:30.080 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.135774.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.135774.log.gz 2024-09-12T14:19:30.080 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.120544.log 2024-09-12T14:19:30.081 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.60902.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.60902.log.gz 2024-09-12T14:19:30.081 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.125975.log 2024-09-12T14:19:30.081 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.120544.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.115581.log 2024-09-12T14:19:30.081 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.120544.log.gz 2024-09-12T14:19:30.082 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.125975.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.125975.log.gz 2024-09-12T14:19:30.082 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.94801.log 2024-09-12T14:19:30.083 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.115581.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.115581.log.gz 2024-09-12T14:19:30.083 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.147498.log 2024-09-12T14:19:30.083 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.94801.log: gzip -5 --verbose -- 0.0% /var/log/ceph/cluster1-client.admin.94487.log 2024-09-12T14:19:30.083 INFO:teuthology.orchestra.run.smithi130.stderr: -- replaced with /var/log/ceph/cluster1-client.mirror.94801.log.gz 2024-09-12T14:19:30.084 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.147498.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.147498.log.gz 2024-09-12T14:19:30.084 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.50077.log 2024-09-12T14:19:30.085 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.94487.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.94487.log.gz 2024-09-12T14:19:30.085 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.68055.log 2024-09-12T14:19:30.086 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.50077.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.50077.log.gz 2024-09-12T14:19:30.086 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.144934.log 2024-09-12T14:19:30.087 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.68055.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.68055.log.gz 2024-09-12T14:19:30.087 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.128185.log 2024-09-12T14:19:30.087 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.144934.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.144934.log.gz 2024-09-12T14:19:30.088 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.54301.log 2024-09-12T14:19:30.088 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.128185.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.128185.log.gz 2024-09-12T14:19:30.088 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.42930.log 2024-09-12T14:19:30.089 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.54301.log: gzip 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.54301.log.gz 2024-09-12T14:19:30.089 INFO:teuthology.orchestra.run.smithi130.stderr: -5 --verbose -- /var/log/ceph/cluster1-client.admin.43443.log 2024-09-12T14:19:30.090 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.42930.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.75162.log 2024-09-12T14:19:30.090 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.42930.log.gz 2024-09-12T14:19:30.091 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.43443.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.43443.log.gz 2024-09-12T14:19:30.091 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.143470.log 2024-09-12T14:19:30.091 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.75162.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.75162.log.gz 2024-09-12T14:19:30.092 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.115432.log 2024-09-12T14:19:30.092 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.143470.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.143470.log.gz 2024-09-12T14:19:30.092 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.133323.log 2024-09-12T14:19:30.093 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.115432.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.115432.log.gz 2024-09-12T14:19:30.093 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.53696.log 2024-09-12T14:19:30.094 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.133323.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.133323.log.gz 2024-09-12T14:19:30.094 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.70506.log 2024-09-12T14:19:30.095 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.53696.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.53696.log.gz 2024-09-12T14:19:30.095 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.42721.log 2024-09-12T14:19:30.095 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.70506.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.70506.log.gz 2024-09-12T14:19:30.096 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.106254.log 2024-09-12T14:19:30.096 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.42721.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.42721.log.gz 2024-09-12T14:19:30.096 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.75480.log 2024-09-12T14:19:30.097 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.106254.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.106254.log.gz 2024-09-12T14:19:30.097 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.60190.log 2024-09-12T14:19:30.098 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.75480.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.75480.log.gz 2024-09-12T14:19:30.098 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.118525.log 2024-09-12T14:19:30.099 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.60190.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.60190.log.gz 2024-09-12T14:19:30.099 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.136042.log 2024-09-12T14:19:30.099 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.118525.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.118525.log.gz 2024-09-12T14:19:30.099 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.53231.log 2024-09-12T14:19:30.100 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.136042.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.136042.log.gz 2024-09-12T14:19:30.100 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.99932.log 2024-09-12T14:19:30.101 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.53231.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.53231.log.gz 2024-09-12T14:19:30.101 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.124334.log 2024-09-12T14:19:30.102 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.99932.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.99932.log.gz 2024-09-12T14:19:30.102 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.64872.log 2024-09-12T14:19:30.103 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.124334.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.124334.log.gz 2024-09-12T14:19:30.103 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.66870.log 2024-09-12T14:19:30.104 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.64872.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.64872.log.gz 2024-09-12T14:19:30.104 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.82739.log 2024-09-12T14:19:30.104 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.66870.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.66870.log.gz 2024-09-12T14:19:30.104 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.72170.log 2024-09-12T14:19:30.105 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.82739.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.82739.log.gz 2024-09-12T14:19:30.105 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.136363.log 2024-09-12T14:19:30.106 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.72170.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.72170.log.gz 2024-09-12T14:19:30.106 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.59452.log 2024-09-12T14:19:30.107 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.136363.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.136363.log.gz 2024-09-12T14:19:30.107 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.50038.log 2024-09-12T14:19:30.107 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.59452.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.59452.log.gz 2024-09-12T14:19:30.108 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.102133.log 2024-09-12T14:19:30.108 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.50038.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.50038.log.gz 2024-09-12T14:19:30.108 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.90381.log 2024-09-12T14:19:30.109 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.102133.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.102133.log.gz 2024-09-12T14:19:30.109 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.137454.log 2024-09-12T14:19:30.110 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.90381.log: gzip 0.0% -5 -- replaced with /var/log/ceph/cluster1-client.mirror.90381.log.gz 2024-09-12T14:19:30.110 INFO:teuthology.orchestra.run.smithi130.stderr: --verbose -- /var/log/ceph/cluster1-client.mirror.77786.log 2024-09-12T14:19:30.111 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.137454.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.137454.log.gz 2024-09-12T14:19:30.111 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.37593.log 2024-09-12T14:19:30.112 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.77786.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.77786.log.gz 2024-09-12T14:19:30.112 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.133523.log 2024-09-12T14:19:30.112 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.37593.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.37593.log.gz 2024-09-12T14:19:30.112 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.110492.log 2024-09-12T14:19:30.113 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.133523.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.133523.log.gz 2024-09-12T14:19:30.113 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.36155.log 2024-09-12T14:19:30.114 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.110492.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.110492.log.gz 2024-09-12T14:19:30.114 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.47870.log 2024-09-12T14:19:30.115 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.36155.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.36155.log.gz 2024-09-12T14:19:30.115 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.77366.log 2024-09-12T14:19:30.116 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.47870.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.47870.log.gz 2024-09-12T14:19:30.116 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.99407.log 2024-09-12T14:19:30.116 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.77366.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.77366.log.gz 2024-09-12T14:19:30.116 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.133944.log 2024-09-12T14:19:30.117 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.99407.log: gzip -5 --verbose 0.0% -- -- replaced with /var/log/ceph/cluster1-client.mirror.99407.log.gz 2024-09-12T14:19:30.117 INFO:teuthology.orchestra.run.smithi130.stderr: /var/log/ceph/cluster1-client.admin.151392.log 2024-09-12T14:19:30.118 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.133944.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.133944.log.gz 2024-09-12T14:19:30.118 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.53891.log 2024-09-12T14:19:30.119 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.151392.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.151392.log.gz 2024-09-12T14:19:30.119 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.62224.log 2024-09-12T14:19:30.120 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.53891.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.53891.log.gz 2024-09-12T14:19:30.120 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.88592.log 2024-09-12T14:19:30.120 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.62224.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.62224.log.gz 2024-09-12T14:19:30.121 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.101086.log 2024-09-12T14:19:30.121 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.88592.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.88592.log.gz 2024-09-12T14:19:30.121 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.65292.log 2024-09-12T14:19:30.122 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.101086.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.101086.log.gz 2024-09-12T14:19:30.122 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.117106.log 2024-09-12T14:19:30.123 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.65292.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.65292.log.gz 2024-09-12T14:19:30.123 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.114191.log 2024-09-12T14:19:30.124 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.117106.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.117106.log.gz 2024-09-12T14:19:30.124 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.107532.log 2024-09-12T14:19:30.124 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.114191.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.114191.log.gz 2024-09-12T14:19:30.124 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.133943.log 2024-09-12T14:19:30.125 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.107532.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.107532.log.gz 2024-09-12T14:19:30.125 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.71364.log 2024-09-12T14:19:30.126 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.133943.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.133943.log.gz 2024-09-12T14:19:30.126 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.62895.log 2024-09-12T14:19:30.127 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.71364.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.71364.log.gz 2024-09-12T14:19:30.127 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.97536.log 2024-09-12T14:19:30.128 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.62895.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.62895.log.gz 2024-09-12T14:19:30.128 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.79189.log 2024-09-12T14:19:30.128 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.97536.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.97536.log.gz 2024-09-12T14:19:30.129 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.41987.log 2024-09-12T14:19:30.129 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.79189.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.79189.log.gz 2024-09-12T14:19:30.129 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.124457.log 2024-09-12T14:19:30.130 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.41987.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.41987.log.gz 2024-09-12T14:19:30.130 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.75518.log 2024-09-12T14:19:30.131 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.124457.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.124457.log.gz 2024-09-12T14:19:30.131 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.62560.log 2024-09-12T14:19:30.132 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.75518.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.75518.log.gz 2024-09-12T14:19:30.132 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.110366.log 2024-09-12T14:19:30.132 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.62560.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.62560.log.gz 2024-09-12T14:19:30.133 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.72603.log 2024-09-12T14:19:30.133 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.110366.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.110366.log.gz 2024-09-12T14:19:30.133 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.37346.log 2024-09-12T14:19:30.134 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.72603.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.72603.log.gz 2024-09-12T14:19:30.134 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.64452.log 2024-09-12T14:19:30.135 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.37346.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.37346.log.gz 2024-09-12T14:19:30.135 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.76916.log 2024-09-12T14:19:30.136 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.64452.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.64452.log.gz 2024-09-12T14:19:30.136 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.65732.log 2024-09-12T14:19:30.137 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.76916.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.76916.log.gz 2024-09-12T14:19:30.137 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.117863.log 2024-09-12T14:19:30.137 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.65732.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.65732.log.gz 2024-09-12T14:19:30.137 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.152061.log 2024-09-12T14:19:30.138 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.117863.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.117863.log.gz 2024-09-12T14:19:30.138 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.95345.log 2024-09-12T14:19:30.139 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.152061.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.152061.log.gz 2024-09-12T14:19:30.139 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.50082.log 2024-09-12T14:19:30.140 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.95345.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.95345.log.gz 2024-09-12T14:19:30.140 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.38581.log 2024-09-12T14:19:30.141 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.50082.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.50082.log.gz 2024-09-12T14:19:30.141 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.107652.log 2024-09-12T14:19:30.141 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.38581.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.38581.log.gz 2024-09-12T14:19:30.141 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.44002.log 2024-09-12T14:19:30.142 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.107652.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.107652.log.gz 2024-09-12T14:19:30.142 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.72401.log 2024-09-12T14:19:30.143 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.44002.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.44002.log.gz 2024-09-12T14:19:30.143 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.91319.log 2024-09-12T14:19:30.144 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.72401.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.72401.log.gz 2024-09-12T14:19:30.144 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.141281.log 2024-09-12T14:19:30.145 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.91319.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.91319.log.gz 2024-09-12T14:19:30.145 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.64703.log 2024-09-12T14:19:30.146 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.141281.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.141281.log.gz 2024-09-12T14:19:30.146 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.104835.log 2024-09-12T14:19:30.146 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.64703.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.64703.log.gz 2024-09-12T14:19:30.146 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.46998.log 2024-09-12T14:19:30.147 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.104835.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.104835.log.gz 2024-09-12T14:19:30.147 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.129959.log 2024-09-12T14:19:30.148 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.46998.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.46998.log.gz 2024-09-12T14:19:30.148 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.133881.log 2024-09-12T14:19:30.148 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.129959.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.129959.log.gz 2024-09-12T14:19:30.149 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.58109.log 2024-09-12T14:19:30.149 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.133881.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.133881.log.gz 2024-09-12T14:19:30.149 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.108542.log 2024-09-12T14:19:30.150 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.58109.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.58109.log.gz 2024-09-12T14:19:30.150 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.54888.log 2024-09-12T14:19:30.151 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.108542.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.108542.log.gz 2024-09-12T14:19:30.151 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.140939.log 2024-09-12T14:19:30.152 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.54888.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.54888.log.gz 2024-09-12T14:19:30.152 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.53397.log 2024-09-12T14:19:30.153 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.140939.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.140939.log.gz 2024-09-12T14:19:30.153 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.117951.log 2024-09-12T14:19:30.153 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.53397.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.53397.log.gz 2024-09-12T14:19:30.153 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.115064.log 2024-09-12T14:19:30.154 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.117951.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.117951.log.gz 2024-09-12T14:19:30.154 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.36527.log 2024-09-12T14:19:30.155 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.115064.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.115064.log.gz 2024-09-12T14:19:30.155 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.151853.log 2024-09-12T14:19:30.156 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.36527.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.36527.log.gz 2024-09-12T14:19:30.156 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.74469.log 2024-09-12T14:19:30.157 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.151853.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.151853.log.gz 2024-09-12T14:19:30.157 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.43091.log 2024-09-12T14:19:30.157 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.74469.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.74469.log.gz 2024-09-12T14:19:30.158 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.41248.log 2024-09-12T14:19:30.158 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.43091.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.43091.log.gz 2024-09-12T14:19:30.159 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.144463.log 2024-09-12T14:19:30.159 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.41248.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.41248.log.gz 2024-09-12T14:19:30.159 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.127427.log 2024-09-12T14:19:30.160 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.144463.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.144463.log.gz 2024-09-12T14:19:30.160 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.99718.log 2024-09-12T14:19:30.161 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.127427.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.127427.log.gz 2024-09-12T14:19:30.161 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.99554.log 2024-09-12T14:19:30.162 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.99718.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.99718.log.gz 2024-09-12T14:19:30.162 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.72243.log 2024-09-12T14:19:30.162 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.99554.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.99554.log.gz 2024-09-12T14:19:30.162 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.145291.log 2024-09-12T14:19:30.163 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.72243.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.72243.log.gz 2024-09-12T14:19:30.163 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.37552.log 2024-09-12T14:19:30.164 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.145291.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.145291.log.gz 2024-09-12T14:19:30.164 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.100519.log 2024-09-12T14:19:30.165 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.37552.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.37552.log.gz 2024-09-12T14:19:30.165 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.64951.log 2024-09-12T14:19:30.166 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.100519.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.100519.log.gz 2024-09-12T14:19:30.166 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.53694.log 2024-09-12T14:19:30.167 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.64951.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.64951.log.gz 2024-09-12T14:19:30.167 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.136549.log 2024-09-12T14:19:30.167 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.53694.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.53694.log.gz 2024-09-12T14:19:30.167 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.105228.log 2024-09-12T14:19:30.168 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.136549.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.136549.log.gz 2024-09-12T14:19:30.168 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.114601.log 2024-09-12T14:19:30.169 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.105228.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.105228.log.gz 2024-09-12T14:19:30.169 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.129979.log 2024-09-12T14:19:30.170 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.114601.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.114601.log.gz 2024-09-12T14:19:30.170 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.95961.log 2024-09-12T14:19:30.170 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.129979.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.129979.log.gz 2024-09-12T14:19:30.170 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.118892.log 2024-09-12T14:19:30.171 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.95961.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.95961.log.gz 2024-09-12T14:19:30.171 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.73312.log 2024-09-12T14:19:30.172 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.118892.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.118892.log.gz 2024-09-12T14:19:30.172 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.48907.log 2024-09-12T14:19:30.173 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.73312.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.73312.log.gz 2024-09-12T14:19:30.173 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.117012.log 2024-09-12T14:19:30.174 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.48907.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.48907.log.gz 2024-09-12T14:19:30.174 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.60523.log 2024-09-12T14:19:30.175 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.117012.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.117012.log.gz 2024-09-12T14:19:30.175 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.114009.log 2024-09-12T14:19:30.175 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.60523.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.60523.log.gz 2024-09-12T14:19:30.175 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.140007.log 2024-09-12T14:19:30.176 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.114009.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.114009.log.gz 2024-09-12T14:19:30.176 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.107950.log 2024-09-12T14:19:30.177 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.140007.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.140007.log.gz 2024-09-12T14:19:30.177 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.84205.log 2024-09-12T14:19:30.177 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.107950.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.107950.log.gz 2024-09-12T14:19:30.178 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.92466.log 2024-09-12T14:19:30.178 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.84205.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.84205.log.gz 2024-09-12T14:19:30.178 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.58606.log 2024-09-12T14:19:30.179 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.92466.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.92466.log.gz 2024-09-12T14:19:30.179 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.60522.log 2024-09-12T14:19:30.180 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.58606.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.58606.log.gz 2024-09-12T14:19:30.180 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.135895.log 2024-09-12T14:19:30.181 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.60522.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.60522.log.gz 2024-09-12T14:19:30.181 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.153805.log 2024-09-12T14:19:30.181 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.135895.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.135895.log.gz 2024-09-12T14:19:30.182 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.83350.log 2024-09-12T14:19:30.182 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.153805.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.153805.log.gz 2024-09-12T14:19:30.183 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.79675.log 2024-09-12T14:19:30.183 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.83350.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.83350.log.gz 2024-09-12T14:19:30.183 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.147036.log 2024-09-12T14:19:30.184 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.79675.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.79675.log.gz 2024-09-12T14:19:30.184 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.89054.log 2024-09-12T14:19:30.185 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.147036.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.147036.log.gz 2024-09-12T14:19:30.185 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.45551.log 2024-09-12T14:19:30.186 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.89054.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.89054.log.gz 2024-09-12T14:19:30.186 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.130306.log 2024-09-12T14:19:30.187 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.45551.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.45551.log.gz 2024-09-12T14:19:30.187 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.149156.log 2024-09-12T14:19:30.188 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.130306.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.130306.log.gz 2024-09-12T14:19:30.188 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.124922.log 2024-09-12T14:19:30.188 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.149156.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.149156.log.gz 2024-09-12T14:19:30.188 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.34788.log 2024-09-12T14:19:30.189 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.124922.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.124922.log.gz 2024-09-12T14:19:30.189 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.130114.log 2024-09-12T14:19:30.190 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.34788.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.34788.log.gz 2024-09-12T14:19:30.190 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.91079.log 2024-09-12T14:19:30.190 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.130114.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.130114.log.gz 2024-09-12T14:19:30.190 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.79124.log 2024-09-12T14:19:30.191 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.91079.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.91079.log.gz 2024-09-12T14:19:30.191 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.68331.log 2024-09-12T14:19:30.192 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.79124.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.79124.log.gz 2024-09-12T14:19:30.192 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.100099.log 2024-09-12T14:19:30.192 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.68331.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.68331.log.gz 2024-09-12T14:19:30.193 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.127139.log 2024-09-12T14:19:30.194 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.100099.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.100099.log.gz 2024-09-12T14:19:30.194 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.38227.log 2024-09-12T14:19:30.194 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.127139.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.127139.log.gz 2024-09-12T14:19:30.194 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.112176.log 2024-09-12T14:19:30.195 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.38227.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.38227.log.gz 2024-09-12T14:19:30.195 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.119686.log 2024-09-12T14:19:30.196 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.112176.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.112176.log.gz 2024-09-12T14:19:30.196 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.75163.log 2024-09-12T14:19:30.197 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.119686.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.119686.log.gz 2024-09-12T14:19:30.197 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.109486.log 2024-09-12T14:19:30.197 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.75163.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.75163.log.gz 2024-09-12T14:19:30.198 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.126765.log 2024-09-12T14:19:30.198 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.109486.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.109486.log.gz 2024-09-12T14:19:30.198 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.116262.log 2024-09-12T14:19:30.199 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.126765.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.126765.log.gz 2024-09-12T14:19:30.199 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.144722.log 2024-09-12T14:19:30.200 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.116262.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.116262.log.gz 2024-09-12T14:19:30.200 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.97812.log 2024-09-12T14:19:30.200 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.144722.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.144722.log.gz 2024-09-12T14:19:30.201 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.77260.log 2024-09-12T14:19:30.202 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.97812.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.97812.log.gz 2024-09-12T14:19:30.202 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.116880.log 2024-09-12T14:19:30.202 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.77260.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.77260.log.gz 2024-09-12T14:19:30.202 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.106669.log 2024-09-12T14:19:30.203 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.116880.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.116880.log.gz 2024-09-12T14:19:30.203 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.142923.log 2024-09-12T14:19:30.204 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.106669.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.106669.log.gz 2024-09-12T14:19:30.204 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.73607.log 2024-09-12T14:19:30.205 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.142923.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.142923.log.gz 2024-09-12T14:19:30.205 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.58405.log 2024-09-12T14:19:30.206 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.73607.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.73607.log.gz 2024-09-12T14:19:30.206 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.110853.log 2024-09-12T14:19:30.206 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.58405.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.58405.log.gz 2024-09-12T14:19:30.207 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.142512.log 2024-09-12T14:19:30.207 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.110853.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.110853.log.gz 2024-09-12T14:19:30.208 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.47803.log 2024-09-12T14:19:30.208 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.142512.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.142512.log.gz 2024-09-12T14:19:30.208 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.92377.log 2024-09-12T14:19:30.209 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.47803.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.47803.log.gz 2024-09-12T14:19:30.209 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.115729.log 2024-09-12T14:19:30.210 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.92377.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.92377.log.gz 2024-09-12T14:19:30.210 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.122267.log 2024-09-12T14:19:30.211 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.115729.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.115729.log.gz 2024-09-12T14:19:30.211 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.76465.log 2024-09-12T14:19:30.212 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.122267.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.122267.log.gz 2024-09-12T14:19:30.212 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.40639.log 2024-09-12T14:19:30.212 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.76465.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.76465.log.gz 2024-09-12T14:19:30.213 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.106234.log 2024-09-12T14:19:30.213 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.40639.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.40639.log.gz 2024-09-12T14:19:30.213 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.86033.log 2024-09-12T14:19:30.214 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.106234.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.106234.log.gz 2024-09-12T14:19:30.214 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.60583.log 2024-09-12T14:19:30.215 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.86033.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.86033.log.gz 2024-09-12T14:19:30.215 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.120481.log 2024-09-12T14:19:30.215 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.60583.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.60583.log.gz 2024-09-12T14:19:30.216 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.141007.log 2024-09-12T14:19:30.216 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.120481.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.120481.log.gz 2024-09-12T14:19:30.216 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.140434.log 2024-09-12T14:19:30.217 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.141007.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.141007.log.gz 2024-09-12T14:19:30.217 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.49996.log 2024-09-12T14:19:30.218 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.140434.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.140434.log.gz 2024-09-12T14:19:30.218 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.127803.log 2024-09-12T14:19:30.218 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.49996.log: 0.0%gzip -- replaced with /var/log/ceph/cluster1-client.mirror.49996.log.gz 2024-09-12T14:19:30.218 INFO:teuthology.orchestra.run.smithi130.stderr: -5 --verbose -- /var/log/ceph/cluster2-client.mirror.129617.log 2024-09-12T14:19:30.219 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.127803.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.127803.log.gz 2024-09-12T14:19:30.219 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.112173.log 2024-09-12T14:19:30.220 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.129617.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.129617.log.gz 2024-09-12T14:19:30.220 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.145603.log 2024-09-12T14:19:30.221 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.112173.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.112173.log.gz 2024-09-12T14:19:30.221 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.151939.log 2024-09-12T14:19:30.222 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.145603.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.145603.log.gz 2024-09-12T14:19:30.222 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.67188.log 2024-09-12T14:19:30.222 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.151939.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.151939.log.gz 2024-09-12T14:19:30.222 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.140623.log 2024-09-12T14:19:30.223 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.67188.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.67188.log.gz 2024-09-12T14:19:30.223 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.80703.log 2024-09-12T14:19:30.224 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.140623.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.140623.log.gz 2024-09-12T14:19:30.224 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.46562.log 2024-09-12T14:19:30.225 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.80703.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.80703.log.gz 2024-09-12T14:19:30.225 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.36418.log 2024-09-12T14:19:30.225 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.46562.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.46562.log.gz 2024-09-12T14:19:30.226 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.133107.log 2024-09-12T14:19:30.226 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.36418.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.36418.log.gz 2024-09-12T14:19:30.227 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.38030.log 2024-09-12T14:19:30.227 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.133107.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.133107.log.gz 2024-09-12T14:19:30.227 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.72455.log 2024-09-12T14:19:30.228 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.38030.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.38030.log.gz 2024-09-12T14:19:30.228 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.148944.log 2024-09-12T14:19:30.229 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.72455.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.72455.log.gz 2024-09-12T14:19:30.229 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.111734.log 2024-09-12T14:19:30.230 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.148944.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.148944.log.gz 2024-09-12T14:19:30.230 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.89712.log 2024-09-12T14:19:30.231 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.111734.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.111734.log.gz 2024-09-12T14:19:30.231 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.142726.log 2024-09-12T14:19:30.232 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.89712.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.89712.log.gz 2024-09-12T14:19:30.232 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.76591.log 2024-09-12T14:19:30.232 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.142726.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.142726.log.gz 2024-09-12T14:19:30.232 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.147601.log 2024-09-12T14:19:30.233 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.76591.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.76591.log.gz 2024-09-12T14:19:30.233 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.70925.log 2024-09-12T14:19:30.234 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.147601.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.147601.log.gz 2024-09-12T14:19:30.234 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.103055.log 2024-09-12T14:19:30.234 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.70925.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.70925.log.gz 2024-09-12T14:19:30.235 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.103098.log 2024-09-12T14:19:30.235 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.103055.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.103055.log.gz 2024-09-12T14:19:30.235 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.114603.log 2024-09-12T14:19:30.236 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.103098.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.103098.log.gz 2024-09-12T14:19:30.236 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.56298.log 2024-09-12T14:19:30.237 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.114603.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.114603.log.gz 2024-09-12T14:19:30.237 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.81068.log 2024-09-12T14:19:30.238 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.56298.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.56298.log.gz 2024-09-12T14:19:30.238 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.102178.log 2024-09-12T14:19:30.238 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.81068.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.81068.log.gz 2024-09-12T14:19:30.239 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.40052.log 2024-09-12T14:19:30.239 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.102178.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.102178.log.gz 2024-09-12T14:19:30.240 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.100732.log 2024-09-12T14:19:30.240 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.40052.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.40052.log.gz 2024-09-12T14:19:30.240 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.105249.log 2024-09-12T14:19:30.241 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.100732.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.100732.log.gz 2024-09-12T14:19:30.241 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.93658.log 2024-09-12T14:19:30.242 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.105249.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.105249.log.gz 2024-09-12T14:19:30.242 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.69524.log 2024-09-12T14:19:30.243 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.93658.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.93658.log.gz 2024-09-12T14:19:30.243 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.140431.log 2024-09-12T14:19:30.244 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.69524.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.69524.log.gz 2024-09-12T14:19:30.244 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.57602.log 2024-09-12T14:19:30.244 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.140431.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.140431.log.gz 2024-09-12T14:19:30.245 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.127720.log 2024-09-12T14:19:30.245 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.57602.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.57602.log.gz 2024-09-12T14:19:30.245 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.135416.log 2024-09-12T14:19:30.246 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.127720.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.127720.log.gz 2024-09-12T14:19:30.246 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.91658.log 2024-09-12T14:19:30.247 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.135416.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.135416.log.gz 2024-09-12T14:19:30.247 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.131469.log 2024-09-12T14:19:30.247 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.91658.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.91658.log.gz 2024-09-12T14:19:30.248 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.149134.log 2024-09-12T14:19:30.248 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.131469.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.131469.log.gz 2024-09-12T14:19:30.249 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.86851.log 2024-09-12T14:19:30.249 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.149134.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.149134.log.gz 2024-09-12T14:19:30.249 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.120669.log 2024-09-12T14:19:30.250 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.86851.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.86851.log.gz 2024-09-12T14:19:30.250 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.86785.log 2024-09-12T14:19:30.251 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.120669.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.120669.log.gz 2024-09-12T14:19:30.251 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.133044.log 2024-09-12T14:19:30.252 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.86785.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.86785.log.gz 2024-09-12T14:19:30.252 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.147917.log 2024-09-12T14:19:30.252 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.133044.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.133044.log.gz 2024-09-12T14:19:30.252 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.78556.log 2024-09-12T14:19:30.253 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.147917.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.147917.log.gz 2024-09-12T14:19:30.253 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.117749.log 2024-09-12T14:19:30.254 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.78556.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.78556.log.gz 2024-09-12T14:19:30.254 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.105705.log 2024-09-12T14:19:30.255 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.117749.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.117749.log.gz 2024-09-12T14:19:30.255 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.116907.log 2024-09-12T14:19:30.255 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.105705.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.105705.log.gz 2024-09-12T14:19:30.256 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.137475.log 2024-09-12T14:19:30.256 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.116907.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.116907.log.gz 2024-09-12T14:19:30.256 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.66625.log 2024-09-12T14:19:30.257 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.137475.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.137475.log.gz 2024-09-12T14:19:30.257 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.77728.log 2024-09-12T14:19:30.258 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.66625.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.66625.log.gz 2024-09-12T14:19:30.258 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.122033.log 2024-09-12T14:19:30.259 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.77728.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.77728.log.gz 2024-09-12T14:19:30.259 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.37090.log 2024-09-12T14:19:30.260 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.122033.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.122033.log.gz 2024-09-12T14:19:30.260 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.62159.log 2024-09-12T14:19:30.260 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.37090.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.37090.log.gz 2024-09-12T14:19:30.260 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.129886.log 2024-09-12T14:19:30.261 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.62159.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.62159.log.gz 2024-09-12T14:19:30.261 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.81149.log 2024-09-12T14:19:30.262 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.129886.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.129886.log.gz 2024-09-12T14:19:30.262 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.60126.log 2024-09-12T14:19:30.263 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.81149.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.81149.log.gz 2024-09-12T14:19:30.263 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.78105.log 2024-09-12T14:19:30.263 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.60126.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.60126.log.gz 2024-09-12T14:19:30.264 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.89055.log 2024-09-12T14:19:30.264 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.78105.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.78105.log.gz 2024-09-12T14:19:30.265 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.29221.log 2024-09-12T14:19:30.265 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.89055.log: gzip -5 --verbose 0.0% -- /var/log/ceph/cluster1-client.mirror.83957.log -- replaced with /var/log/ceph/cluster1-client.mirror.89055.log.gz 2024-09-12T14:19:30.265 INFO:teuthology.orchestra.run.smithi130.stderr: 2024-09-12T14:19:30.266 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.29221.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.29221.log.gz 2024-09-12T14:19:30.266 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.55053.log 2024-09-12T14:19:30.267 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.83957.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.83957.log.gz 2024-09-12T14:19:30.267 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.83324.log 2024-09-12T14:19:30.268 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.55053.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.55053.log.gz 2024-09-12T14:19:30.268 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.134176.log 2024-09-12T14:19:30.268 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.83324.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.83324.log.gz 2024-09-12T14:19:30.269 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.136047.log 2024-09-12T14:19:30.269 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.134176.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.134176.log.gz 2024-09-12T14:19:30.270 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.143540.log 2024-09-12T14:19:30.270 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.136047.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.136047.log.gz 2024-09-12T14:19:30.270 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.35160.log 2024-09-12T14:19:30.271 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.143540.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.143540.log.gz 2024-09-12T14:19:30.271 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.51276.log 2024-09-12T14:19:30.272 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.35160.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.35160.log.gz 2024-09-12T14:19:30.272 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.113559.log 2024-09-12T14:19:30.273 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.51276.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.51276.log.gz 2024-09-12T14:19:30.273 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.90315.log 2024-09-12T14:19:30.274 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.113559.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.113559.log.gz 2024-09-12T14:19:30.274 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.68278.log 2024-09-12T14:19:30.275 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.90315.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.90315.log.gz 2024-09-12T14:19:30.275 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.62476.log 2024-09-12T14:19:30.275 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.68278.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.68278.log.gz 2024-09-12T14:19:30.275 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.44143.log 2024-09-12T14:19:30.276 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.62476.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.62476.log.gz 2024-09-12T14:19:30.276 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.44682.log 2024-09-12T14:19:30.277 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.44143.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.44143.log.gz 2024-09-12T14:19:30.277 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.125783.log 2024-09-12T14:19:30.277 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.44682.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.44682.log.gz 2024-09-12T14:19:30.277 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.132430.log 2024-09-12T14:19:30.278 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.125783.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.125783.log.gz 2024-09-12T14:19:30.278 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.153917.log 2024-09-12T14:19:30.279 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.132430.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.132430.log.gz 2024-09-12T14:19:30.279 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.37638.log 2024-09-12T14:19:30.279 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.153917.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.153917.log.gz 2024-09-12T14:19:30.279 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.121827.log 2024-09-12T14:19:30.280 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.37638.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.37638.log.gz 2024-09-12T14:19:30.280 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.121090.log 2024-09-12T14:19:30.281 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.121827.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.121827.log.gz 2024-09-12T14:19:30.281 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.75538.log 2024-09-12T14:19:30.282 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.121090.log: gzip -5 --verbose 0.0% -- -- replaced with /var/log/ceph/cluster1-client.mirror.121090.log.gz /var/log/ceph/cluster1-client.mirror.52893.log 2024-09-12T14:19:30.282 INFO:teuthology.orchestra.run.smithi130.stderr: 2024-09-12T14:19:30.283 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.75538.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.75538.log.gz 2024-09-12T14:19:30.283 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.49218.log 2024-09-12T14:19:30.284 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.52893.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.52893.log.gz 2024-09-12T14:19:30.284 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.125392.log 2024-09-12T14:19:30.284 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.49218.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.49218.log.gz 2024-09-12T14:19:30.284 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.57331.log 2024-09-12T14:19:30.285 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.125392.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.125392.log.gz 2024-09-12T14:19:30.285 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.114867.log 2024-09-12T14:19:30.286 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.57331.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.57331.log.gz 2024-09-12T14:19:30.286 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.107316.log 2024-09-12T14:19:30.287 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.114867.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.114867.log.gz 2024-09-12T14:19:30.287 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.68674.log 2024-09-12T14:19:30.288 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.107316.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.107316.log.gz 2024-09-12T14:19:30.288 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.40432.log 2024-09-12T14:19:30.288 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.68674.log: gzip 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.68674.log.gz 2024-09-12T14:19:30.288 INFO:teuthology.orchestra.run.smithi130.stderr: -5 --verbose -- /var/log/ceph/cluster1-client.mirror.124965.log 2024-09-12T14:19:30.289 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.40432.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.40432.log.gz 2024-09-12T14:19:30.289 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.61590.log 2024-09-12T14:19:30.290 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.124965.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.124965.log.gz 2024-09-12T14:19:30.290 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.146927.log 2024-09-12T14:19:30.291 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.61590.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.61590.log.gz 2024-09-12T14:19:30.291 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.48347.log 2024-09-12T14:19:30.292 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.146927.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.146927.log.gz 2024-09-12T14:19:30.292 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.145819.log 2024-09-12T14:19:30.292 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.48347.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.48347.log.gz 2024-09-12T14:19:30.292 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.87690.log 2024-09-12T14:19:30.293 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.145819.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.145819.log.gz 2024-09-12T14:19:30.293 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.115042.log 2024-09-12T14:19:30.294 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.87690.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.87690.log.gz 2024-09-12T14:19:30.294 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.61927.log 2024-09-12T14:19:30.295 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.115042.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.115042.log.gz 2024-09-12T14:19:30.295 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.141436.log 2024-09-12T14:19:30.296 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.61927.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.61927.log.gz 2024-09-12T14:19:30.296 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.99617.log 2024-09-12T14:19:30.297 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.141436.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.141436.log.gz 2024-09-12T14:19:30.297 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.94707.log 2024-09-12T14:19:30.297 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.99617.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.99617.log.gz 2024-09-12T14:19:30.298 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.87969.log 2024-09-12T14:19:30.298 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.94707.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.94707.log.gz 2024-09-12T14:19:30.298 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.107779.log 2024-09-12T14:19:30.299 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.87969.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.87969.log.gz 2024-09-12T14:19:30.299 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.98736.log 2024-09-12T14:19:30.300 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.107779.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.107779.log.gz 2024-09-12T14:19:30.300 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.126077.log 2024-09-12T14:19:30.300 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.98736.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.98736.log.gz 2024-09-12T14:19:30.301 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.87211.log 2024-09-12T14:19:30.301 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.126077.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.126077.log.gz 2024-09-12T14:19:30.301 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.116690.log 2024-09-12T14:19:30.302 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.87211.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.87211.log.gz 2024-09-12T14:19:30.302 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.67628.log 2024-09-12T14:19:30.303 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.116690.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.116690.log.gz 2024-09-12T14:19:30.303 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.141302.log 2024-09-12T14:19:30.303 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.67628.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.67628.log.gz 2024-09-12T14:19:30.304 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.118018.log 2024-09-12T14:19:30.304 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.141302.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.141302.log.gz 2024-09-12T14:19:30.304 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.93119.log 2024-09-12T14:19:30.305 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.118018.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.118018.log.gz 2024-09-12T14:19:30.305 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.36635.log 2024-09-12T14:19:30.306 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.93119.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.93119.log.gz 2024-09-12T14:19:30.306 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.96130.log 2024-09-12T14:19:30.307 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.36635.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.36635.log.gz 2024-09-12T14:19:30.307 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.79726.log 2024-09-12T14:19:30.308 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.96130.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.96130.log.gz 2024-09-12T14:19:30.308 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.84270.log 2024-09-12T14:19:30.308 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.79726.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.79726.log.gz 2024-09-12T14:19:30.309 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.121195.log 2024-09-12T14:19:30.309 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.84270.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.84270.log.gz 2024-09-12T14:19:30.309 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.58203.log 2024-09-12T14:19:30.310 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.121195.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.121195.log.gz 2024-09-12T14:19:30.310 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.118526.log 2024-09-12T14:19:30.311 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.58203.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.58203.log.gz 2024-09-12T14:19:30.311 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.132014.log 2024-09-12T14:19:30.312 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.118526.log: gzip -5 --verbose 0.0% -- -- replaced with /var/log/ceph/cluster1-client.mirror.118526.log.gz /var/log/ceph/cluster1-client.mirror.84206.log 2024-09-12T14:19:30.312 INFO:teuthology.orchestra.run.smithi130.stderr: 2024-09-12T14:19:30.312 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.132014.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.132014.log.gz 2024-09-12T14:19:30.313 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.122352.log 2024-09-12T14:19:30.314 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.84206.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.84206.log.gz 2024-09-12T14:19:30.314 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.52769.log 2024-09-12T14:19:30.314 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.122352.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.122352.log.gz 2024-09-12T14:19:30.314 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.89205.log 2024-09-12T14:19:30.315 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.52769.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.52769.log.gz 2024-09-12T14:19:30.315 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.142285.log 2024-09-12T14:19:30.316 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.89205.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.89205.log.gz 2024-09-12T14:19:30.316 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.112258.log 2024-09-12T14:19:30.317 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.142285.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.142285.log.gz 2024-09-12T14:19:30.317 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.152574.log 2024-09-12T14:19:30.317 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.112258.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.112258.log.gz 2024-09-12T14:19:30.318 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.102120.log 2024-09-12T14:19:30.318 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.152574.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.152574.log.gz 2024-09-12T14:19:30.318 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.127695.log 2024-09-12T14:19:30.319 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.102120.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.102120.log.gz 2024-09-12T14:19:30.319 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.115554.log 2024-09-12T14:19:30.320 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.127695.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.127695.log.gz 2024-09-12T14:19:30.320 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.137010.log 2024-09-12T14:19:30.321 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.115554.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.115554.log.gz 2024-09-12T14:19:30.321 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.109404.log 2024-09-12T14:19:30.322 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.137010.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.137010.log.gz 2024-09-12T14:19:30.322 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.117179.log 2024-09-12T14:19:30.322 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.109404.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.109404.log.gz 2024-09-12T14:19:30.322 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.49598.log 2024-09-12T14:19:30.323 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.117179.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.117179.log.gz 2024-09-12T14:19:30.323 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.83135.log 2024-09-12T14:19:30.324 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.49598.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.49598.log.gz 2024-09-12T14:19:30.324 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.138895.log 2024-09-12T14:19:30.325 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.83135.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.83135.log.gz 2024-09-12T14:19:30.325 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.93226.log 2024-09-12T14:19:30.326 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.138895.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.138895.log.gz 2024-09-12T14:19:30.326 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.98061.log 2024-09-12T14:19:30.326 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.93226.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.93226.log.gz 2024-09-12T14:19:30.327 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.70713.log 2024-09-12T14:19:30.327 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.98061.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.98061.log.gz 2024-09-12T14:19:30.327 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.119894.log 2024-09-12T14:19:30.328 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.70713.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.70713.log.gz 2024-09-12T14:19:30.328 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.45544.log 2024-09-12T14:19:30.329 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.119894.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.119894.log.gz 2024-09-12T14:19:30.329 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.99765.log 2024-09-12T14:19:30.330 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.45544.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.45544.log.gz 2024-09-12T14:19:30.330 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.77136.log 2024-09-12T14:19:30.331 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.99765.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.99765.log.gz 2024-09-12T14:19:30.331 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.47299.log 2024-09-12T14:19:30.331 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.77136.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.77136.log.gz 2024-09-12T14:19:30.331 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.125222.log 2024-09-12T14:19:30.332 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.47299.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.47299.log.gz 2024-09-12T14:19:30.332 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.140828.log 2024-09-12T14:19:30.333 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.125222.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.125222.log.gz 2024-09-12T14:19:30.333 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.89915.log 2024-09-12T14:19:30.333 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.140828.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.140828.log.gz 2024-09-12T14:19:30.334 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.93211.log 2024-09-12T14:19:30.335 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.89915.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.89915.log.gz 2024-09-12T14:19:30.335 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.58177.log 2024-09-12T14:19:30.335 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.93211.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.93211.log.gz 2024-09-12T14:19:30.336 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.97958.log 2024-09-12T14:19:30.336 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.58177.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.58177.log.gz 2024-09-12T14:19:30.336 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.45225.log 2024-09-12T14:19:30.337 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.97958.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.97958.log.gz 2024-09-12T14:19:30.337 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.85443.log 2024-09-12T14:19:30.338 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.45225.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.45225.log.gz 2024-09-12T14:19:30.338 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.78422.log 2024-09-12T14:19:30.339 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.85443.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.85443.log.gz 2024-09-12T14:19:30.339 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.130443.log 2024-09-12T14:19:30.339 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.78422.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.78422.log.gz 2024-09-12T14:19:30.339 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.61675.log 2024-09-12T14:19:30.340 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.130443.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.130443.log.gz 2024-09-12T14:19:30.340 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.116987.log 2024-09-12T14:19:30.341 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.61675.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.61675.log.gz 2024-09-12T14:19:30.341 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.40178.log 2024-09-12T14:19:30.342 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.116987.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.116987.log.gz 2024-09-12T14:19:30.342 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.146615.log 2024-09-12T14:19:30.343 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.40178.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.40178.log.gz 2024-09-12T14:19:30.343 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.152934.log 2024-09-12T14:19:30.343 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.146615.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.146615.log.gz 2024-09-12T14:19:30.343 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.74050.log 2024-09-12T14:19:30.344 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.152934.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.152934.log.gz 2024-09-12T14:19:30.344 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.91341.log 2024-09-12T14:19:30.345 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.74050.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.74050.log.gz 2024-09-12T14:19:30.345 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.125899.log 2024-09-12T14:19:30.346 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.91341.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.91341.log.gz 2024-09-12T14:19:30.346 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.128196.log 2024-09-12T14:19:30.347 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.125899.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.125899.log.gz 2024-09-12T14:19:30.347 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.127137.log 2024-09-12T14:19:30.347 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.128196.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.128196.log.gz 2024-09-12T14:19:30.347 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.99934.log 2024-09-12T14:19:30.348 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.127137.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.127137.log.gz 2024-09-12T14:19:30.348 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.150297.log 2024-09-12T14:19:30.349 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.99934.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.99934.log.gz 2024-09-12T14:19:30.349 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.126985.log 2024-09-12T14:19:30.350 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.150297.log: gzip -5 --verbose -- 0.0% /var/log/ceph/cluster1-client.admin.44442.log -- replaced with /var/log/ceph/cluster1-client.admin.150297.log.gz 2024-09-12T14:19:30.350 INFO:teuthology.orchestra.run.smithi130.stderr: 2024-09-12T14:19:30.351 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.126985.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.126985.log.gz 2024-09-12T14:19:30.351 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.89875.log 2024-09-12T14:19:30.351 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.44442.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.44442.log.gz 2024-09-12T14:19:30.351 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.75220.log 2024-09-12T14:19:30.352 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.89875.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.89875.log.gz 2024-09-12T14:19:30.352 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.50563.log 2024-09-12T14:19:30.353 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.75220.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.75220.log.gz 2024-09-12T14:19:30.353 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.36114.log 2024-09-12T14:19:30.354 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.50563.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.50563.log.gz 2024-09-12T14:19:30.354 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.145648.log 2024-09-12T14:19:30.355 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.36114.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.36114.log.gz 2024-09-12T14:19:30.355 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.98859.log 2024-09-12T14:19:30.355 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.145648.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.145648.log.gz 2024-09-12T14:19:30.356 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.104221.log 2024-09-12T14:19:30.356 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.98859.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.98859.log.gz 2024-09-12T14:19:30.356 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.61340.log 2024-09-12T14:19:30.364 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.104221.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.51930.log 2024-09-12T14:19:30.365 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.104221.log.gz 2024-09-12T14:19:30.365 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.61340.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.61340.log.gz 2024-09-12T14:19:30.365 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.38477.log 2024-09-12T14:19:30.365 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.51930.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.51930.log.gz 2024-09-12T14:19:30.365 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.46884.log 2024-09-12T14:19:30.365 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.38477.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.38477.log.gz 2024-09-12T14:19:30.365 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.39214.log 2024-09-12T14:19:30.365 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.46884.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.46884.log.gz 2024-09-12T14:19:30.365 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.77493.log 2024-09-12T14:19:30.365 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.39214.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.39214.log.gz 2024-09-12T14:19:30.366 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.77305.log 2024-09-12T14:19:30.366 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.77493.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.77493.log.gz 2024-09-12T14:19:30.366 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.95012.log 2024-09-12T14:19:30.366 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.77305.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.77305.log.gz 2024-09-12T14:19:30.366 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.41438.log 2024-09-12T14:19:30.366 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.95012.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.95012.log.gz 2024-09-12T14:19:30.366 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.147011.log 2024-09-12T14:19:30.366 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.41438.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.91603.log 2024-09-12T14:19:30.366 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.41438.log.gz 2024-09-12T14:19:30.369 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.147011.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.147011.log.gz 2024-09-12T14:19:30.369 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.51761.log 2024-09-12T14:19:30.369 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.91603.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.91603.log.gz 2024-09-12T14:19:30.369 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.40638.log 2024-09-12T14:19:30.369 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.51761.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.51761.log.gz 2024-09-12T14:19:30.369 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.149200.log 2024-09-12T14:19:30.369 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.40638.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.40638.log.gz 2024-09-12T14:19:30.369 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.86534.log 2024-09-12T14:19:30.369 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.149200.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.149200.log.gz 2024-09-12T14:19:30.369 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.153235.log 2024-09-12T14:19:30.371 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.86534.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.86534.log.gz 2024-09-12T14:19:30.371 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.38622.log 2024-09-12T14:19:30.371 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.153235.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.153235.log.gz 2024-09-12T14:19:30.371 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.135602.log 2024-09-12T14:19:30.371 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.38622.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.38622.log.gz 2024-09-12T14:19:30.371 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.123013.log 2024-09-12T14:19:30.372 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.135602.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.135602.log.gz 2024-09-12T14:19:30.373 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.103705.log 2024-09-12T14:19:30.373 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.123013.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.123013.log.gz 2024-09-12T14:19:30.373 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.85317.log 2024-09-12T14:19:30.374 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.103705.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.103705.log.gz 2024-09-12T14:19:30.374 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.95582.log 2024-09-12T14:19:30.375 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.85317.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.85317.log.gz 2024-09-12T14:19:30.375 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.93289.log 2024-09-12T14:19:30.375 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.95582.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.95582.log.gz 2024-09-12T14:19:30.375 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.110452.log 2024-09-12T14:19:30.376 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.93289.log: gzip 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.93289.log.gz 2024-09-12T14:19:30.376 INFO:teuthology.orchestra.run.smithi130.stderr: -5 --verbose -- /var/log/ceph/cluster1-client.mirror.100604.log 2024-09-12T14:19:30.377 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.110452.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.110452.log.gz 2024-09-12T14:19:30.377 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.82315.log 2024-09-12T14:19:30.377 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.100604.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.100604.log.gz 2024-09-12T14:19:30.378 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.106537.log 2024-09-12T14:19:30.379 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.82315.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.82315.log.gz 2024-09-12T14:19:30.379 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.103959.log 2024-09-12T14:19:30.379 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.106537.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.106537.log.gz 2024-09-12T14:19:30.379 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.38753.log 2024-09-12T14:19:30.380 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.103959.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.37091.log 0.0% 2024-09-12T14:19:30.380 INFO:teuthology.orchestra.run.smithi130.stderr: -- replaced with /var/log/ceph/cluster2-client.admin.103959.log.gz 2024-09-12T14:19:30.381 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.38753.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.38753.log.gz 2024-09-12T14:19:30.381 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.153276.log 2024-09-12T14:19:30.382 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.37091.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.37091.log.gz 2024-09-12T14:19:30.382 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.35126.log 2024-09-12T14:19:30.383 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.153276.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.153276.log.gz 2024-09-12T14:19:30.383 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.86074.log 2024-09-12T14:19:30.383 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.35126.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.35126.log.gz 2024-09-12T14:19:30.383 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.34870.log 2024-09-12T14:19:30.384 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.86074.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.86074.log.gz 2024-09-12T14:19:30.384 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.105037.log 2024-09-12T14:19:30.385 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.34870.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.34870.log.gz 2024-09-12T14:19:30.385 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.61192.log 2024-09-12T14:19:30.386 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.105037.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.105037.log.gz 2024-09-12T14:19:30.386 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.40835.log 2024-09-12T14:19:30.386 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.61192.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.61192.log.gz 2024-09-12T14:19:30.387 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.82154.log 2024-09-12T14:19:30.388 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.40835.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.40835.log.gz 2024-09-12T14:19:30.388 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.53019.log 2024-09-12T14:19:30.388 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.82154.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.82154.log.gz 2024-09-12T14:19:30.388 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.141033.log 2024-09-12T14:19:30.389 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.53019.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.53019.log.gz 2024-09-12T14:19:30.389 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.86346.log 2024-09-12T14:19:30.389 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.141033.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.141033.log.gz 2024-09-12T14:19:30.389 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.124966.log 2024-09-12T14:19:30.391 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.86346.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.86346.log.gz 2024-09-12T14:19:30.391 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.60943.log 2024-09-12T14:19:30.391 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.124966.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.124966.log.gz 2024-09-12T14:19:30.391 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.110537.log 2024-09-12T14:19:30.392 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.60943.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.60943.log.gz 2024-09-12T14:19:30.392 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.37343.log 2024-09-12T14:19:30.393 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.110537.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.110537.log.gz 2024-09-12T14:19:30.393 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.69537.log 2024-09-12T14:19:30.393 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.37343.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.37343.log.gz 2024-09-12T14:19:30.393 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.105611.log 2024-09-12T14:19:30.394 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.69537.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.69537.log.gz 2024-09-12T14:19:30.394 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.90345.log 2024-09-12T14:19:30.395 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.105611.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.105611.log.gz 2024-09-12T14:19:30.395 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.49261.log 2024-09-12T14:19:30.396 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.90345.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.90345.log.gz 2024-09-12T14:19:30.396 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.76633.log 2024-09-12T14:19:30.397 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.49261.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.49261.log.gz 2024-09-12T14:19:30.397 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.104167.log 2024-09-12T14:19:30.397 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.76633.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.76633.log.gz 2024-09-12T14:19:30.397 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.100813.log 2024-09-12T14:19:30.398 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.104167.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.104167.log.gz 2024-09-12T14:19:30.398 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.143035.log 2024-09-12T14:19:30.399 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.100813.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.100813.log.gz 2024-09-12T14:19:30.399 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.145462.log 2024-09-12T14:19:30.400 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.143035.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.143035.log.gz 2024-09-12T14:19:30.400 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.135075.log 2024-09-12T14:19:30.401 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.145462.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.145462.log.gz 2024-09-12T14:19:30.401 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.139819.log 2024-09-12T14:19:30.401 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.135075.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.135075.log.gz 2024-09-12T14:19:30.401 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.56172.log 2024-09-12T14:19:30.402 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.139819.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.139819.log.gz 2024-09-12T14:19:30.402 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.51821.log 2024-09-12T14:19:30.403 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.56172.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.56172.log.gz 2024-09-12T14:19:30.403 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.81490.log 2024-09-12T14:19:30.404 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.51821.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.51821.log.gz 2024-09-12T14:19:30.404 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.33723.log 2024-09-12T14:19:30.404 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.81490.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.81490.log.gz 2024-09-12T14:19:30.405 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.76637.log 2024-09-12T14:19:30.405 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.33723.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.33723.log.gz 2024-09-12T14:19:30.405 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.126767.log 2024-09-12T14:19:30.406 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.76637.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.76637.log.gz 2024-09-12T14:19:30.406 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.41238.log 2024-09-12T14:19:30.407 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.126767.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.126767.log.gz 2024-09-12T14:19:30.407 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.83441.log 2024-09-12T14:19:30.408 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.41238.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.41238.log.gz 2024-09-12T14:19:30.408 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.55549.log 2024-09-12T14:19:30.409 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.83441.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.83441.log.gz 2024-09-12T14:19:30.409 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.79590.log 2024-09-12T14:19:30.409 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.55549.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.55549.log.gz 2024-09-12T14:19:30.410 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.67762.log 2024-09-12T14:19:30.411 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.79590.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.79590.log.gz 2024-09-12T14:19:30.411 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.80764.log 2024-09-12T14:19:30.411 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.67762.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.67762.log.gz 2024-09-12T14:19:30.411 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.98716.log 2024-09-12T14:19:30.412 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.80764.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.80764.log.gz 2024-09-12T14:19:30.413 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.100936.log 2024-09-12T14:19:30.413 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.98716.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.98716.log.gz 2024-09-12T14:19:30.413 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.43940.log 2024-09-12T14:19:30.414 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.100936.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.100936.log.gz 2024-09-12T14:19:30.414 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.50541.log 2024-09-12T14:19:30.414 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.43940.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.43940.log.gz 2024-09-12T14:19:30.414 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.140748.log 2024-09-12T14:19:30.416 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.50541.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.50541.log.gz 2024-09-12T14:19:30.416 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.42122.log 2024-09-12T14:19:30.416 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.140748.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.140748.log.gz 2024-09-12T14:19:30.416 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.117199.log 2024-09-12T14:19:30.417 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.42122.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.42122.log.gz 2024-09-12T14:19:30.417 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.134994.log 2024-09-12T14:19:30.418 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.117199.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.117199.log.gz 2024-09-12T14:19:30.418 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.76488.log 2024-09-12T14:19:30.419 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.134994.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.134994.log.gz 2024-09-12T14:19:30.419 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.131692.log 2024-09-12T14:19:30.419 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.76488.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.76488.log.gz 2024-09-12T14:19:30.419 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.115827.log 2024-09-12T14:19:30.421 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.131692.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.131692.log.gz 2024-09-12T14:19:30.421 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.86704.log 2024-09-12T14:19:30.421 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.115827.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.115827.log.gz 2024-09-12T14:19:30.421 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.116243.log 2024-09-12T14:19:30.422 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.86704.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.86704.log.gz 2024-09-12T14:19:30.422 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.78524.log 2024-09-12T14:19:30.423 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.116243.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.116243.log.gz 2024-09-12T14:19:30.423 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.129933.log 2024-09-12T14:19:30.424 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.78524.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.78524.log.gz 2024-09-12T14:19:30.424 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.98419.log 2024-09-12T14:19:30.424 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.129933.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.129933.log.gz 2024-09-12T14:19:30.424 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.116483.log 2024-09-12T14:19:30.425 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.98419.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.98419.log.gz 2024-09-12T14:19:30.426 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.51239.log 2024-09-12T14:19:30.426 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.116483.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.116483.log.gz 2024-09-12T14:19:30.426 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.75707.log 2024-09-12T14:19:30.427 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.51239.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.51239.log.gz 2024-09-12T14:19:30.427 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.134135.log 2024-09-12T14:19:30.427 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.75707.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.75707.log.gz 2024-09-12T14:19:30.427 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.142511.log 2024-09-12T14:19:30.428 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.134135.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.134135.log.gz 2024-09-12T14:19:30.428 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.67853.log 2024-09-12T14:19:30.430 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.142511.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.142511.log.gz 2024-09-12T14:19:30.430 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.38311.log 2024-09-12T14:19:30.430 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.67853.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.67853.log.gz 2024-09-12T14:19:30.430 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.113688.log 2024-09-12T14:19:30.431 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.38311.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.38311.log.gz 2024-09-12T14:19:30.431 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.100352.log 2024-09-12T14:19:30.432 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.113688.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.113688.log.gz 2024-09-12T14:19:30.432 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.111668.log 2024-09-12T14:19:30.432 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.100352.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.100352.log.gz 2024-09-12T14:19:30.432 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.90893.log 2024-09-12T14:19:30.433 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.111668.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.111668.log.gz 2024-09-12T14:19:30.434 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.125434.log 2024-09-12T14:19:30.434 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.90893.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.90893.log.gz 2024-09-12T14:19:30.434 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.115127.log 2024-09-12T14:19:30.435 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.125434.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.125434.log.gz 2024-09-12T14:19:30.435 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.72286.log 2024-09-12T14:19:30.436 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.115127.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.115127.log.gz 2024-09-12T14:19:30.436 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.43265.log 2024-09-12T14:19:30.436 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.72286.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.72286.log.gz 2024-09-12T14:19:30.436 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.50144.log 2024-09-12T14:19:30.438 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.43265.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.43265.log.gz 2024-09-12T14:19:30.438 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.108416.log 2024-09-12T14:19:30.438 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.50144.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.50144.log.gz 2024-09-12T14:19:30.438 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.77284.log 2024-09-12T14:19:30.439 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.108416.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.108416.log.gz 2024-09-12T14:19:30.440 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.115791.log 2024-09-12T14:19:30.440 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.77284.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.77284.log.gz 2024-09-12T14:19:30.440 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.140079.log 2024-09-12T14:19:30.441 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.115791.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.115791.log.gz 2024-09-12T14:19:30.441 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.77138.log 2024-09-12T14:19:30.441 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.140079.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.140079.log.gz 2024-09-12T14:19:30.441 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.86073.log 2024-09-12T14:19:30.442 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.77138.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.77138.log.gz 2024-09-12T14:19:30.443 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.78058.log 2024-09-12T14:19:30.443 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.86073.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.86073.log.gz 2024-09-12T14:19:30.443 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.105456.log 2024-09-12T14:19:30.444 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.78058.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.82569.log 2024-09-12T14:19:30.444 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.78058.log.gz 2024-09-12T14:19:30.444 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.105456.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.105456.log.gz 2024-09-12T14:19:30.444 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.72496.log 2024-09-12T14:19:30.446 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.82569.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.82569.log.gz 2024-09-12T14:19:30.446 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.116098.log 2024-09-12T14:19:30.446 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.72496.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.72496.log.gz 2024-09-12T14:19:30.446 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.96799.log 2024-09-12T14:19:30.447 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.116098.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.116098.log.gz 2024-09-12T14:19:30.447 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.75848.log 2024-09-12T14:19:30.447 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.96799.log: gzip 0.0% -5 -- replaced with /var/log/ceph/cluster1-client.mirror.96799.log.gz --verbose 2024-09-12T14:19:30.447 INFO:teuthology.orchestra.run.smithi130.stderr: -- /var/log/ceph/cluster1-client.mirror.38854.log 2024-09-12T14:19:30.449 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.75848.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.75848.log.gz 2024-09-12T14:19:30.449 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.97853.log 2024-09-12T14:19:30.449 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.38854.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.38854.log.gz 2024-09-12T14:19:30.449 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.131405.log 2024-09-12T14:19:30.450 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.97853.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.97853.log.gz 2024-09-12T14:19:30.450 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.41568.log 2024-09-12T14:19:30.450 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.131405.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.131405.log.gz 2024-09-12T14:19:30.450 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.109403.log 2024-09-12T14:19:30.451 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.41568.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.41568.log.gz 2024-09-12T14:19:30.451 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.87742.log 2024-09-12T14:19:30.452 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.109403.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.109403.log.gz 2024-09-12T14:19:30.452 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.48844.log 2024-09-12T14:19:30.453 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.87742.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.87742.log.gz 2024-09-12T14:19:30.453 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.139737.log 2024-09-12T14:19:30.454 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.48844.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.48844.log.gz 2024-09-12T14:19:30.454 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.98504.log 2024-09-12T14:19:30.455 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.139737.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.139737.log.gz 2024-09-12T14:19:30.455 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.85442.log 2024-09-12T14:19:30.455 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.98504.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.98504.log.gz 2024-09-12T14:19:30.456 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.77306.log 2024-09-12T14:19:30.456 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.85442.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.85442.log.gz 2024-09-12T14:19:30.456 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.60436.log 2024-09-12T14:19:30.457 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.77306.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.77306.log.gz 2024-09-12T14:19:30.457 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.67880.log 2024-09-12T14:19:30.458 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.60436.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.60436.log.gz 2024-09-12T14:19:30.458 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.52305.log 2024-09-12T14:19:30.459 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.67880.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.67880.log.gz 2024-09-12T14:19:30.459 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.129591.log 2024-09-12T14:19:30.459 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.52305.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.52305.log.gz 2024-09-12T14:19:30.460 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.107411.log 2024-09-12T14:19:30.461 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.129591.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.129591.log.gz 2024-09-12T14:19:30.461 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.118299.log 2024-09-12T14:19:30.461 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.107411.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.107411.log.gz 2024-09-12T14:19:30.461 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.47973.log 2024-09-12T14:19:30.462 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.118299.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.118299.log.gz 2024-09-12T14:19:30.462 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.82738.log 2024-09-12T14:19:30.462 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.47973.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.47973.log.gz 2024-09-12T14:19:30.463 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.84375.log 2024-09-12T14:19:30.463 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.82738.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.82738.log.gz 2024-09-12T14:19:30.463 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.91473.log 2024-09-12T14:19:30.464 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.84375.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.84375.log.gz 2024-09-12T14:19:30.464 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.92647.log 2024-09-12T14:19:30.465 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.91473.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.91473.log.gz 2024-09-12T14:19:30.465 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.49871.log 2024-09-12T14:19:30.466 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.92647.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.92647.log.gz 2024-09-12T14:19:30.466 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.151499.log 2024-09-12T14:19:30.467 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.49871.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.49871.log.gz 2024-09-12T14:19:30.467 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.119479.log 2024-09-12T14:19:30.467 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.151499.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.151499.log.gz 2024-09-12T14:19:30.468 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.119569.log 2024-09-12T14:19:30.468 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.119479.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.119479.log.gz 2024-09-12T14:19:30.469 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.40221.log 2024-09-12T14:19:30.469 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.119569.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.119569.log.gz 2024-09-12T14:19:30.470 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.53355.log 2024-09-12T14:19:30.470 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.40221.log: 0.0%gzip -- replaced with /var/log/ceph/cluster1-client.mirror.40221.log.gz 2024-09-12T14:19:30.470 INFO:teuthology.orchestra.run.smithi130.stderr: -5 --verbose -- /var/log/ceph/cluster1-client.mirror.153142.log 2024-09-12T14:19:30.470 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.53355.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.53355.log.gz 2024-09-12T14:19:30.471 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.134656.log 2024-09-12T14:19:30.471 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.153142.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.153142.log.gz 2024-09-12T14:19:30.471 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.36356.log 2024-09-12T14:19:30.472 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.134656.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.134656.log.gz 2024-09-12T14:19:30.472 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.107171.log 2024-09-12T14:19:30.473 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.36356.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.36356.log.gz 2024-09-12T14:19:30.473 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.51319.log 2024-09-12T14:19:30.474 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.107171.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.107171.log.gz 2024-09-12T14:19:30.474 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.135561.log 2024-09-12T14:19:30.474 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.51319.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.51319.log.gz 2024-09-12T14:19:30.475 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.120567.log 2024-09-12T14:19:30.475 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.135561.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.135561.log.gz 2024-09-12T14:19:30.475 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.86238.log 2024-09-12T14:19:30.476 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.120567.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.120567.log.gz 2024-09-12T14:19:30.476 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.130817.log 2024-09-12T14:19:30.477 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.86238.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.86238.log.gz 2024-09-12T14:19:30.477 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.49513.log 2024-09-12T14:19:30.478 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.130817.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.130817.log.gz 2024-09-12T14:19:30.478 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.127345.log 2024-09-12T14:19:30.478 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.49513.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.49513.log.gz 2024-09-12T14:19:30.479 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.65328.log 2024-09-12T14:19:30.479 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.127345.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.127345.log.gz 2024-09-12T14:19:30.479 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.70097.log 2024-09-12T14:19:30.480 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.65328.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.65328.log.gz 2024-09-12T14:19:30.480 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.107143.log 2024-09-12T14:19:30.482 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.70097.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.70097.log.gz 2024-09-12T14:19:30.482 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.51530.log 2024-09-12T14:19:30.482 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.107143.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.107143.log.gz 2024-09-12T14:19:30.482 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.132873.log 2024-09-12T14:19:30.483 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.51530.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.51530.log.gz 2024-09-12T14:19:30.483 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.77598.log 2024-09-12T14:19:30.484 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.132873.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.132873.log.gz 2024-09-12T14:19:30.484 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.35058.log 2024-09-12T14:19:30.485 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.77598.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.77598.log.gz 2024-09-12T14:19:30.485 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.108354.log 2024-09-12T14:19:30.485 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.35058.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.35058.log.gz 2024-09-12T14:19:30.486 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.152280.log 2024-09-12T14:19:30.487 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.108354.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.108354.log.gz 2024-09-12T14:19:30.487 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.38115.log 2024-09-12T14:19:30.487 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.152280.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.152280.log.gz 2024-09-12T14:19:30.487 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.46972.log 2024-09-12T14:19:30.488 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.38115.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.38115.log.gz 2024-09-12T14:19:30.488 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.91275.log 2024-09-12T14:19:30.488 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.46972.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.46972.log.gz 2024-09-12T14:19:30.489 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.132139.log 2024-09-12T14:19:30.490 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.91275.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.91275.log.gz 2024-09-12T14:19:30.490 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.89454.log 2024-09-12T14:19:30.490 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.132139.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.132139.log.gz 2024-09-12T14:19:30.490 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.106353.log 2024-09-12T14:19:30.491 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.89454.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.89454.log.gz 2024-09-12T14:19:30.492 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.102106.log 2024-09-12T14:19:30.492 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.106353.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.106353.log.gz 2024-09-12T14:19:30.492 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.94669.log 2024-09-12T14:19:30.493 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.102106.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.102106.log.gz 2024-09-12T14:19:30.493 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.41230.log 2024-09-12T14:19:30.493 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.94669.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.94669.log.gz 2024-09-12T14:19:30.493 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.96882.log 2024-09-12T14:19:30.494 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.41230.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.41230.log.gz 2024-09-12T14:19:30.495 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.124188.log 2024-09-12T14:19:30.495 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.96882.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.96882.log.gz 2024-09-12T14:19:30.495 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.126685.log 2024-09-12T14:19:30.496 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.124188.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.124188.log.gz 2024-09-12T14:19:30.496 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.86850.log 2024-09-12T14:19:30.496 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.126685.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.126685.log.gz 2024-09-12T14:19:30.496 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.76908.log 2024-09-12T14:19:30.497 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.86850.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.86850.log.gz 2024-09-12T14:19:30.497 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.150562.log 2024-09-12T14:19:30.498 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.76908.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.115109.log 2024-09-12T14:19:30.498 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.76908.log.gz 2024-09-12T14:19:30.499 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.150562.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.150562.log.gz 2024-09-12T14:19:30.499 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.148777.log 2024-09-12T14:19:30.499 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.115109.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.115109.log.gz 2024-09-12T14:19:30.500 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.127480.log 2024-09-12T14:19:30.500 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.148777.log: gzip -5 0.0% --verbose -- replaced with /var/log/ceph/cluster1-client.mirror.148777.log.gz -- 2024-09-12T14:19:30.500 INFO:teuthology.orchestra.run.smithi130.stderr: /var/log/ceph/cluster1-client.mirror.62642.log 2024-09-12T14:19:30.501 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.127480.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.127480.log.gz 2024-09-12T14:19:30.501 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.142247.log 2024-09-12T14:19:30.502 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.62642.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.62642.log.gz 2024-09-12T14:19:30.502 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.61361.log 2024-09-12T14:19:30.503 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.142247.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.142247.log.gz 2024-09-12T14:19:30.503 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.134067.log 2024-09-12T14:19:30.503 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.61361.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.61361.log.gz 2024-09-12T14:19:30.504 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.73189.log 2024-09-12T14:19:30.504 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.134067.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.134067.log.gz 2024-09-12T14:19:30.505 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.147078.log 2024-09-12T14:19:30.506 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.73189.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.73189.log.gz 2024-09-12T14:19:30.506 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.110160.log 2024-09-12T14:19:30.506 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.147078.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.147078.log.gz 2024-09-12T14:19:30.506 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.60410.log 2024-09-12T14:19:30.507 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.110160.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.110160.log.gz 2024-09-12T14:19:30.507 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.146869.log 2024-09-12T14:19:30.507 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.60410.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.60410.log.gz 2024-09-12T14:19:30.507 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.148040.log 2024-09-12T14:19:30.508 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.146869.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.146869.log.gz 2024-09-12T14:19:30.508 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.129731.log 2024-09-12T14:19:30.509 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.148040.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.148040.log.gz 2024-09-12T14:19:30.509 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.50835.log 2024-09-12T14:19:30.510 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.129731.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.129731.log.gz 2024-09-12T14:19:30.510 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.148398.log 2024-09-12T14:19:30.511 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.50835.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.50835.log.gz 2024-09-12T14:19:30.511 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.125352.log 2024-09-12T14:19:30.511 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.148398.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.148398.log.gz 2024-09-12T14:19:30.512 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.70994.log 2024-09-12T14:19:30.513 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.125352.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.125352.log.gz 2024-09-12T14:19:30.513 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.90831.log 2024-09-12T14:19:30.513 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.70994.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.70994.log.gz 2024-09-12T14:19:30.513 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.113862.log 2024-09-12T14:19:30.514 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.90831.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.90831.log.gz 2024-09-12T14:19:30.515 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.86433.log 2024-09-12T14:19:30.515 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.113862.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.113862.log.gz 2024-09-12T14:19:30.515 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.81543.log 2024-09-12T14:19:30.516 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.86433.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.86433.log.gz 2024-09-12T14:19:30.516 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.101335.log 2024-09-12T14:19:30.516 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.81543.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.81543.log.gz 2024-09-12T14:19:30.517 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.79753.log 2024-09-12T14:19:30.517 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.101335.log: 0.0%gzip -- replaced with /var/log/ceph/cluster1-client.mirror.101335.log.gz 2024-09-12T14:19:30.517 INFO:teuthology.orchestra.run.smithi130.stderr: -5 --verbose -- /var/log/ceph/cluster1-client.admin.117240.log 2024-09-12T14:19:30.519 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.79753.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.79753.log.gz 2024-09-12T14:19:30.519 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.125494.log 2024-09-12T14:19:30.519 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.117240.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.117240.log.gz 2024-09-12T14:19:30.519 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.42558.log 2024-09-12T14:19:30.520 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.125494.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.125494.log.gz 2024-09-12T14:19:30.520 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.65121.log 2024-09-12T14:19:30.520 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.42558.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.42558.log.gz 2024-09-12T14:19:30.521 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.114690.log 2024-09-12T14:19:30.522 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.65121.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.65121.log.gz 2024-09-12T14:19:30.522 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.122749.log 2024-09-12T14:19:30.522 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.114690.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.114690.log.gz 2024-09-12T14:19:30.522 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.62726.log 2024-09-12T14:19:30.523 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.122749.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.122749.log.gz 2024-09-12T14:19:30.523 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.153048.log 2024-09-12T14:19:30.524 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.62726.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.62726.log.gz 2024-09-12T14:19:30.524 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.37406.log 2024-09-12T14:19:30.524 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.153048.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.153048.log.gz 2024-09-12T14:19:30.525 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.35400.log 2024-09-12T14:19:30.525 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.37406.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.37406.log.gz 2024-09-12T14:19:30.525 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.82543.log 2024-09-12T14:19:30.526 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.35400.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.35400.log.gz 2024-09-12T14:19:30.526 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.147179.log 2024-09-12T14:19:30.527 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.82543.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.82543.log.gz 2024-09-12T14:19:30.528 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.64155.log 2024-09-12T14:19:30.528 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.147179.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.147179.log.gz 2024-09-12T14:19:30.528 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.119642.log 2024-09-12T14:19:30.528 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.64155.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.64155.log.gz 2024-09-12T14:19:30.529 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.80423.log 2024-09-12T14:19:30.529 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.119642.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.119642.log.gz 2024-09-12T14:19:30.529 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.41689.log 2024-09-12T14:19:30.530 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.80423.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.80423.log.gz 2024-09-12T14:19:30.530 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.131510.log 2024-09-12T14:19:30.531 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.41689.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.41689.log.gz 2024-09-12T14:19:30.531 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.137139.log 2024-09-12T14:19:30.532 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.131510.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.131510.log.gz 2024-09-12T14:19:30.532 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.99158.log 2024-09-12T14:19:30.533 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.137139.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.137139.log.gz 2024-09-12T14:19:30.533 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.87944.log 2024-09-12T14:19:30.533 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.99158.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.99158.log.gz 2024-09-12T14:19:30.533 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.121093.log 2024-09-12T14:19:30.534 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.87944.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.87944.log.gz 2024-09-12T14:19:30.534 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.80656.log 2024-09-12T14:19:30.535 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.121093.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.121093.log.gz 2024-09-12T14:19:30.535 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.61150.log 2024-09-12T14:19:30.536 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.80656.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.80656.log.gz 2024-09-12T14:19:30.536 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.96423.log 2024-09-12T14:19:30.537 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.61150.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.61150.log.gz 2024-09-12T14:19:30.537 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.106294.log 2024-09-12T14:19:30.537 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.96423.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.96423.log.gz 2024-09-12T14:19:30.537 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.110727.log 2024-09-12T14:19:30.538 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.106294.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.106294.log.gz 2024-09-12T14:19:30.538 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.89393.log 2024-09-12T14:19:30.539 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.110727.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.110727.log.gz 2024-09-12T14:19:30.539 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.135413.log 2024-09-12T14:19:30.540 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.89393.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.89393.log.gz 2024-09-12T14:19:30.540 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.151043.log 2024-09-12T14:19:30.541 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.135413.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.135413.log.gz 2024-09-12T14:19:30.541 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.150783.log 2024-09-12T14:19:30.541 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.151043.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.151043.log.gz 2024-09-12T14:19:30.541 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.122437.log 2024-09-12T14:19:30.542 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.150783.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.150783.log.gz 2024-09-12T14:19:30.542 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.148989.log 2024-09-12T14:19:30.543 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.122437.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.122437.log.gz 2024-09-12T14:19:30.543 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.103443.log 2024-09-12T14:19:30.544 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.148989.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.148989.log.gz 2024-09-12T14:19:30.544 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.58223.log 2024-09-12T14:19:30.545 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.103443.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.103443.log.gz 2024-09-12T14:19:30.545 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.44425.log 2024-09-12T14:19:30.545 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.58223.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.58223.log.gz 2024-09-12T14:19:30.545 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.46583.log 2024-09-12T14:19:30.547 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.44425.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.44425.log.gz 2024-09-12T14:19:30.547 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.83477.log 2024-09-12T14:19:30.547 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.46583.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.46583.log.gz 2024-09-12T14:19:30.547 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.102966.log 2024-09-12T14:19:30.548 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.83477.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.83477.log.gz 2024-09-12T14:19:30.548 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.59606.log 2024-09-12T14:19:30.548 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.102966.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.102966.log.gz 2024-09-12T14:19:30.548 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.118431.log 2024-09-12T14:19:30.549 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.59606.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.59606.log.gz 2024-09-12T14:19:30.549 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.120271.log 2024-09-12T14:19:30.550 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.118431.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.118431.log.gz 2024-09-12T14:19:30.550 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.63313.log 2024-09-12T14:19:30.551 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.120271.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.120271.log.gz 2024-09-12T14:19:30.551 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.53274.log 2024-09-12T14:19:30.552 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.63313.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.63313.log.gz 2024-09-12T14:19:30.552 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.146658.log 2024-09-12T14:19:30.552 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.53274.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.53274.log.gz 2024-09-12T14:19:30.552 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.146111.log 2024-09-12T14:19:30.553 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.146658.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.146658.log.gz 2024-09-12T14:19:30.553 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.40998.log 2024-09-12T14:19:30.554 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.146111.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.146111.log.gz 2024-09-12T14:19:30.554 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.123702.log 2024-09-12T14:19:30.555 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.40998.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.40998.log.gz 2024-09-12T14:19:30.556 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.83630.log 2024-09-12T14:19:30.556 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.123702.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.123702.log.gz 2024-09-12T14:19:30.556 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.48633.log 2024-09-12T14:19:30.557 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.83630.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.83630.log.gz 2024-09-12T14:19:30.557 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.89146.log 2024-09-12T14:19:30.557 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.48633.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.48633.log.gz 2024-09-12T14:19:30.557 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.40304.log 2024-09-12T14:19:30.559 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.89146.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.89146.log.gz 2024-09-12T14:19:30.559 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.54743.log 2024-09-12T14:19:30.559 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.40304.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.40304.log.gz 2024-09-12T14:19:30.559 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.95845.log 2024-09-12T14:19:30.560 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.54743.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.54743.log.gz 2024-09-12T14:19:30.560 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.134363.log 2024-09-12T14:19:30.560 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.95845.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.95845.log.gz 2024-09-12T14:19:30.560 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.35916.log 2024-09-12T14:19:30.562 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.134363.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.134363.log.gz 2024-09-12T14:19:30.562 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.64008.log 2024-09-12T14:19:30.562 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.35916.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.35916.log.gz 2024-09-12T14:19:30.562 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.53883.log 2024-09-12T14:19:30.563 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.64008.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.64008.log.gz 2024-09-12T14:19:30.563 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.133043.log 2024-09-12T14:19:30.564 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.53883.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.53883.log.gz 2024-09-12T14:19:30.564 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.144398.log 2024-09-12T14:19:30.565 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.133043.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.133043.log.gz 2024-09-12T14:19:30.565 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.101942.log 2024-09-12T14:19:30.565 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.144398.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.144398.log.gz 2024-09-12T14:19:30.565 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.80446.log 2024-09-12T14:19:30.567 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.101942.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.101942.log.gz 2024-09-12T14:19:30.567 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.74699.log 2024-09-12T14:19:30.567 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.80446.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.80446.log.gz 2024-09-12T14:19:30.567 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.76486.log 2024-09-12T14:19:30.568 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.74699.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.74699.log.gz 2024-09-12T14:19:30.568 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.116371.log 2024-09-12T14:19:30.569 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.76486.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.76486.log.gz 2024-09-12T14:19:30.569 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.39192.log 2024-09-12T14:19:30.570 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.116371.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.116371.log.gz 2024-09-12T14:19:30.570 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.81631.log 2024-09-12T14:19:30.570 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.39192.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.39192.log.gz 2024-09-12T14:19:30.570 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.60731.log 2024-09-12T14:19:30.571 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.81631.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.81631.log.gz 2024-09-12T14:19:30.571 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.36591.log 2024-09-12T14:19:30.572 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.60731.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.60731.log.gz 2024-09-12T14:19:30.572 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.111569.log 2024-09-12T14:19:30.573 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.36591.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.36591.log.gz 2024-09-12T14:19:30.573 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.51110.log 2024-09-12T14:19:30.573 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.111569.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.111569.log.gz 2024-09-12T14:19:30.573 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.153732.log 2024-09-12T14:19:30.574 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.51110.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.51110.log.gz 2024-09-12T14:19:30.574 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.109926.log 2024-09-12T14:19:30.575 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.153732.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.153732.log.gz 2024-09-12T14:19:30.575 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.48021.log 2024-09-12T14:19:30.576 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.109926.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.109926.log.gz 2024-09-12T14:19:30.576 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.109191.log 2024-09-12T14:19:30.576 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.48021.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.48021.log.gz 2024-09-12T14:19:30.577 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.49597.log 2024-09-12T14:19:30.577 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.109191.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.109191.log.gz 2024-09-12T14:19:30.577 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.76152.log 2024-09-12T14:19:30.578 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.49597.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.49597.log.gz 2024-09-12T14:19:30.578 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.38412.log 2024-09-12T14:19:30.579 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.76152.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.76152.log.gz 2024-09-12T14:19:30.579 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.54900.log 2024-09-12T14:19:30.580 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.38412.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.38412.log.gz 2024-09-12T14:19:30.580 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.29400.log 2024-09-12T14:19:30.581 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.54900.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.54900.log.gz 2024-09-12T14:19:30.581 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.82934.log 2024-09-12T14:19:30.582 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.29400.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.29400.log.gz 2024-09-12T14:19:30.582 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.119332.log 2024-09-12T14:19:30.583 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.82934.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.82934.log.gz 2024-09-12T14:19:30.583 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.67189.log 2024-09-12T14:19:30.583 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.119332.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.119332.log.gz 2024-09-12T14:19:30.583 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.149539.log 2024-09-12T14:19:30.584 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.67189.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.67189.log.gz 2024-09-12T14:19:30.585 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.73520.log 2024-09-12T14:19:30.585 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.149539.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.149539.log.gz 2024-09-12T14:19:30.585 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.59226.log 2024-09-12T14:19:30.586 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.73520.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.73520.log.gz 2024-09-12T14:19:30.586 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.37238.log 2024-09-12T14:19:30.586 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.59226.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.59226.log.gz 2024-09-12T14:19:30.586 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.82247.log 2024-09-12T14:19:30.588 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.37238.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.37238.log.gz 2024-09-12T14:19:30.588 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.58089.log 2024-09-12T14:19:30.588 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.82247.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.82247.log.gz 2024-09-12T14:19:30.588 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.40261.log 2024-09-12T14:19:30.589 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.58089.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.58089.log.gz 2024-09-12T14:19:30.592 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.111229.log 2024-09-12T14:19:30.592 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.40261.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.40261.log.gz 2024-09-12T14:19:30.597 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.65908.log 2024-09-12T14:19:30.597 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.111229.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.111229.log.gz 2024-09-12T14:19:30.597 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.45175.log 2024-09-12T14:19:30.597 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.65908.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.65908.log.gz 2024-09-12T14:19:30.597 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.116894.log 2024-09-12T14:19:30.597 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.45175.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.45175.log.gz 2024-09-12T14:19:30.597 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.109255.log 2024-09-12T14:19:30.597 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.116894.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.116894.log.gz 2024-09-12T14:19:30.597 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.40120.log 2024-09-12T14:19:30.598 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.109255.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.109255.log.gz 2024-09-12T14:19:30.598 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.132704.log 2024-09-12T14:19:30.598 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.40120.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.40120.log.gz 2024-09-12T14:19:30.598 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.68378.log 2024-09-12T14:19:30.598 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.132704.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.132704.log.gz 2024-09-12T14:19:30.598 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.137802.log 2024-09-12T14:19:30.598 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.68378.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.68378.log.gz 2024-09-12T14:19:30.598 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.49521.log 2024-09-12T14:19:30.598 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.137802.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.137802.log.gz 2024-09-12T14:19:30.598 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.40220.log 2024-09-12T14:19:30.602 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.49521.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.49521.log.gz 2024-09-12T14:19:30.602 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.36417.log 2024-09-12T14:19:30.602 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.40220.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.40220.log.gz 2024-09-12T14:19:30.602 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.56752.log 2024-09-12T14:19:30.602 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.36417.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.36417.log.gz 2024-09-12T14:19:30.602 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.74449.log 2024-09-12T14:19:30.602 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.56752.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.56752.log.gz 2024-09-12T14:19:30.602 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.71716.log 2024-09-12T14:19:30.602 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.74449.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.74449.log.gz 2024-09-12T14:19:30.603 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.134576.log 2024-09-12T14:19:30.603 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.71716.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.71716.log.gz 2024-09-12T14:19:30.603 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.138155.log 2024-09-12T14:19:30.605 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.134576.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.134576.log.gz 2024-09-12T14:19:30.605 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.125304.log 2024-09-12T14:19:30.605 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.138155.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.138155.log.gz 2024-09-12T14:19:30.605 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.85001.log 2024-09-12T14:19:30.605 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.125304.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.125304.log.gz 2024-09-12T14:19:30.606 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.145881.log 2024-09-12T14:19:30.606 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.85001.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.85001.log.gz 2024-09-12T14:19:30.606 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.103463.log 2024-09-12T14:19:30.607 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.145881.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.145881.log.gz 2024-09-12T14:19:30.607 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.108145.log 2024-09-12T14:19:30.607 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.103463.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.103463.log.gz 2024-09-12T14:19:30.607 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.86640.log 2024-09-12T14:19:30.607 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.108145.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.108145.log.gz 2024-09-12T14:19:30.608 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.147708.log 2024-09-12T14:19:30.609 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.86640.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.86640.log.gz 2024-09-12T14:19:30.609 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.46402.log 2024-09-12T14:19:30.610 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.147708.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.147708.log.gz 2024-09-12T14:19:30.610 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.81718.log 2024-09-12T14:19:30.610 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.46402.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.46402.log.gz 2024-09-12T14:19:30.610 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.43918.log 2024-09-12T14:19:30.611 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.81718.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.81718.log.gz 2024-09-12T14:19:30.612 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.120919.log 2024-09-12T14:19:30.612 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.43918.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.43918.log.gz 2024-09-12T14:19:30.612 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.77913.log 2024-09-12T14:19:30.613 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.120919.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.120919.log.gz 2024-09-12T14:19:30.613 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.84625.log 2024-09-12T14:19:30.613 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.77913.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.77913.log.gz 2024-09-12T14:19:30.613 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.83240.log 2024-09-12T14:19:30.615 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.84625.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.84625.log.gz 2024-09-12T14:19:30.615 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.121006.log 2024-09-12T14:19:30.615 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.83240.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.83240.log.gz 2024-09-12T14:19:30.615 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.34559.log 2024-09-12T14:19:30.616 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.121006.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.121006.log.gz 2024-09-12T14:19:30.616 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.144337.log 2024-09-12T14:19:30.617 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.34559.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.34559.log.gz 2024-09-12T14:19:30.617 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.131012.log 2024-09-12T14:19:30.618 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.144337.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.144337.log.gz 2024-09-12T14:19:30.618 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.35493.log 2024-09-12T14:19:30.618 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.131012.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.131012.log.gz 2024-09-12T14:19:30.618 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.46428.log 2024-09-12T14:19:30.619 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.35493.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.35493.log.gz 2024-09-12T14:19:30.619 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.40957.log 2024-09-12T14:19:30.620 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.46428.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.46428.log.gz 2024-09-12T14:19:30.620 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.79378.log 2024-09-12T14:19:30.621 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.40957.log: 0.0%gzip -- replaced with /var/log/ceph/cluster1-client.mirror.40957.log.gz 2024-09-12T14:19:30.621 INFO:teuthology.orchestra.run.smithi130.stderr: -5 --verbose -- /var/log/ceph/cluster1-client.mirror.87836.log 2024-09-12T14:19:30.621 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.79378.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.79378.log.gz 2024-09-12T14:19:30.621 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.92470.log 2024-09-12T14:19:30.622 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.87836.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.87836.log.gz 2024-09-12T14:19:30.623 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.62871.log 2024-09-12T14:19:30.623 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.92470.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.92470.log.gz 2024-09-12T14:19:30.623 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.79868.log 2024-09-12T14:19:30.624 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.62871.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.62871.log.gz 2024-09-12T14:19:30.624 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.109086.log 2024-09-12T14:19:30.625 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.79868.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.79868.log.gz 2024-09-12T14:19:30.625 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.98081.log 2024-09-12T14:19:30.625 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.109086.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.109086.log.gz 2024-09-12T14:19:30.625 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.137097.log 2024-09-12T14:19:30.627 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.98081.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.98081.log.gz 2024-09-12T14:19:30.627 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.51864.log 2024-09-12T14:19:30.627 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.137097.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.137097.log.gz 2024-09-12T14:19:30.627 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.140433.log 2024-09-12T14:19:30.628 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.51864.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.51864.log.gz 2024-09-12T14:19:30.628 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.92841.log 2024-09-12T14:19:30.629 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.140433.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.140433.log.gz 2024-09-12T14:19:30.629 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.100226.log 2024-09-12T14:19:30.630 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.92841.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.92841.log.gz 2024-09-12T14:19:30.630 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.94254.log 2024-09-12T14:19:30.630 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.100226.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.100226.log.gz 2024-09-12T14:19:30.630 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.66746.log 2024-09-12T14:19:30.631 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.94254.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.94254.log.gz 2024-09-12T14:19:30.632 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.146868.log 2024-09-12T14:19:30.632 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.66746.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.66746.log.gz 2024-09-12T14:19:30.632 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.39379.log 2024-09-12T14:19:30.633 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.146868.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.146868.log.gz 2024-09-12T14:19:30.633 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.107932.log 2024-09-12T14:19:30.633 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.39379.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.39379.log.gz 2024-09-12T14:19:30.633 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.35954.log 2024-09-12T14:19:30.635 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.107932.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.107932.log.gz 2024-09-12T14:19:30.635 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.109823.log 2024-09-12T14:19:30.635 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.35954.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.35954.log.gz 2024-09-12T14:19:30.635 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.51538.log 2024-09-12T14:19:30.636 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.109823.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.109823.log.gz 2024-09-12T14:19:30.636 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.119180.log 2024-09-12T14:19:30.637 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.51538.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.51538.log.gz 2024-09-12T14:19:30.637 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.95545.log 2024-09-12T14:19:30.638 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.119180.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.119180.log.gz 2024-09-12T14:19:30.638 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.133796.log 2024-09-12T14:19:30.638 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.95545.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.95545.log.gz 2024-09-12T14:19:30.638 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.103523.log 2024-09-12T14:19:30.639 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.133796.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.133796.log.gz 2024-09-12T14:19:30.640 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.51593.log 2024-09-12T14:19:30.640 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.103523.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.103523.log.gz 2024-09-12T14:19:30.640 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.51026.log 2024-09-12T14:19:30.641 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.51593.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.51593.log.gz 2024-09-12T14:19:30.641 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.152694.log 2024-09-12T14:19:30.641 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.51026.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.51026.log.gz 2024-09-12T14:19:30.641 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.81478.log 2024-09-12T14:19:30.642 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.152694.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.152694.log.gz 2024-09-12T14:19:30.643 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.66216.log 2024-09-12T14:19:30.643 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.81478.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.81478.log.gz 2024-09-12T14:19:30.643 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.67716.log 2024-09-12T14:19:30.644 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.66216.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.66216.log.gz 2024-09-12T14:19:30.644 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.107412.log 2024-09-12T14:19:30.644 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.67716.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.67716.log.gz 2024-09-12T14:19:30.644 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.112300.log 2024-09-12T14:19:30.645 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.107412.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.107412.log.gz 2024-09-12T14:19:30.645 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.104827.log 2024-09-12T14:19:30.646 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.112300.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.112300.log.gz 2024-09-12T14:19:30.646 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.75893.log 2024-09-12T14:19:30.647 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.104827.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.104827.log.gz 2024-09-12T14:19:30.647 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.102583.log 2024-09-12T14:19:30.648 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.75893.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.75893.log.gz 2024-09-12T14:19:30.648 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.81999.log 2024-09-12T14:19:30.649 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.102583.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.102583.log.gz 2024-09-12T14:19:30.649 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.118809.log 2024-09-12T14:19:30.649 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.81999.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.81999.log.gz 2024-09-12T14:19:30.649 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.139899.log 2024-09-12T14:19:30.650 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.118809.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.118809.log.gz 2024-09-12T14:19:30.650 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.107059.log 2024-09-12T14:19:30.651 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.139899.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.139899.log.gz 2024-09-12T14:19:30.651 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.114149.log 2024-09-12T14:19:30.652 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.107059.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.107059.log.gz 2024-09-12T14:19:30.652 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.78477.log 2024-09-12T14:19:30.653 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.114149.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.114149.log.gz 2024-09-12T14:19:30.653 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.82113.log 2024-09-12T14:19:30.653 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.78477.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.78477.log.gz 2024-09-12T14:19:30.653 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.79105.log 2024-09-12T14:19:30.654 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.82113.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.82113.log.gz 2024-09-12T14:19:30.654 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.138587.log 2024-09-12T14:19:30.655 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.79105.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.79105.log.gz 2024-09-12T14:19:30.655 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.115326.log 2024-09-12T14:19:30.656 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.138587.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.138587.log.gz 2024-09-12T14:19:30.657 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.148714.log 2024-09-12T14:19:30.657 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.115326.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.115326.log.gz 2024-09-12T14:19:30.657 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.56739.log 2024-09-12T14:19:30.657 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.148714.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.148714.log.gz 2024-09-12T14:19:30.658 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.143873.log 2024-09-12T14:19:30.659 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.56739.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.56739.log.gz 2024-09-12T14:19:30.659 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.39909.log 2024-09-12T14:19:30.659 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.143873.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.143873.log.gz 2024-09-12T14:19:30.659 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.104447.log 2024-09-12T14:19:30.661 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.39909.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.39909.log.gz 2024-09-12T14:19:30.661 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.90257.log 2024-09-12T14:19:30.661 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.104447.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.104447.log.gz 2024-09-12T14:19:30.661 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.129430.log 2024-09-12T14:19:30.662 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.90257.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.90257.log.gz 2024-09-12T14:19:30.662 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.124880.log 2024-09-12T14:19:30.662 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.129430.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.129430.log.gz 2024-09-12T14:19:30.662 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.63650.log 2024-09-12T14:19:30.663 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.124880.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.124880.log.gz 2024-09-12T14:19:30.664 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.71476.log 2024-09-12T14:19:30.664 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.63650.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.63650.log.gz 2024-09-12T14:19:30.664 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.153215.log 2024-09-12T14:19:30.664 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.71476.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.71476.log.gz 2024-09-12T14:19:30.665 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.57796.log 2024-09-12T14:19:30.665 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.153215.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.153215.log.gz 2024-09-12T14:19:30.665 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.78602.log 2024-09-12T14:19:30.666 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.57796.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.57796.log.gz 2024-09-12T14:19:30.666 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.128524.log 2024-09-12T14:19:30.667 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.78602.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.78602.log.gz 2024-09-12T14:19:30.668 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.147812.log 2024-09-12T14:19:30.668 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.128524.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.128524.log.gz 2024-09-12T14:19:30.668 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.45228.log 2024-09-12T14:19:30.668 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.147812.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.147812.log.gz 2024-09-12T14:19:30.669 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.115924.log 2024-09-12T14:19:30.669 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.45228.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.45228.log.gz 2024-09-12T14:19:30.669 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.136361.log 2024-09-12T14:19:30.670 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.115924.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.115924.log.gz 2024-09-12T14:19:30.670 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.44197.log 2024-09-12T14:19:30.671 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.136361.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.136361.log.gz 2024-09-12T14:19:30.671 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.53062.log 2024-09-12T14:19:30.672 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.44197.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.44197.log.gz 2024-09-12T14:19:30.672 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.48803.log 2024-09-12T14:19:30.673 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.53062.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.53062.log.gz 2024-09-12T14:19:30.673 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.132348.log 2024-09-12T14:19:30.673 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.48803.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.48803.log.gz 2024-09-12T14:19:30.674 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.78386.log 2024-09-12T14:19:30.674 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.132348.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.132348.log.gz 2024-09-12T14:19:30.674 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.114801.log 2024-09-12T14:19:30.675 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.78386.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.78386.log.gz 2024-09-12T14:19:30.675 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.92061.log 2024-09-12T14:19:30.676 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.114801.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.114801.log.gz 2024-09-12T14:19:30.676 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.74992.log 2024-09-12T14:19:30.677 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.92061.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.92061.log.gz 2024-09-12T14:19:30.677 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.49177.log 2024-09-12T14:19:30.677 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.74992.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.74992.log.gz 2024-09-12T14:19:30.678 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.72748.log 2024-09-12T14:19:30.678 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.49177.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.49177.log.gz 2024-09-12T14:19:30.679 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.133942.log 2024-09-12T14:19:30.679 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.72748.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.72748.log.gz 2024-09-12T14:19:30.680 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.144981.log 2024-09-12T14:19:30.680 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.133942.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.133942.log.gz 2024-09-12T14:19:30.680 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.133545.log 2024-09-12T14:19:30.681 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.144981.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.144981.log.gz 2024-09-12T14:19:30.681 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.119788.log 2024-09-12T14:19:30.681 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.133545.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.133545.log.gz 2024-09-12T14:19:30.682 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.126022.log 2024-09-12T14:19:30.683 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.119788.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.119788.log.gz 2024-09-12T14:19:30.683 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.80889.log 2024-09-12T14:19:30.683 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.126022.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.126022.log.gz 2024-09-12T14:19:30.683 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.54709.log 2024-09-12T14:19:30.685 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.80889.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.80889.log.gz 2024-09-12T14:19:30.685 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.58451.log 2024-09-12T14:19:30.685 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.54709.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.54709.log.gz 2024-09-12T14:19:30.685 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.61005.log 2024-09-12T14:19:30.686 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.58451.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.58451.log.gz 2024-09-12T14:19:30.686 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.80158.log 2024-09-12T14:19:30.686 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.61005.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.61005.log.gz 2024-09-12T14:19:30.687 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.71080.log 2024-09-12T14:19:30.688 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.80158.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.80158.log.gz 2024-09-12T14:19:30.688 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.132224.log 2024-09-12T14:19:30.688 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.71080.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.71080.log.gz 2024-09-12T14:19:30.688 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.104777.log 2024-09-12T14:19:30.689 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.132224.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.132224.log.gz 2024-09-12T14:19:30.689 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.42346.log 2024-09-12T14:19:30.690 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.104777.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.104777.log.gz 2024-09-12T14:19:30.690 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.109402.log 2024-09-12T14:19:30.691 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.42346.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.42346.log.gz 2024-09-12T14:19:30.691 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.138565.log 2024-09-12T14:19:30.691 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.109402.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.109402.log.gz 2024-09-12T14:19:30.691 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.112553.log 2024-09-12T14:19:30.693 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.138565.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.138565.log.gz 2024-09-12T14:19:30.693 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.151352.log 2024-09-12T14:19:30.693 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.112553.log: 0.0%gzip -- replaced with /var/log/ceph/cluster1-client.mirror.112553.log.gz 2024-09-12T14:19:30.693 INFO:teuthology.orchestra.run.smithi130.stderr: -5 --verbose -- /var/log/ceph/cluster1-client.mirror.88973.log 2024-09-12T14:19:30.694 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.151352.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.151352.log.gz 2024-09-12T14:19:30.694 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.37697.log 2024-09-12T14:19:30.694 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.88973.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.88973.log.gz 2024-09-12T14:19:30.694 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.47018.log 2024-09-12T14:19:30.695 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.37697.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.37697.log.gz 2024-09-12T14:19:30.695 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.64092.log 2024-09-12T14:19:30.696 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.47018.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.47018.log.gz 2024-09-12T14:19:30.696 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.64990.log 2024-09-12T14:19:30.697 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.64092.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.64092.log.gz 2024-09-12T14:19:30.697 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.101233.log 2024-09-12T14:19:30.698 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.64990.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.64990.log.gz 2024-09-12T14:19:30.698 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.153598.log 2024-09-12T14:19:30.698 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.101233.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.101233.log.gz 2024-09-12T14:19:30.699 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.143491.log 2024-09-12T14:19:30.699 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.153598.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.153598.log.gz 2024-09-12T14:19:30.699 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.82288.log 2024-09-12T14:19:30.700 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.143491.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.143491.log.gz 2024-09-12T14:19:30.700 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.133165.log 2024-09-12T14:19:30.701 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.82288.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.69915.log 2024-09-12T14:19:30.701 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.82288.log.gz 2024-09-12T14:19:30.702 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.133165.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.133165.log.gz 2024-09-12T14:19:30.702 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.135498.log 2024-09-12T14:19:30.703 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.69915.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.69915.log.gz 2024-09-12T14:19:30.703 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.146087.log 2024-09-12T14:19:30.703 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.135498.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.135498.log.gz 2024-09-12T14:19:30.703 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.69589.log 2024-09-12T14:19:30.704 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.146087.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.146087.log.gz 2024-09-12T14:19:30.704 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.98377.log 2024-09-12T14:19:30.705 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.69589.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.69589.log.gz 2024-09-12T14:19:30.705 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.107995.log 2024-09-12T14:19:30.706 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.98377.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.98377.log.gz 2024-09-12T14:19:30.706 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.108961.log 2024-09-12T14:19:30.706 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.107995.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.107995.log.gz 2024-09-12T14:19:30.707 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.69614.log 2024-09-12T14:19:30.707 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.108961.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.108961.log.gz 2024-09-12T14:19:30.707 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.118132.log 2024-09-12T14:19:30.708 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.69614.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.69614.log.gz 2024-09-12T14:19:30.708 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.77704.log 2024-09-12T14:19:30.709 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.118132.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.118132.log.gz 2024-09-12T14:19:30.709 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.62434.log 2024-09-12T14:19:30.710 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.77704.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.77704.log.gz 2024-09-12T14:19:30.710 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.132349.log 2024-09-12T14:19:30.711 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.62434.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.62434.log.gz 2024-09-12T14:19:30.711 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.118262.log 2024-09-12T14:19:30.712 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.132349.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.132349.log.gz 2024-09-12T14:19:30.712 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.72285.log 2024-09-12T14:19:30.712 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.118262.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.118262.log.gz 2024-09-12T14:19:30.712 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.87123.log 2024-09-12T14:19:30.713 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.72285.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.72285.log.gz 2024-09-12T14:19:30.713 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.124711.log 2024-09-12T14:19:30.713 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.87123.log: 0.0%gzip -- replaced with /var/log/ceph/cluster1-client.mirror.87123.log.gz -5 2024-09-12T14:19:30.713 INFO:teuthology.orchestra.run.smithi130.stderr: --verbose -- /var/log/ceph/cluster1-client.mirror.49933.log 2024-09-12T14:19:30.714 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.124711.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.124711.log.gz 2024-09-12T14:19:30.714 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.47168.log 2024-09-12T14:19:30.716 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.49933.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.49933.log.gz 2024-09-12T14:19:30.716 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.88721.log 2024-09-12T14:19:30.716 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.47168.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.47168.log.gz 2024-09-12T14:19:30.716 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.45226.log 2024-09-12T14:19:30.717 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.88721.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.88721.log.gz 2024-09-12T14:19:30.717 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.70900.log 2024-09-12T14:19:30.717 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.45226.log: gzip -5 --verbose -- 0.0% /var/log/ceph/cluster2-client.admin.79828.log 2024-09-12T14:19:30.717 INFO:teuthology.orchestra.run.smithi130.stderr: -- replaced with /var/log/ceph/cluster2-client.admin.45226.log.gz 2024-09-12T14:19:30.718 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.70900.log: gzip 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.70900.log.gz -5 2024-09-12T14:19:30.718 INFO:teuthology.orchestra.run.smithi130.stderr: --verbose -- /var/log/ceph/cluster1-client.mirror.42943.log 2024-09-12T14:19:30.720 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.79828.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.79828.log.gz 2024-09-12T14:19:30.720 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.62510.log 2024-09-12T14:19:30.720 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.42943.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.42943.log.gz 2024-09-12T14:19:30.720 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.119222.log 2024-09-12T14:19:30.721 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.62510.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.62510.log.gz 2024-09-12T14:19:30.721 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.103497.log 2024-09-12T14:19:30.721 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.119222.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.119222.log.gz 2024-09-12T14:19:30.722 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.108986.log 2024-09-12T14:19:30.723 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.103497.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.103497.log.gz 2024-09-12T14:19:30.723 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.39637.log 2024-09-12T14:19:30.723 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.108986.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.108986.log.gz 2024-09-12T14:19:30.723 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.127071.log 2024-09-12T14:19:30.724 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.39637.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.39637.log.gz 2024-09-12T14:19:30.724 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.63023.log 2024-09-12T14:19:30.725 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.127071.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.39945.log 2024-09-12T14:19:30.725 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.127071.log.gz 2024-09-12T14:19:30.725 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.63023.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.63023.log.gz 2024-09-12T14:19:30.725 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.97916.log 2024-09-12T14:19:30.726 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.39945.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.42617.log 2024-09-12T14:19:30.727 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.39945.log.gz 2024-09-12T14:19:30.727 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.97916.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.97916.log.gz 2024-09-12T14:19:30.727 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.52473.log 2024-09-12T14:19:30.728 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.42617.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.42617.log.gz 2024-09-12T14:19:30.728 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.84119.log 2024-09-12T14:19:30.728 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.52473.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.52473.log.gz 2024-09-12T14:19:30.728 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.122226.log 2024-09-12T14:19:30.729 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.84119.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.84119.log.gz 2024-09-12T14:19:30.729 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.42091.log 2024-09-12T14:19:30.730 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.122226.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.122226.log.gz 2024-09-12T14:19:30.730 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.135206.log 2024-09-12T14:19:30.731 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.42091.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.42091.log.gz 2024-09-12T14:19:30.731 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.115327.log 2024-09-12T14:19:30.731 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.135206.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.135206.log.gz 2024-09-12T14:19:30.732 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.90943.log 2024-09-12T14:19:30.732 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.115327.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.115327.log.gz 2024-09-12T14:19:30.732 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.135208.log 2024-09-12T14:19:30.733 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.90943.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.90943.log.gz 2024-09-12T14:19:30.733 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.114192.log 2024-09-12T14:19:30.734 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.135208.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.135208.log.gz 2024-09-12T14:19:30.735 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.64093.log 2024-09-12T14:19:30.735 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.114192.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.114192.log.gz 2024-09-12T14:19:30.735 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.41800.log 2024-09-12T14:19:30.736 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.64093.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.64093.log.gz 2024-09-12T14:19:30.736 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.149561.log 2024-09-12T14:19:30.737 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.41800.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.41800.log.gz 2024-09-12T14:19:30.737 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.77096.log 2024-09-12T14:19:30.737 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.149561.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.149561.log.gz 2024-09-12T14:19:30.737 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.65666.log 2024-09-12T14:19:30.738 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.77096.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.77096.log.gz 2024-09-12T14:19:30.739 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.112802.log 2024-09-12T14:19:30.739 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.65666.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.65666.log.gz 2024-09-12T14:19:30.739 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.136337.log 2024-09-12T14:19:30.740 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.112802.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.112802.log.gz 2024-09-12T14:19:30.740 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.59757.log 2024-09-12T14:19:30.740 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.136337.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.136337.log.gz 2024-09-12T14:19:30.740 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.154467.log 2024-09-12T14:19:30.742 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.59757.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.59757.log.gz 2024-09-12T14:19:30.742 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.136968.log 2024-09-12T14:19:30.742 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.154467.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.154467.log.gz 2024-09-12T14:19:30.742 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.60646.log 2024-09-12T14:19:30.743 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.136968.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.136968.log.gz 2024-09-12T14:19:30.743 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.40832.log 2024-09-12T14:19:30.744 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.60646.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.60646.log.gz 2024-09-12T14:19:30.744 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.46904.log 2024-09-12T14:19:30.744 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.40832.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.40832.log.gz 2024-09-12T14:19:30.744 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.64662.log 2024-09-12T14:19:30.746 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.46904.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.46904.log.gz 2024-09-12T14:19:30.746 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.99067.log 2024-09-12T14:19:30.746 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.64662.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.64662.log.gz 2024-09-12T14:19:30.746 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.58565.log 2024-09-12T14:19:30.747 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.99067.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.99067.log.gz 2024-09-12T14:19:30.747 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.141577.log 2024-09-12T14:19:30.747 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.58565.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.58565.log.gz 2024-09-12T14:19:30.748 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.146299.log 2024-09-12T14:19:30.748 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.141577.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.141577.log.gz 2024-09-12T14:19:30.748 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.53060.log 2024-09-12T14:19:30.749 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.146299.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.146299.log.gz 2024-09-12T14:19:30.749 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.78263.log 2024-09-12T14:19:30.750 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.53060.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.53060.log.gz 2024-09-12T14:19:30.750 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.115267.log 2024-09-12T14:19:30.751 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.78263.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.78263.log.gz 2024-09-12T14:19:30.751 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.55790.log 2024-09-12T14:19:30.752 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.115267.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.115267.log.gz 2024-09-12T14:19:30.752 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.129792.log 2024-09-12T14:19:30.752 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.55790.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.55790.log.gz 2024-09-12T14:19:30.752 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.76444.log 2024-09-12T14:19:30.753 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.129792.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.129792.log.gz 2024-09-12T14:19:30.753 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.113945.log 2024-09-12T14:19:30.754 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.76444.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.76444.log.gz 2024-09-12T14:19:30.754 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.149283.log 2024-09-12T14:19:30.755 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.113945.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.113945.log.gz 2024-09-12T14:19:30.755 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.85799.log 2024-09-12T14:19:30.756 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.149283.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.149283.log.gz 2024-09-12T14:19:30.756 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.75422.log 2024-09-12T14:19:30.756 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.85799.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.85799.log.gz 2024-09-12T14:19:30.756 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.120564.log 2024-09-12T14:19:30.757 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.75422.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.75422.log.gz 2024-09-12T14:19:30.757 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.58016.log 2024-09-12T14:19:30.758 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.120564.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.120564.log.gz 2024-09-12T14:19:30.758 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.81905.log 2024-09-12T14:19:30.759 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.58016.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.58016.log.gz 2024-09-12T14:19:30.759 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.127204.log 2024-09-12T14:19:30.759 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.81905.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.81905.log.gz 2024-09-12T14:19:30.759 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.153162.log 2024-09-12T14:19:30.760 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.127204.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.127204.log.gz 2024-09-12T14:19:30.760 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.141053.log 2024-09-12T14:19:30.761 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.153162.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.153162.log.gz 2024-09-12T14:19:30.761 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.142817.log 2024-09-12T14:19:30.762 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.141053.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.141053.log.gz 2024-09-12T14:19:30.762 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.50250.log 2024-09-12T14:19:30.762 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.142817.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.142817.log.gz 2024-09-12T14:19:30.763 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.139820.log 2024-09-12T14:19:30.763 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.50250.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.50250.log.gz 2024-09-12T14:19:30.764 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.85399.log 2024-09-12T14:19:30.764 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.139820.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.139820.log.gz 2024-09-12T14:19:30.764 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.113099.log 2024-09-12T14:19:30.770 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.85399.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.85399.log.gz 2024-09-12T14:19:30.771 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.68062.log 2024-09-12T14:19:30.771 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.113099.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.113099.log.gz 2024-09-12T14:19:30.771 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.147140.log 2024-09-12T14:19:30.771 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.68062.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.68062.log.gz 2024-09-12T14:19:30.771 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.91959.log 2024-09-12T14:19:30.771 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.147140.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.147140.log.gz 2024-09-12T14:19:30.771 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.92076.log 2024-09-12T14:19:30.771 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.91959.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.91959.log.gz 2024-09-12T14:19:30.771 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.148692.log 2024-09-12T14:19:30.772 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.92076.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.92076.log.gz 2024-09-12T14:19:30.772 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.150128.log 2024-09-12T14:19:30.772 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.148692.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.148692.log.gz 2024-09-12T14:19:30.772 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.47362.log 2024-09-12T14:19:30.772 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.150128.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.150128.log.gz 2024-09-12T14:19:30.772 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.105208.log 2024-09-12T14:19:30.774 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.47362.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.47362.log.gz 2024-09-12T14:19:30.775 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.94465.log 2024-09-12T14:19:30.775 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.105208.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.105208.log.gz 2024-09-12T14:19:30.775 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.96235.log 2024-09-12T14:19:30.775 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.94465.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.94465.log.gz 2024-09-12T14:19:30.775 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.57757.log 2024-09-12T14:19:30.775 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.96235.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.96235.log.gz 2024-09-12T14:19:30.775 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.111494.log 2024-09-12T14:19:30.775 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.57757.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.57757.log.gz 2024-09-12T14:19:30.775 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.46770.log 2024-09-12T14:19:30.777 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.111494.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.111494.log.gz 2024-09-12T14:19:30.777 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.40750.log 2024-09-12T14:19:30.777 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.46770.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.46770.log.gz 2024-09-12T14:19:30.777 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.128154.log 2024-09-12T14:19:30.778 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.40750.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.40750.log.gz 2024-09-12T14:19:30.778 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.142005.log 2024-09-12T14:19:30.779 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.128154.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.128154.log.gz 2024-09-12T14:19:30.779 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.108896.log 2024-09-12T14:19:30.779 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.142005.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.142005.log.gz 2024-09-12T14:19:30.779 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.100392.log 2024-09-12T14:19:30.780 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.108896.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.108896.log.gz 2024-09-12T14:19:30.780 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.148460.log 2024-09-12T14:19:30.781 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.100392.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.100392.log.gz 2024-09-12T14:19:30.781 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.122141.log 2024-09-12T14:19:30.781 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.148460.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.148460.log.gz 2024-09-12T14:19:30.781 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.106094.log 2024-09-12T14:19:30.782 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.122141.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.122141.log.gz 2024-09-12T14:19:30.782 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.83172.log 2024-09-12T14:19:30.783 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.106094.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.106094.log.gz 2024-09-12T14:19:30.783 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.72080.log 2024-09-12T14:19:30.783 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.83172.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.83172.log.gz 2024-09-12T14:19:30.783 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.119395.log 2024-09-12T14:19:30.784 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.72080.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.72080.log.gz 2024-09-12T14:19:30.784 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.60063.log 2024-09-12T14:19:30.785 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.119395.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.119395.log.gz 2024-09-12T14:19:30.785 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.93157.log 2024-09-12T14:19:30.786 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.60063.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.60063.log.gz 2024-09-12T14:19:30.786 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.46516.log 2024-09-12T14:19:30.787 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.93157.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.93157.log.gz 2024-09-12T14:19:30.787 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.72391.log 2024-09-12T14:19:30.787 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.46516.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.46516.log.gz 2024-09-12T14:19:30.787 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.70790.log 2024-09-12T14:19:30.788 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.72391.log: 0.0%gzip -- replaced with /var/log/ceph/cluster1-client.mirror.72391.log.gz 2024-09-12T14:19:30.788 INFO:teuthology.orchestra.run.smithi130.stderr: -5 --verbose -- /var/log/ceph/cluster1-client.admin.70439.log 2024-09-12T14:19:30.789 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.70790.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.70790.log.gz 2024-09-12T14:19:30.789 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.122690.log 2024-09-12T14:19:30.790 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.70439.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.70439.log.gz 2024-09-12T14:19:30.790 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.36420.log 2024-09-12T14:19:30.791 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.122690.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.122690.log.gz 2024-09-12T14:19:30.791 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.66340.log 2024-09-12T14:19:30.792 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.36420.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.36420.log.gz 2024-09-12T14:19:30.792 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.58244.log 2024-09-12T14:19:30.792 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.66340.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.66340.log.gz 2024-09-12T14:19:30.792 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.103659.log 2024-09-12T14:19:30.794 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.58244.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.58244.log.gz 2024-09-12T14:19:30.794 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.42347.log 2024-09-12T14:19:30.794 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.103659.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.103659.log.gz 2024-09-12T14:19:30.794 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.38796.log 2024-09-12T14:19:30.795 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.42347.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.42347.log.gz 2024-09-12T14:19:30.796 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.82060.log 2024-09-12T14:19:30.796 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.38796.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.38796.log.gz 2024-09-12T14:19:30.796 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.88696.log 2024-09-12T14:19:30.797 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.82060.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.82060.log.gz 2024-09-12T14:19:30.797 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.105933.log 2024-09-12T14:19:30.797 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.88696.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.88696.log.gz 2024-09-12T14:19:30.797 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.73797.log 2024-09-12T14:19:30.799 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.105933.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.105933.log.gz 2024-09-12T14:19:30.799 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.115829.log 2024-09-12T14:19:30.799 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.73797.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.73797.log.gz 2024-09-12T14:19:30.799 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.38538.log 2024-09-12T14:19:30.800 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.115829.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.115829.log.gz 2024-09-12T14:19:30.800 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.84583.log 2024-09-12T14:19:30.800 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.38538.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.38538.log.gz 2024-09-12T14:19:30.801 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.45840.log 2024-09-12T14:19:30.802 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.84583.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.84583.log.gz 2024-09-12T14:19:30.802 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.70371.log 2024-09-12T14:19:30.802 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.45840.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.45840.log.gz 2024-09-12T14:19:30.802 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.39047.log 2024-09-12T14:19:30.803 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.70371.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.70371.log.gz 2024-09-12T14:19:30.803 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.57343.log 2024-09-12T14:19:30.804 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.39047.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.39047.log.gz 2024-09-12T14:19:30.804 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.141758.log 2024-09-12T14:19:30.804 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.57343.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.57343.log.gz 2024-09-12T14:19:30.805 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.76043.log 2024-09-12T14:19:30.806 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.141758.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.141758.log.gz 2024-09-12T14:19:30.806 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.60300.log 2024-09-12T14:19:30.806 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.76043.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.76043.log.gz 2024-09-12T14:19:30.806 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.76294.log 2024-09-12T14:19:30.807 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.60300.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.60300.log.gz 2024-09-12T14:19:30.807 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.111835.log 2024-09-12T14:19:30.808 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.76294.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.76294.log.gz 2024-09-12T14:19:30.808 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.137962.log 2024-09-12T14:19:30.808 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.111835.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.111835.log.gz 2024-09-12T14:19:30.808 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.119335.log 2024-09-12T14:19:30.809 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.137962.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.137962.log.gz 2024-09-12T14:19:30.809 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.130685.log 2024-09-12T14:19:30.810 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.119335.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.119335.log.gz 2024-09-12T14:19:30.811 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.52708.log 2024-09-12T14:19:30.811 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.130685.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.130685.log.gz 2024-09-12T14:19:30.811 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.111184.log 2024-09-12T14:19:30.812 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.52708.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.52708.log.gz 2024-09-12T14:19:30.812 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.91495.log 2024-09-12T14:19:30.812 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.111184.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.111184.log.gz 2024-09-12T14:19:30.812 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.84690.log 2024-09-12T14:19:30.814 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.91495.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.91495.log.gz 2024-09-12T14:19:30.814 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.125945.log 2024-09-12T14:19:30.814 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.84690.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.84690.log.gz 2024-09-12T14:19:30.814 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.101594.log 2024-09-12T14:19:30.815 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.125945.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.125945.log.gz 2024-09-12T14:19:30.816 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.100140.log 2024-09-12T14:19:30.816 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.101594.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.101594.log.gz 2024-09-12T14:19:30.816 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.36808.log 2024-09-12T14:19:30.817 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.100140.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.100140.log.gz 2024-09-12T14:19:30.817 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.59094.log 2024-09-12T14:19:30.817 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.36808.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.36808.log.gz 2024-09-12T14:19:30.817 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.137916.log 2024-09-12T14:19:30.818 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.59094.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.69781.log 2024-09-12T14:19:30.818 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.59094.log.gz 2024-09-12T14:19:30.819 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.137916.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.137916.log.gz 2024-09-12T14:19:30.819 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.145820.log 2024-09-12T14:19:30.819 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.69781.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.69781.log.gz 2024-09-12T14:19:30.819 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.105638.log 2024-09-12T14:19:30.820 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.145820.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.47095.log 2024-09-12T14:19:30.820 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.145820.log.gz 2024-09-12T14:19:30.821 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.105638.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.105638.log.gz 2024-09-12T14:19:30.821 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.93631.log 2024-09-12T14:19:30.821 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.47095.log: gzip 0.0% -5 -- replaced with /var/log/ceph/cluster1-client.mirror.47095.log.gz --verbose 2024-09-12T14:19:30.822 INFO:teuthology.orchestra.run.smithi130.stderr: -- /var/log/ceph/cluster2-client.admin.140802.log 2024-09-12T14:19:30.822 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.93631.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.93631.log.gz 2024-09-12T14:19:30.822 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.124585.log 2024-09-12T14:19:30.823 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.140802.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.140802.log.gz 2024-09-12T14:19:30.823 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.106067.log 2024-09-12T14:19:30.824 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.124585.log: gzip -5 --verbose -- 0.0% /var/log/ceph/cluster1-client.admin.91496.log 2024-09-12T14:19:30.824 INFO:teuthology.orchestra.run.smithi130.stderr: -- replaced with /var/log/ceph/cluster1-client.mirror.124585.log.gz 2024-09-12T14:19:30.825 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.106067.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.106067.log.gz 2024-09-12T14:19:30.825 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.122793.log 2024-09-12T14:19:30.826 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.91496.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.91496.log.gz 2024-09-12T14:19:30.826 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.109526.log 2024-09-12T14:19:30.827 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.122793.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.122793.log.gz 2024-09-12T14:19:30.827 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.35559.log 2024-09-12T14:19:30.827 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.109526.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.62666.log 2024-09-12T14:19:30.827 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.109526.log.gz 2024-09-12T14:19:30.828 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.35559.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.35559.log.gz 2024-09-12T14:19:30.828 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.142879.log 2024-09-12T14:19:30.829 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.62666.log: 0.0%gzip -- replaced with /var/log/ceph/cluster1-client.mirror.62666.log.gz 2024-09-12T14:19:30.829 INFO:teuthology.orchestra.run.smithi130.stderr: -5 --verbose -- /var/log/ceph/cluster1-client.mirror.95238.log 2024-09-12T14:19:30.829 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.142879.log: gzip -5 --verbose 0.0% -- /var/log/ceph/cluster2-osd.2.log -- replaced with /var/log/ceph/cluster1-client.mirror.142879.log.gz 2024-09-12T14:19:30.829 INFO:teuthology.orchestra.run.smithi130.stderr: 2024-09-12T14:19:30.830 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.95238.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.95238.log.gz 2024-09-12T14:19:30.830 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.107585.log 2024-09-12T14:19:30.831 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-osd.2.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.129845.log 2024-09-12T14:19:30.831 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.107585.log: gzip 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.107585.log.gz -5 2024-09-12T14:19:30.831 INFO:teuthology.orchestra.run.smithi130.stderr: --verbose -- /var/log/ceph/cluster2-client.admin.80839.log 2024-09-12T14:19:30.832 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.129845.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.129845.log.gz 2024-09-12T14:19:30.832 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.102967.log 2024-09-12T14:19:30.834 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.80839.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.80839.log.gz 2024-09-12T14:19:30.834 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.87922.log 2024-09-12T14:19:30.834 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.102967.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.102967.log.gz 2024-09-12T14:19:30.834 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.154086.log 2024-09-12T14:19:30.835 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.87922.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.77917.log 2024-09-12T14:19:30.835 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.87922.log.gz 2024-09-12T14:19:30.835 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.154086.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.154086.log.gz 2024-09-12T14:19:30.836 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.87124.log 2024-09-12T14:19:30.837 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.77917.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.37510.log 2024-09-12T14:19:30.837 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.77917.log.gz 2024-09-12T14:19:30.837 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.87124.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.87124.log.gz 2024-09-12T14:19:30.837 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.140163.log 2024-09-12T14:19:30.838 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.37510.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.37510.log.gz 2024-09-12T14:19:30.838 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.149198.log 2024-09-12T14:19:30.839 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.140163.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.140163.log.gz 2024-09-12T14:19:30.839 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.152058.log 2024-09-12T14:19:30.840 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.149198.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.149198.log.gz 2024-09-12T14:19:30.840 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.99366.log 2024-09-12T14:19:30.840 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.152058.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.152058.log.gz 2024-09-12T14:19:30.840 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.141603.log 2024-09-12T14:19:30.841 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.99366.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.99366.log.gz 2024-09-12T14:19:30.841 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.63734.log 2024-09-12T14:19:30.842 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.141603.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.141603.log.gz 2024-09-12T14:19:30.842 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.122099.log 2024-09-12T14:19:30.843 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.63734.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.63734.log.gz 2024-09-12T14:19:30.844 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.105892.log 2024-09-12T14:19:30.844 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.122099.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.122099.log.gz 2024-09-12T14:19:30.844 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.91657.log 2024-09-12T14:19:30.844 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.105892.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.105892.log.gz 2024-09-12T14:19:30.844 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.89265.log 2024-09-12T14:19:30.846 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.91657.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.91657.log.gz 2024-09-12T14:19:30.846 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.58964.log 2024-09-12T14:19:30.846 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.89265.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.89265.log.gz 2024-09-12T14:19:30.846 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.61949.log 2024-09-12T14:19:30.846 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.58964.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.58964.log.gz 2024-09-12T14:19:30.846 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.37764.log 2024-09-12T14:19:30.848 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.61949.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.61949.log.gz 2024-09-12T14:19:30.848 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.53525.log 2024-09-12T14:19:30.848 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.37764.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.37764.log.gz 2024-09-12T14:19:30.848 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.66129.log 2024-09-12T14:19:30.850 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.53525.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.53525.log.gz 2024-09-12T14:19:30.850 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.71970.log 2024-09-12T14:19:30.850 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.66129.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.66129.log.gz 2024-09-12T14:19:30.850 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.39780.log 2024-09-12T14:19:30.850 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.71970.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.71970.log.gz 2024-09-12T14:19:30.850 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.99408.log 2024-09-12T14:19:30.852 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.39780.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.39780.log.gz 2024-09-12T14:19:30.852 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.74489.log 2024-09-12T14:19:30.852 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.99408.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.99408.log.gz 2024-09-12T14:19:30.852 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.80157.log 2024-09-12T14:19:30.852 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.74489.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.74489.log.gz 2024-09-12T14:19:30.852 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.49448.log 2024-09-12T14:19:30.854 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.80157.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.80157.log.gz 2024-09-12T14:19:30.854 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.40749.log 2024-09-12T14:19:30.854 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.49448.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.49448.log.gz 2024-09-12T14:19:30.854 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.148903.log 2024-09-12T14:19:30.855 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.40749.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.40749.log.gz 2024-09-12T14:19:30.855 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.121302.log 2024-09-12T14:19:30.855 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.148903.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.148903.log.gz 2024-09-12T14:19:30.855 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.122466.log 2024-09-12T14:19:30.856 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.121302.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.121302.log.gz 2024-09-12T14:19:30.856 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.102285.log 2024-09-12T14:19:30.857 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.122466.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.122466.log.gz 2024-09-12T14:19:30.857 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.122064.log 2024-09-12T14:19:30.857 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.102285.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.102285.log.gz 2024-09-12T14:19:30.858 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.58659.log 2024-09-12T14:19:30.858 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.122064.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.122064.log.gz 2024-09-12T14:19:30.858 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.48174.log 2024-09-12T14:19:30.859 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.58659.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.58659.log.gz 2024-09-12T14:19:30.859 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.109613.log 2024-09-12T14:19:30.860 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.48174.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.48174.log.gz 2024-09-12T14:19:30.860 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.64533.log 2024-09-12T14:19:30.861 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.109613.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.109613.log.gz 2024-09-12T14:19:30.861 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.152440.log 2024-09-12T14:19:30.861 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.64533.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.130791.log 2024-09-12T14:19:30.861 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.64533.log.gz 2024-09-12T14:19:30.862 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.152440.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.152440.log.gz 2024-09-12T14:19:30.862 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.68285.log 2024-09-12T14:19:30.863 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.130791.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.130791.log.gz 2024-09-12T14:19:30.863 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.82455.log 2024-09-12T14:19:30.864 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.68285.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.68285.log.gz 2024-09-12T14:19:30.864 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.91550.log 2024-09-12T14:19:30.865 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.82455.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.82455.log.gz 2024-09-12T14:19:30.865 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.122857.log 2024-09-12T14:19:30.865 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.91550.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.91550.log.gz 2024-09-12T14:19:30.866 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.78434.log 2024-09-12T14:19:30.866 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.122857.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.122857.log.gz 2024-09-12T14:19:30.866 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.109314.log 2024-09-12T14:19:30.867 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.78434.log: gzip 0.0% -5 -- replaced with /var/log/ceph/cluster1-client.mirror.78434.log.gz --verbose 2024-09-12T14:19:30.867 INFO:teuthology.orchestra.run.smithi130.stderr: -- /var/log/ceph/cluster1-client.mirror.124839.log 2024-09-12T14:19:30.868 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.109314.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.109314.log.gz 2024-09-12T14:19:30.868 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.80206.log 2024-09-12T14:19:30.869 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.124839.log: 0.0%gzip -- replaced with /var/log/ceph/cluster1-client.mirror.124839.log.gz 2024-09-12T14:19:30.869 INFO:teuthology.orchestra.run.smithi130.stderr: -5 --verbose -- /var/log/ceph/cluster2-client.mirror.105322.log 2024-09-12T14:19:30.870 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.80206.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.80206.log.gz 2024-09-12T14:19:30.870 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.63442.log 2024-09-12T14:19:30.870 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.105322.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.105322.log.gz 2024-09-12T14:19:30.871 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.39778.log 2024-09-12T14:19:30.872 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.63442.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.108207.log 2024-09-12T14:19:30.872 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.63442.log.gz 2024-09-12T14:19:30.872 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.39778.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.39778.log.gz 2024-09-12T14:19:30.872 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.80913.log 2024-09-12T14:19:30.873 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.108207.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.108207.log.gz 2024-09-12T14:19:30.873 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.143792.log 2024-09-12T14:19:30.874 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.80913.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.80913.log.gz 2024-09-12T14:19:30.874 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.133546.log 2024-09-12T14:19:30.874 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.143792.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.143792.log.gz 2024-09-12T14:19:30.874 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.114012.log 2024-09-12T14:19:30.875 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.133546.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.139626.log 2024-09-12T14:19:30.875 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.133546.log.gz 2024-09-12T14:19:30.876 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.114012.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.114012.log.gz 2024-09-12T14:19:30.876 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.132874.log 2024-09-12T14:19:30.877 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.139626.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.139626.log.gz 2024-09-12T14:19:30.877 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.91776.log 2024-09-12T14:19:30.877 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.132874.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.138785.log 2024-09-12T14:19:30.877 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.132874.log.gz 2024-09-12T14:19:30.878 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.91776.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.91776.log.gz 2024-09-12T14:19:30.878 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.89625.log 2024-09-12T14:19:30.879 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.138785.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.138785.log.gz 2024-09-12T14:19:30.880 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.40898.log 2024-09-12T14:19:30.880 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.89625.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.89625.log.gz 2024-09-12T14:19:30.880 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.69563.log 2024-09-12T14:19:30.880 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.40898.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.40898.log.gz 2024-09-12T14:19:30.881 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.98232.log 2024-09-12T14:19:30.881 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.69563.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.69563.log.gz 2024-09-12T14:19:30.881 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.66781.log 2024-09-12T14:19:30.882 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.98232.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.98232.log.gz 2024-09-12T14:19:30.882 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.145251.log 2024-09-12T14:19:30.883 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.66781.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.66781.log.gz 2024-09-12T14:19:30.883 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.56145.log 2024-09-12T14:19:30.884 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.145251.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.145251.log.gz 2024-09-12T14:19:30.884 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.116664.log 2024-09-12T14:19:30.884 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.56145.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.56145.log.gz 2024-09-12T14:19:30.885 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.41918.log 2024-09-12T14:19:30.885 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.116664.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.133501.log 2024-09-12T14:19:30.885 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.116664.log.gz 2024-09-12T14:19:30.886 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.41918.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.41918.log.gz 2024-09-12T14:19:30.886 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.137390.log 2024-09-12T14:19:30.887 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.133501.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.133501.log.gz 2024-09-12T14:19:30.887 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.85107.log 2024-09-12T14:19:30.888 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.137390.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.137390.log.gz 2024-09-12T14:19:30.888 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.41690.log 2024-09-12T14:19:30.889 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.85107.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.85107.log.gz 2024-09-12T14:19:30.889 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.141395.log 2024-09-12T14:19:30.889 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.41690.log: gzip 0.0% -5 --verbose -- replaced with /var/log/ceph/cluster1-client.mirror.41690.log.gz -- /var/log/ceph/cluster1-client.mirror.100474.log 2024-09-12T14:19:30.889 INFO:teuthology.orchestra.run.smithi130.stderr: 2024-09-12T14:19:30.890 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.141395.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.141395.log.gz 2024-09-12T14:19:30.890 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.132096.log 2024-09-12T14:19:30.891 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.100474.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.100474.log.gz 2024-09-12T14:19:30.891 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.131058.log 2024-09-12T14:19:30.892 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.132096.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.104322.log 2024-09-12T14:19:30.892 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.132096.log.gz 2024-09-12T14:19:30.893 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.131058.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.131058.log.gz 2024-09-12T14:19:30.893 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.105269.log 2024-09-12T14:19:30.894 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.104322.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.104322.log.gz 2024-09-12T14:19:30.894 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.33898.log 2024-09-12T14:19:30.894 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.105269.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.105269.log.gzgzip 2024-09-12T14:19:30.895 INFO:teuthology.orchestra.run.smithi130.stderr: -5 --verbose -- /var/log/ceph/cluster1-client.mirror.83439.log 2024-09-12T14:19:30.895 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.33898.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.33898.log.gz 2024-09-12T14:19:30.896 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.90832.log 2024-09-12T14:19:30.896 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.83439.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.83439.log.gz 2024-09-12T14:19:30.896 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.41569.log 2024-09-12T14:19:30.897 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.90832.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.93839.log 2024-09-12T14:19:30.897 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.90832.log.gz 2024-09-12T14:19:30.898 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.41569.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.41569.log.gz 2024-09-12T14:19:30.898 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.133376.log 2024-09-12T14:19:30.899 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.93839.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.93839.log.gz 2024-09-12T14:19:30.899 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.143122.log 2024-09-12T14:19:30.899 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.133376.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.79501.log 2024-09-12T14:19:30.899 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.133376.log.gz 2024-09-12T14:19:30.900 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.143122.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.143122.log.gz 2024-09-12T14:19:30.900 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.60586.log 2024-09-12T14:19:30.901 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.79501.log: 0.0%gzip -- replaced with /var/log/ceph/cluster1-client.admin.79501.log.gz 2024-09-12T14:19:30.901 INFO:teuthology.orchestra.run.smithi130.stderr: -5 --verbose -- /var/log/ceph/cluster1-client.mirror.71189.log 2024-09-12T14:19:30.902 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.60586.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.60586.log.gz 2024-09-12T14:19:30.902 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.97663.log 2024-09-12T14:19:30.903 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.71189.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.125867.log 2024-09-12T14:19:30.903 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.71189.log.gz 2024-09-12T14:19:30.903 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.97663.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.97663.log.gz 2024-09-12T14:19:30.904 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.70862.log 2024-09-12T14:19:30.904 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.125867.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.125867.log.gz 2024-09-12T14:19:30.904 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.149685.log 2024-09-12T14:19:30.905 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.70862.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.114347.log 2024-09-12T14:19:30.905 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.70862.log.gz 2024-09-12T14:19:30.906 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.149685.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.149685.log.gz 2024-09-12T14:19:30.906 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.119505.log 2024-09-12T14:19:30.906 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.114347.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.114347.log.gz 2024-09-12T14:19:30.907 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.134446.log 2024-09-12T14:19:30.908 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.119505.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.111731.log 2024-09-12T14:19:30.908 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.119505.log.gz 2024-09-12T14:19:30.908 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.134446.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.134446.log.gz 2024-09-12T14:19:30.908 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.44463.log 2024-09-12T14:19:30.909 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.111731.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.111731.log.gz 2024-09-12T14:19:30.909 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.113267.log 2024-09-12T14:19:30.910 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.44463.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.44463.log.gz 2024-09-12T14:19:30.910 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.46288.log 2024-09-12T14:19:30.911 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.113267.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.113267.log.gz 2024-09-12T14:19:30.911 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.50606.log 2024-09-12T14:19:30.912 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.46288.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.46288.log.gz 2024-09-12T14:19:30.912 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.121428.log 2024-09-12T14:19:30.913 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.50606.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.50606.log.gz 2024-09-12T14:19:30.913 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.121345.log 2024-09-12T14:19:30.914 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.121428.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.96004.log 2024-09-12T14:19:30.914 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.121428.log.gz 2024-09-12T14:19:30.915 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.121345.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.121345.log.gz 2024-09-12T14:19:30.915 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.87100.log 2024-09-12T14:19:30.916 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.96004.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.96004.log.gz 2024-09-12T14:19:30.916 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.95781.log 2024-09-12T14:19:30.916 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.87100.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.146405.log 2024-09-12T14:19:30.916 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.87100.log.gz 2024-09-12T14:19:30.917 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.95781.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.95781.log.gz 2024-09-12T14:19:30.917 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.85736.log 2024-09-12T14:19:30.917 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.146405.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.146405.log.gz 2024-09-12T14:19:30.918 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.73815.log 2024-09-12T14:19:30.919 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.85736.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.85736.log.gz 2024-09-12T14:19:30.919 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.98771.log 2024-09-12T14:19:30.919 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.73815.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.73815.log.gz 2024-09-12T14:19:30.919 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.85545.log 2024-09-12T14:19:30.920 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.98771.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.98771.log.gz 2024-09-12T14:19:30.920 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.36087.log 2024-09-12T14:19:30.921 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.85545.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.85545.log.gz 2024-09-12T14:19:30.921 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.89226.log 2024-09-12T14:19:30.921 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.36087.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.35633.log 2024-09-12T14:19:30.922 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.36087.log.gz 2024-09-12T14:19:30.923 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.89226.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.89226.log.gz 2024-09-12T14:19:30.923 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.93611.log 2024-09-12T14:19:30.923 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.35633.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.35633.log.gz 2024-09-12T14:19:30.923 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.54240.log 2024-09-12T14:19:30.924 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.93611.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.93611.log.gz 2024-09-12T14:19:30.924 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.141778.log 2024-09-12T14:19:30.925 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.54240.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.54240.log.gz 2024-09-12T14:19:30.925 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.147730.log 2024-09-12T14:19:30.926 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.141778.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.152847.log 2024-09-12T14:19:30.926 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.141778.log.gz 2024-09-12T14:19:30.926 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.147730.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.147730.log.gz 2024-09-12T14:19:30.926 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.113456.log 2024-09-12T14:19:30.927 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.152847.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.152847.log.gz 2024-09-12T14:19:30.927 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.124791.log 2024-09-12T14:19:30.928 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.113456.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.113456.log.gz 2024-09-12T14:19:30.928 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.99277.log 2024-09-12T14:19:30.929 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.124791.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.124791.log.gz 2024-09-12T14:19:30.929 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.94539.log 2024-09-12T14:19:30.929 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.99277.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.40539.log 2024-09-12T14:19:30.929 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.99277.log.gz 2024-09-12T14:19:30.930 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.94539.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.94539.log.gz 2024-09-12T14:19:30.930 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.116469.log 2024-09-12T14:19:30.931 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.40539.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.40539.log.gz 2024-09-12T14:19:30.931 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.56944.log 2024-09-12T14:19:30.932 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.116469.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.116469.log.gz 2024-09-12T14:19:30.932 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.55614.log 2024-09-12T14:19:30.932 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.56944.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.56944.log.gz 2024-09-12T14:19:30.932 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.115108.log 2024-09-12T14:19:30.933 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.55614.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.76527.log 2024-09-12T14:19:30.933 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.55614.log.gz 2024-09-12T14:19:30.934 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.115108.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.115108.log.gz 2024-09-12T14:19:30.934 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.60732.log 2024-09-12T14:19:30.935 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.76527.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.76527.log.gz 2024-09-12T14:19:30.935 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.94428.log 2024-09-12T14:19:30.936 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.60732.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.60732.log.gz 2024-09-12T14:19:30.936 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.70123.log 2024-09-12T14:19:30.937 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.94428.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.94428.log.gz 2024-09-12T14:19:30.937 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.127669.log 2024-09-12T14:19:30.937 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.70123.log: gzip -5 --verbose 0.0% -- /var/log/ceph/cluster1-client.mirror.121154.log -- replaced with /var/log/ceph/cluster2-client.mirror.70123.log.gz 2024-09-12T14:19:30.938 INFO:teuthology.orchestra.run.smithi130.stderr: 2024-09-12T14:19:30.938 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.127669.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.127669.log.gz 2024-09-12T14:19:30.938 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.93932.log 2024-09-12T14:19:30.939 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.121154.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.121154.log.gz 2024-09-12T14:19:30.939 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.67934.log 2024-09-12T14:19:30.940 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.93932.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.93932.log.gz 2024-09-12T14:19:30.940 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.94274.log 2024-09-12T14:19:30.941 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.67934.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.67934.log.gz 2024-09-12T14:19:30.941 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.53526.log 2024-09-12T14:19:30.942 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.94274.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.94274.log.gz 2024-09-12T14:19:30.942 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.87732.log 2024-09-12T14:19:30.943 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.53526.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.125979.log 2024-09-12T14:19:30.943 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.53526.log.gz 2024-09-12T14:19:30.943 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.87732.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.87732.log.gz 2024-09-12T14:19:30.944 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.110789.log 2024-09-12T14:19:30.945 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.125979.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.125979.log.gz 2024-09-12T14:19:30.945 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.136711.log 2024-09-12T14:19:30.945 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.110789.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.110789.log.gz 2024-09-12T14:19:30.945 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.97374.log 2024-09-12T14:19:30.946 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.136711.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.113518.log 2024-09-12T14:19:30.946 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.136711.log.gz 2024-09-12T14:19:30.946 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.97374.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.97374.log.gz 2024-09-12T14:19:30.947 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.73333.log 2024-09-12T14:19:30.947 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.113518.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.113518.log.gz 2024-09-12T14:19:30.947 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.153121.log 2024-09-12T14:19:30.948 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.73333.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.88213.log 2024-09-12T14:19:30.948 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.73333.log.gz 2024-09-12T14:19:30.949 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.153121.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.153121.log.gz 2024-09-12T14:19:30.949 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.89580.log 2024-09-12T14:19:30.949 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.88213.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.88213.log.gz 2024-09-12T14:19:30.950 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.68337.log 2024-09-12T14:19:30.950 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.89580.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.65728.log 2024-09-12T14:19:30.950 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.89580.log.gz 2024-09-12T14:19:30.951 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.68337.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.68337.log.gz 2024-09-12T14:19:30.951 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.123012.log 2024-09-12T14:19:30.952 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.65728.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.65728.log.gz 2024-09-12T14:19:30.952 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.104948.log 2024-09-12T14:19:30.953 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.123012.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.123012.log.gz 2024-09-12T14:19:30.953 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.110049.log 2024-09-12T14:19:30.953 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.104948.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.104948.log.gz 2024-09-12T14:19:30.954 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.77810.log 2024-09-12T14:19:30.954 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.110049.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.73693.log 2024-09-12T14:19:30.954 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.110049.log.gz 2024-09-12T14:19:30.955 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.77810.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.77810.log.gz 2024-09-12T14:19:30.955 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.144914.log 2024-09-12T14:19:30.956 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.73693.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.73693.log.gz 2024-09-12T14:19:30.956 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.71601.log 2024-09-12T14:19:30.956 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.144914.log: 0.0%gzip -- replaced with /var/log/ceph/cluster1-client.mirror.144914.log.gz -5 --verbose 2024-09-12T14:19:30.956 INFO:teuthology.orchestra.run.smithi130.stderr: -- /var/log/ceph/cluster1-client.mirror.63229.log 2024-09-12T14:19:30.957 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.71601.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.71601.log.gz 2024-09-12T14:19:30.957 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.123283.log 2024-09-12T14:19:30.958 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.63229.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.63229.log.gz 2024-09-12T14:19:30.958 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.97203.log 2024-09-12T14:19:30.959 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.123283.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.113476.log 2024-09-12T14:19:30.959 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.123283.log.gz 2024-09-12T14:19:30.960 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.97203.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.97203.log.gz 2024-09-12T14:19:30.960 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.37241.log 2024-09-12T14:19:30.961 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.113476.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.113476.log.gz 2024-09-12T14:19:30.961 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.93772.log 2024-09-12T14:19:30.961 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.37241.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.55525.log 2024-09-12T14:19:30.962 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.37241.log.gz 2024-09-12T14:19:30.963 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.93772.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.93772.log.gz 2024-09-12T14:19:30.963 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.134971.log 2024-09-12T14:19:30.963 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.55525.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.55525.log.gz 2024-09-12T14:19:30.963 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.85274.log 2024-09-12T14:19:30.964 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.134971.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.134971.log.gz 2024-09-12T14:19:30.964 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.125181.log 2024-09-12T14:19:30.965 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.85274.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.39088.log 2024-09-12T14:19:30.965 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.85274.log.gz 2024-09-12T14:19:30.966 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.125181.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.125181.log.gz 2024-09-12T14:19:30.966 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.88089.log 2024-09-12T14:19:30.966 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.39088.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.39088.log.gz 2024-09-12T14:19:30.967 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.115476.log 2024-09-12T14:19:30.967 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.88089.log: gzip -5 --verbose -- 0.0% /var/log/ceph/cluster1-client.admin.95479.log 2024-09-12T14:19:30.968 INFO:teuthology.orchestra.run.smithi130.stderr: -- replaced with /var/log/ceph/cluster1-client.mirror.88089.log.gz 2024-09-12T14:19:30.968 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.115476.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.115476.log.gz 2024-09-12T14:19:30.968 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.61109.log 2024-09-12T14:19:30.969 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.95479.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.95479.log.gz 2024-09-12T14:19:30.969 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.43719.log 2024-09-12T14:19:30.970 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.61109.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.80309.log 2024-09-12T14:19:30.971 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.61109.log.gz 2024-09-12T14:19:30.974 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.43719.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.43719.log.gz 2024-09-12T14:19:30.974 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.144977.log 2024-09-12T14:19:30.974 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.80309.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.80309.log.gz 2024-09-12T14:19:30.974 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.70620.log 2024-09-12T14:19:30.974 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.144977.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.144977.log.gz 2024-09-12T14:19:30.974 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.39638.log 2024-09-12T14:19:30.974 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.70620.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.70620.log.gz 2024-09-12T14:19:30.974 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.132097.log 2024-09-12T14:19:30.974 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.39638.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.39638.log.gz 2024-09-12T14:19:30.975 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.85193.log 2024-09-12T14:19:30.976 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.132097.log: 0.0%gzip -- replaced with /var/log/ceph/cluster1-client.mirror.132097.log.gz -5 2024-09-12T14:19:30.976 INFO:teuthology.orchestra.run.smithi130.stderr: --verbose -- /var/log/ceph/cluster1-client.admin.68807.log 2024-09-12T14:19:30.976 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.85193.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.85193.log.gz 2024-09-12T14:19:30.976 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.132762.log 2024-09-12T14:19:30.976 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.68807.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.68807.log.gz 2024-09-12T14:19:30.976 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.110008.log 2024-09-12T14:19:30.977 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.132762.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.127965.log 2024-09-12T14:19:30.978 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.132762.log.gz 2024-09-12T14:19:30.978 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.110008.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.110008.log.gz 2024-09-12T14:19:30.978 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.148569.log 2024-09-12T14:19:30.979 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.127965.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.127965.log.gz 2024-09-12T14:19:30.979 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.44819.log 2024-09-12T14:19:30.980 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.148569.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.36239.log 2024-09-12T14:19:30.980 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.148569.log.gz 2024-09-12T14:19:30.980 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.44819.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.44819.log.gz 2024-09-12T14:19:30.980 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.124357.log 2024-09-12T14:19:30.981 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.36239.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.36239.log.gz 2024-09-12T14:19:30.981 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.106574.log 2024-09-12T14:19:30.982 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.124357.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.124357.log.gz 2024-09-12T14:19:30.982 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.80422.log 2024-09-12T14:19:30.982 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.106574.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.106574.log.gz 2024-09-12T14:19:30.982 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.60794.log 2024-09-12T14:19:30.983 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.80422.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.80422.log.gz 2024-09-12T14:19:30.983 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.135141.log 2024-09-12T14:19:30.984 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.60794.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.60794.log.gz 2024-09-12T14:19:30.984 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.36003.log 2024-09-12T14:19:30.985 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.135141.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.135141.log.gz 2024-09-12T14:19:30.985 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.50310.log 2024-09-12T14:19:30.985 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.36003.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.36003.log.gz 2024-09-12T14:19:30.985 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.65942.log 2024-09-12T14:19:30.986 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.50310.log: 0.0%gzip -- replaced with /var/log/ceph/cluster1-client.mirror.50310.log.gz 2024-09-12T14:19:30.986 INFO:teuthology.orchestra.run.smithi130.stderr: -5 --verbose -- /var/log/ceph/cluster1-client.admin.127801.log 2024-09-12T14:19:30.987 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.65942.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.96588.log 2024-09-12T14:19:30.987 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.65942.log.gz 2024-09-12T14:19:30.988 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.127801.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.127801.log.gzgzip 2024-09-12T14:19:30.988 INFO:teuthology.orchestra.run.smithi130.stderr: -5 --verbose -- /var/log/ceph/cluster1-client.admin.94315.log 2024-09-12T14:19:30.988 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.96588.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.96588.log.gz 2024-09-12T14:19:30.988 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.49555.log 2024-09-12T14:19:30.989 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.94315.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.94315.log.gz 2024-09-12T14:19:30.989 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.85275.log 2024-09-12T14:19:30.990 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.49555.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.154299.log 2024-09-12T14:19:30.990 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.49555.log.gz 2024-09-12T14:19:30.991 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.85275.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.85275.log.gz 2024-09-12T14:19:30.991 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.135595.log 2024-09-12T14:19:30.992 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.154299.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.36354.log 2024-09-12T14:19:30.992 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.154299.log.gz 2024-09-12T14:19:30.992 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.135595.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.135595.log.gz 2024-09-12T14:19:30.992 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.61803.log 2024-09-12T14:19:30.993 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.36354.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.47563.log 2024-09-12T14:19:30.993 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.36354.log.gz 2024-09-12T14:19:30.993 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.61803.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.61803.log.gz 2024-09-12T14:19:30.994 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.102341.log 2024-09-12T14:19:30.994 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.47563.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.47563.log.gz 2024-09-12T14:19:30.994 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.59056.log 2024-09-12T14:19:30.995 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.102341.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.102341.log.gz 2024-09-12T14:19:30.995 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.107622.log 2024-09-12T14:19:30.996 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.59056.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.59056.log.gz 2024-09-12T14:19:30.996 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.130073.log 2024-09-12T14:19:30.996 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.107622.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.107622.log.gz 2024-09-12T14:19:30.996 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.133816.log 2024-09-12T14:19:30.997 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.130073.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.130073.log.gz 2024-09-12T14:19:30.997 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.64048.log 2024-09-12T14:19:30.998 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.133816.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.133816.log.gz 2024-09-12T14:19:30.998 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.124127.log 2024-09-12T14:19:30.999 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.64048.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.64048.log.gz 2024-09-12T14:19:30.999 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.144004.log 2024-09-12T14:19:31.000 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.124127.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.124127.log.gz 2024-09-12T14:19:31.000 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.77390.log 2024-09-12T14:19:31.000 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.144004.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.144004.log.gz 2024-09-12T14:19:31.000 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.109866.log 2024-09-12T14:19:31.001 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.77390.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.77390.log.gz 2024-09-12T14:19:31.001 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.139049.log 2024-09-12T14:19:31.002 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.109866.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.109866.log.gz 2024-09-12T14:19:31.002 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.138763.log 2024-09-12T14:19:31.003 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.139049.log: gzip 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.139049.log.gz -5 2024-09-12T14:19:31.003 INFO:teuthology.orchestra.run.smithi130.stderr: --verbose -- /var/log/ceph/cluster1-client.mirror.61235.log 2024-09-12T14:19:31.004 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.138763.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.138763.log.gz 2024-09-12T14:19:31.004 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.71032.log 2024-09-12T14:19:31.005 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.61235.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.61235.log.gz 2024-09-12T14:19:31.005 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.97748.log 2024-09-12T14:19:31.005 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.71032.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.71032.log.gzgzip 2024-09-12T14:19:31.005 INFO:teuthology.orchestra.run.smithi130.stderr: -5 --verbose -- /var/log/ceph/cluster1-client.mirror.124756.log 2024-09-12T14:19:31.006 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.97748.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.97748.log.gz 2024-09-12T14:19:31.006 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.47780.log 2024-09-12T14:19:31.007 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.124756.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.124756.log.gz 2024-09-12T14:19:31.007 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.118657.log 2024-09-12T14:19:31.008 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.47780.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.47780.log.gz 2024-09-12T14:19:31.008 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.145186.log 2024-09-12T14:19:31.008 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.118657.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.118657.log.gzgzip 2024-09-12T14:19:31.009 INFO:teuthology.orchestra.run.smithi130.stderr: -5 --verbose -- /var/log/ceph/cluster1-client.mirror.145396.log 2024-09-12T14:19:31.009 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.145186.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.145186.log.gz 2024-09-12T14:19:31.009 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.154172.log 2024-09-12T14:19:31.010 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.145396.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.145396.log.gz 2024-09-12T14:19:31.010 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.29261.log 2024-09-12T14:19:31.011 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.154172.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.154172.log.gz 2024-09-12T14:19:31.011 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.78824.log 2024-09-12T14:19:31.012 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.29261.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.29261.log.gz 2024-09-12T14:19:31.012 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.48757.log 2024-09-12T14:19:31.013 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.78824.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.78824.log.gz 2024-09-12T14:19:31.013 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.88846.log 2024-09-12T14:19:31.013 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.48757.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.48757.log.gz 2024-09-12T14:19:31.013 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.109799.log 2024-09-12T14:19:31.014 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.88846.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.88846.log.gz 2024-09-12T14:19:31.014 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.98502.log 2024-09-12T14:19:31.015 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.109799.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.109799.log.gz 2024-09-12T14:19:31.015 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.44994.log 2024-09-12T14:19:31.016 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.98502.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.98502.log.gz 2024-09-12T14:19:31.016 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.73735.log 2024-09-12T14:19:31.017 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.44994.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.44994.log.gz 2024-09-12T14:19:31.017 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.115273.log 2024-09-12T14:19:31.017 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.73735.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.73735.log.gz 2024-09-12T14:19:31.018 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.153785.log 2024-09-12T14:19:31.019 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.115273.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.115273.log.gz 2024-09-12T14:19:31.019 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.129016.log 2024-09-12T14:19:31.019 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.153785.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.153785.log.gz 2024-09-12T14:19:31.019 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.129389.log 2024-09-12T14:19:31.020 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.129016.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.129016.log.gz 2024-09-12T14:19:31.020 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.54311.log 2024-09-12T14:19:31.020 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.129389.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.138675.log 2024-09-12T14:19:31.020 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.129389.log.gz 2024-09-12T14:19:31.022 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.54311.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.54311.log.gz 2024-09-12T14:19:31.022 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.150782.log 2024-09-12T14:19:31.022 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.138675.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.138675.log.gz 2024-09-12T14:19:31.022 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.78145.log 2024-09-12T14:19:31.023 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.150782.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.150782.log.gz 2024-09-12T14:19:31.023 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.59670.log 2024-09-12T14:19:31.023 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.78145.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.78145.log.gz 2024-09-12T14:19:31.023 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.146239.log 2024-09-12T14:19:31.024 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.59670.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.59670.log.gz 2024-09-12T14:19:31.024 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.48301.log 2024-09-12T14:19:31.025 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.146239.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.146239.log.gz 2024-09-12T14:19:31.025 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.113244.log 2024-09-12T14:19:31.026 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.48301.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.48301.log.gz 2024-09-12T14:19:31.026 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.85864.log 2024-09-12T14:19:31.027 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.113244.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.113244.log.gz 2024-09-12T14:19:31.027 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.147852.log 2024-09-12T14:19:31.027 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.85864.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.85864.log.gz 2024-09-12T14:19:31.027 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.57083.log 2024-09-12T14:19:31.028 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.147852.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.147852.log.gz 2024-09-12T14:19:31.028 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.103318.log 2024-09-12T14:19:31.029 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.57083.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.57083.log.gz 2024-09-12T14:19:31.029 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.110367.log 2024-09-12T14:19:31.030 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.103318.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.103318.log.gz 2024-09-12T14:19:31.030 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.121718.log 2024-09-12T14:19:31.031 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.110367.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.110367.log.gz 2024-09-12T14:19:31.031 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.117126.log 2024-09-12T14:19:31.031 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.121718.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.121718.log.gz 2024-09-12T14:19:31.032 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.38856.log 2024-09-12T14:19:31.032 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.117126.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.122545.log 2024-09-12T14:19:31.032 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.117126.log.gz 2024-09-12T14:19:31.033 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.38856.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.38856.log.gz 2024-09-12T14:19:31.033 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.91547.log 2024-09-12T14:19:31.034 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.122545.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.122545.log.gz 2024-09-12T14:19:31.034 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.83983.log 2024-09-12T14:19:31.035 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.91547.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.91547.log.gz 2024-09-12T14:19:31.035 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.144746.log 2024-09-12T14:19:31.035 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.83983.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.83983.log.gz 2024-09-12T14:19:31.036 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.39486.log 2024-09-12T14:19:31.036 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.144746.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.144746.log.gz 2024-09-12T14:19:31.037 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.131992.log 2024-09-12T14:19:31.038 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.39486.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.39486.log.gz 2024-09-12T14:19:31.038 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.105155.log 2024-09-12T14:19:31.038 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.131992.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.131992.log.gz 2024-09-12T14:19:31.038 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.56592.log 2024-09-12T14:19:31.039 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.105155.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.105155.log.gz 2024-09-12T14:19:31.039 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.111579.log 2024-09-12T14:19:31.040 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.56592.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.56592.log.gz 2024-09-12T14:19:31.040 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.133416.log 2024-09-12T14:19:31.040 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.111579.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.111579.log.gz 2024-09-12T14:19:31.040 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.103804.log 2024-09-12T14:19:31.041 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.133416.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.133416.log.gz 2024-09-12T14:19:31.041 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.114726.log 2024-09-12T14:19:31.042 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.103804.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.103804.log.gz 2024-09-12T14:19:31.042 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.36926.log 2024-09-12T14:19:31.042 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.114726.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.114726.log.gz 2024-09-12T14:19:31.043 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.50502.log 2024-09-12T14:19:31.043 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.36926.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.36926.log.gz 2024-09-12T14:19:31.043 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.153671.log 2024-09-12T14:19:31.044 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.50502.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.50502.log.gz 2024-09-12T14:19:31.044 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.111545.log 2024-09-12T14:19:31.045 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.153671.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.153671.log.gz 2024-09-12T14:19:31.045 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.51468.log 2024-09-12T14:19:31.046 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.111545.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.111545.log.gz 2024-09-12T14:19:31.046 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.114757.log 2024-09-12T14:19:31.047 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.51468.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.51468.log.gz 2024-09-12T14:19:31.047 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.39970.log 2024-09-12T14:19:31.048 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.114757.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.114757.log.gz 2024-09-12T14:19:31.048 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.73566.log 2024-09-12T14:19:31.049 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.39970.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.39970.log.gz 2024-09-12T14:19:31.049 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.144956.log 2024-09-12T14:19:31.049 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.73566.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.73566.log.gz 2024-09-12T14:19:31.049 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.74972.log 2024-09-12T14:19:31.050 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.144956.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.144956.log.gz 2024-09-12T14:19:31.050 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.98674.log 2024-09-12T14:19:31.051 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.74972.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.74972.log.gz 2024-09-12T14:19:31.051 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.70029.log 2024-09-12T14:19:31.051 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.98674.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.98674.log.gz 2024-09-12T14:19:31.051 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.140281.log 2024-09-12T14:19:31.052 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.70029.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.70029.log.gz 2024-09-12T14:19:31.052 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.96341.log 2024-09-12T14:19:31.053 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.140281.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.140281.log.gz 2024-09-12T14:19:31.053 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.54974.log 2024-09-12T14:19:31.054 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.96341.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.96341.log.gz 2024-09-12T14:19:31.054 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.154215.log 2024-09-12T14:19:31.055 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.54974.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.54974.log.gz 2024-09-12T14:19:31.055 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.122920.log 2024-09-12T14:19:31.056 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.154215.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.154215.log.gz 2024-09-12T14:19:31.056 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.135562.log 2024-09-12T14:19:31.056 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.122920.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.122920.log.gz 2024-09-12T14:19:31.057 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.90313.log 2024-09-12T14:19:31.057 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.135562.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.88173.log 2024-09-12T14:19:31.057 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.135562.log.gz 2024-09-12T14:19:31.058 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.90313.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.90313.log.gz 2024-09-12T14:19:31.058 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.46059.log 2024-09-12T14:19:31.059 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.88173.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.88173.log.gz 2024-09-12T14:19:31.059 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.153189.log 2024-09-12T14:19:31.060 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.46059.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.46059.log.gz 2024-09-12T14:19:31.060 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.90528.log 2024-09-12T14:19:31.061 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.153189.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.153189.log.gz 2024-09-12T14:19:31.061 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.84416.log 2024-09-12T14:19:31.061 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.90528.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.90528.log.gz 2024-09-12T14:19:31.061 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.37723.log 2024-09-12T14:19:31.062 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.84416.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.84416.log.gz 2024-09-12T14:19:31.062 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.103142.log 2024-09-12T14:19:31.063 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.37723.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.37723.log.gz 2024-09-12T14:19:31.063 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.96674.log 2024-09-12T14:19:31.064 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.103142.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.103142.log.gz 2024-09-12T14:19:31.064 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.113863.log 2024-09-12T14:19:31.064 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.96674.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.96674.log.gz 2024-09-12T14:19:31.065 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.79257.log 2024-09-12T14:19:31.065 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.113863.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.113863.log.gz 2024-09-12T14:19:31.066 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.65291.log 2024-09-12T14:19:31.067 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.79257.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.79257.log.gz 2024-09-12T14:19:31.067 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.95192.log 2024-09-12T14:19:31.067 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.65291.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.65291.log.gz 2024-09-12T14:19:31.067 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.73082.log 2024-09-12T14:19:31.068 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.95192.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.95192.log.gz 2024-09-12T14:19:31.068 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.108248.log 2024-09-12T14:19:31.068 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.73082.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.73082.log.gz 2024-09-12T14:19:31.069 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.148339.log 2024-09-12T14:19:31.069 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.108248.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.108248.log.gz 2024-09-12T14:19:31.069 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.121357.log 2024-09-12T14:19:31.070 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.148339.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.148339.log.gz 2024-09-12T14:19:31.070 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.55465.log 2024-09-12T14:19:31.071 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.121357.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.121357.log.gz 2024-09-12T14:19:31.071 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.80311.log 2024-09-12T14:19:31.072 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.55465.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.55465.log.gz 2024-09-12T14:19:31.072 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.114312.log 2024-09-12T14:19:31.073 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.80311.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.80311.log.gz 2024-09-12T14:19:31.073 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.134851.log 2024-09-12T14:19:31.074 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.114312.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.114312.log.gz 2024-09-12T14:19:31.074 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.122649.log 2024-09-12T14:19:31.075 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.134851.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.134851.log.gz 2024-09-12T14:19:31.075 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.70667.log 2024-09-12T14:19:31.075 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.122649.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.122649.log.gz 2024-09-12T14:19:31.075 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.131803.log 2024-09-12T14:19:31.076 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.70667.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.70667.log.gz 2024-09-12T14:19:31.076 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.63104.log 2024-09-12T14:19:31.077 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.131803.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.131803.log.gz 2024-09-12T14:19:31.077 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.96744.log 2024-09-12T14:19:31.078 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.63104.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.63104.log.gz 2024-09-12T14:19:31.078 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.95455.log 2024-09-12T14:19:31.079 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.96744.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.96744.log.gz 2024-09-12T14:19:31.079 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.38307.log 2024-09-12T14:19:31.079 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.95455.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.95455.log.gz 2024-09-12T14:19:31.079 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.53634.log 2024-09-12T14:19:31.080 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.38307.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.38307.log.gz 2024-09-12T14:19:31.080 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.86786.log 2024-09-12T14:19:31.081 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.53634.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.53634.log.gz 2024-09-12T14:19:31.081 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.133672.log 2024-09-12T14:19:31.081 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.86786.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.86786.log.gz 2024-09-12T14:19:31.082 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.65081.log 2024-09-12T14:19:31.082 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.133672.log: 0.0%gzip -- replaced with /var/log/ceph/cluster1-client.mirror.133672.log.gz 2024-09-12T14:19:31.082 INFO:teuthology.orchestra.run.smithi130.stderr: -5 --verbose -- /var/log/ceph/cluster1-client.mirror.96716.log 2024-09-12T14:19:31.083 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.65081.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.65081.log.gz 2024-09-12T14:19:31.083 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.152603.log 2024-09-12T14:19:31.084 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.96716.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.96716.log.gz 2024-09-12T14:19:31.084 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.113098.log 2024-09-12T14:19:31.084 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.152603.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.152603.log.gz 2024-09-12T14:19:31.085 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.130623.log 2024-09-12T14:19:31.085 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.113098.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.113098.log.gz 2024-09-12T14:19:31.085 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.85253.log 2024-09-12T14:19:31.086 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.130623.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.130623.log.gz 2024-09-12T14:19:31.086 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.100309.log 2024-09-12T14:19:31.087 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.85253.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.85253.log.gz 2024-09-12T14:19:31.087 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.111436.log 2024-09-12T14:19:31.088 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.100309.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.100309.log.gz 2024-09-12T14:19:31.088 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.93430.log 2024-09-12T14:19:31.089 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.111436.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.111436.log.gz 2024-09-12T14:19:31.089 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.90468.log 2024-09-12T14:19:31.089 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.93430.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.93430.log.gz 2024-09-12T14:19:31.090 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.58928.log 2024-09-12T14:19:31.090 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.90468.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.90468.log.gz 2024-09-12T14:19:31.090 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.117609.log 2024-09-12T14:19:31.091 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.58928.log: gzip 0.0% -5 -- replaced with /var/log/ceph/cluster1-client.mirror.58928.log.gz 2024-09-12T14:19:31.091 INFO:teuthology.orchestra.run.smithi130.stderr: --verbose -- /var/log/ceph/cluster1-client.mirror.97242.log 2024-09-12T14:19:31.092 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.117609.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.117609.log.gz 2024-09-12T14:19:31.092 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.102320.log 2024-09-12T14:19:31.093 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.97242.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.97242.log.gz 2024-09-12T14:19:31.093 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.63527.log 2024-09-12T14:19:31.093 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.102320.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.102320.log.gz 2024-09-12T14:19:31.094 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.84875.log 2024-09-12T14:19:31.094 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.63527.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.63527.log.gz 2024-09-12T14:19:31.094 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.66003.log 2024-09-12T14:19:31.095 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.84875.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.84875.log.gz 2024-09-12T14:19:31.095 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.46199.log 2024-09-12T14:19:31.097 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.66003.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.66003.log.gz 2024-09-12T14:19:31.097 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.72664.log 2024-09-12T14:19:31.097 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.46199.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.46199.log.gz 2024-09-12T14:19:31.097 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.41605.log 2024-09-12T14:19:31.098 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.72664.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.72664.log.gz 2024-09-12T14:19:31.098 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.76337.log 2024-09-12T14:19:31.098 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.41605.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.41605.log.gz 2024-09-12T14:19:31.099 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.124477.log 2024-09-12T14:19:31.100 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.76337.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.76337.log.gz 2024-09-12T14:19:31.100 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.57101.log 2024-09-12T14:19:31.100 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.124477.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.63879.log 2024-09-12T14:19:31.100 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.124477.log.gz 2024-09-12T14:19:31.101 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.57101.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.57101.log.gz 2024-09-12T14:19:31.101 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.34063.log 2024-09-12T14:19:31.102 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.63879.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.63879.log.gz 2024-09-12T14:19:31.102 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.145250.log 2024-09-12T14:19:31.102 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.34063.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.34063.log.gz 2024-09-12T14:19:31.102 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.84793.log 2024-09-12T14:19:31.104 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.145250.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.145250.log.gz 2024-09-12T14:19:31.104 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.99157.log 2024-09-12T14:19:31.104 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.84793.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.84793.log.gz 2024-09-12T14:19:31.104 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.63943.log 2024-09-12T14:19:31.105 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.99157.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.64781.log 2024-09-12T14:19:31.105 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.99157.log.gz 2024-09-12T14:19:31.105 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.63943.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.63943.log.gz 2024-09-12T14:19:31.106 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.76866.log 2024-09-12T14:19:31.106 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.64781.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.64781.log.gz 2024-09-12T14:19:31.106 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.149663.log 2024-09-12T14:19:31.107 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.76866.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.76866.log.gz 2024-09-12T14:19:31.107 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.124017.log 2024-09-12T14:19:31.108 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.149663.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.149663.log.gz 2024-09-12T14:19:31.108 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.92267.log 2024-09-12T14:19:31.109 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.124017.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.124017.log.gz 2024-09-12T14:19:31.109 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.146529.log 2024-09-12T14:19:31.109 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.92267.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.92267.log.gz 2024-09-12T14:19:31.110 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.119504.log 2024-09-12T14:19:31.110 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.146529.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.146529.log.gz 2024-09-12T14:19:31.110 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.85693.log 2024-09-12T14:19:31.111 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.119504.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.119504.log.gz 2024-09-12T14:19:31.111 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.74913.log 2024-09-12T14:19:31.112 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.85693.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.85693.log.gz 2024-09-12T14:19:31.112 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.69801.log 2024-09-12T14:19:31.113 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.74913.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.74913.log.gz 2024-09-12T14:19:31.113 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.142201.log 2024-09-12T14:19:31.114 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.69801.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.69801.log.gz 2024-09-12T14:19:31.114 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.113519.log 2024-09-12T14:19:31.114 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.142201.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.142201.log.gz 2024-09-12T14:19:31.114 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.76190.log 2024-09-12T14:19:31.115 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.113519.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.113519.log.gz 2024-09-12T14:19:31.115 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.76867.log 2024-09-12T14:19:31.116 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.76190.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.76190.log.gz 2024-09-12T14:19:31.116 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.38476.log 2024-09-12T14:19:31.117 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.76867.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.76867.log.gz 2024-09-12T14:19:31.117 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.69708.log 2024-09-12T14:19:31.118 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.38476.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.38476.log.gz 2024-09-12T14:19:31.118 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.104019.log 2024-09-12T14:19:31.119 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.69708.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.69708.log.gz 2024-09-12T14:19:31.119 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.133293.log 2024-09-12T14:19:31.119 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.104019.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.104019.log.gz 2024-09-12T14:19:31.120 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.94201.log 2024-09-12T14:19:31.120 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.133293.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.133293.log.gz 2024-09-12T14:19:31.120 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.62347.log 2024-09-12T14:19:31.121 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.94201.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.94201.log.gz 2024-09-12T14:19:31.121 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.50419.log 2024-09-12T14:19:31.122 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.62347.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.62347.log.gz 2024-09-12T14:19:31.122 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.67364.log 2024-09-12T14:19:31.122 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.50419.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.50419.log.gz 2024-09-12T14:19:31.122 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.134575.log 2024-09-12T14:19:31.123 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.67364.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.67364.log.gz 2024-09-12T14:19:31.123 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.89667.log 2024-09-12T14:19:31.124 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.134575.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.134575.log.gz 2024-09-12T14:19:31.124 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.132921.log 2024-09-12T14:19:31.125 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.89667.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.89667.log.gz 2024-09-12T14:19:31.125 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.64973.log 2024-09-12T14:19:31.126 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.132921.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.132921.log.gz 2024-09-12T14:19:31.126 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.122455.log 2024-09-12T14:19:31.127 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.64973.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.106208.log 2024-09-12T14:19:31.127 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.64973.log.gz 2024-09-12T14:19:31.127 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.122455.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.122455.log.gz 2024-09-12T14:19:31.127 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.71321.log 2024-09-12T14:19:31.128 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.106208.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.106208.log.gz 2024-09-12T14:19:31.128 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.106706.log 2024-09-12T14:19:31.129 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.71321.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.71321.log.gz 2024-09-12T14:19:31.129 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.153893.log 2024-09-12T14:19:31.130 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.106706.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.106706.log.gz 2024-09-12T14:19:31.130 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.151861.log 2024-09-12T14:19:31.131 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.153893.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.153893.log.gz 2024-09-12T14:19:31.131 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.48521.log 2024-09-12T14:19:31.131 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.151861.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.151861.log.gz 2024-09-12T14:19:31.131 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.131096.log 2024-09-12T14:19:31.132 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.48521.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.48521.log.gz 2024-09-12T14:19:31.132 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.51128.log 2024-09-12T14:19:31.132 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.131096.log: 0.0%gzip -- replaced with /var/log/ceph/cluster1-client.mirror.131096.log.gz -5 2024-09-12T14:19:31.132 INFO:teuthology.orchestra.run.smithi130.stderr: --verbose -- /var/log/ceph/cluster2-client.admin.152536.log 2024-09-12T14:19:31.133 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.51128.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.51128.log.gz 2024-09-12T14:19:31.133 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.123407.log 2024-09-12T14:19:31.134 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.152536.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.152536.log.gz 2024-09-12T14:19:31.134 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.111061.log 2024-09-12T14:19:31.135 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.123407.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.123407.log.gz 2024-09-12T14:19:31.135 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.120398.log 2024-09-12T14:19:31.135 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.111061.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.111061.log.gz 2024-09-12T14:19:31.135 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.117354.log 2024-09-12T14:19:31.136 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.120398.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.120398.log.gz 2024-09-12T14:19:31.136 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.135771.log 2024-09-12T14:19:31.137 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.117354.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.117354.log.gz 2024-09-12T14:19:31.137 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.111302.log 2024-09-12T14:19:31.138 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.135771.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.135771.log.gz 2024-09-12T14:19:31.138 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.109716.log 2024-09-12T14:19:31.138 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.111302.log: 0.0%gzip -- replaced with /var/log/ceph/cluster1-client.mirror.111302.log.gz 2024-09-12T14:19:31.138 INFO:teuthology.orchestra.run.smithi130.stderr: -5 --verbose -- /var/log/ceph/cluster2-client.admin.152668.log 2024-09-12T14:19:31.139 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.109716.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.109716.log.gz 2024-09-12T14:19:31.139 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.42096.log 2024-09-12T14:19:31.140 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.152668.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.143359.log 2024-09-12T14:19:31.140 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.152668.log.gz 2024-09-12T14:19:31.141 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.42096.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.42096.log.gz 2024-09-12T14:19:31.141 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.97960.log 2024-09-12T14:19:31.141 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.143359.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.143359.log.gz 2024-09-12T14:19:31.142 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.33803.log 2024-09-12T14:19:31.142 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.97960.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.97960.log.gz 2024-09-12T14:19:31.142 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.122856.log 2024-09-12T14:19:31.143 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.33803.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.33803.log.gz 2024-09-12T14:19:31.143 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.110494.log 2024-09-12T14:19:31.144 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.122856.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.122856.log.gz 2024-09-12T14:19:31.144 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.80264.log 2024-09-12T14:19:31.145 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.110494.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.110494.log.gz 2024-09-12T14:19:31.145 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.122119.log 2024-09-12T14:19:31.146 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.80264.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.80264.log.gz 2024-09-12T14:19:31.146 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.97076.log 2024-09-12T14:19:31.146 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.122119.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.122119.log.gz 2024-09-12T14:19:31.146 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.136150.log 2024-09-12T14:19:31.147 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.97076.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.97076.log.gz 2024-09-12T14:19:31.147 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.108771.log 2024-09-12T14:19:31.148 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.136150.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.136150.log.gz 2024-09-12T14:19:31.148 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.60951.log 2024-09-12T14:19:31.149 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.108771.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.108771.log.gz 2024-09-12T14:19:31.149 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.101510.log 2024-09-12T14:19:31.149 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.60951.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.60951.log.gz 2024-09-12T14:19:31.149 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.83546.log 2024-09-12T14:19:31.150 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.101510.log: gzip -5 0.0% --verbose -- -- replaced with /var/log/ceph/cluster1-client.mirror.101510.log.gz /var/log/ceph/cluster1-client.mirror.120973.log 2024-09-12T14:19:31.150 INFO:teuthology.orchestra.run.smithi130.stderr: 2024-09-12T14:19:31.151 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.83546.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.83546.log.gz 2024-09-12T14:19:31.151 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.85106.log 2024-09-12T14:19:31.152 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.120973.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.120973.log.gz 2024-09-12T14:19:31.152 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.140158.log 2024-09-12T14:19:31.153 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.85106.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.85106.log.gz 2024-09-12T14:19:31.153 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.54196.log 2024-09-12T14:19:31.153 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.140158.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.140158.log.gz 2024-09-12T14:19:31.153 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.105819.log 2024-09-12T14:19:31.154 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.54196.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.54196.log.gz 2024-09-12T14:19:31.155 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.61414.log 2024-09-12T14:19:31.155 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.105819.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.105819.log.gz 2024-09-12T14:19:31.155 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.108355.log 2024-09-12T14:19:31.155 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.61414.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.61414.log.gz 2024-09-12T14:19:31.156 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.100646.log 2024-09-12T14:19:31.156 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.108355.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.108355.log.gz 2024-09-12T14:19:31.156 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.134045.log 2024-09-12T14:19:31.158 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.100646.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.100646.log.gz 2024-09-12T14:19:31.158 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.91749.log 2024-09-12T14:19:31.158 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.134045.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.134045.log.gz 2024-09-12T14:19:31.158 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.37971.log 2024-09-12T14:19:31.159 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.91749.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.91749.log.gz 2024-09-12T14:19:31.159 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.42749.log 2024-09-12T14:19:31.159 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.37971.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.37971.log.gz 2024-09-12T14:19:31.159 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.53698.log 2024-09-12T14:19:31.160 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.42749.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.42749.log.gz 2024-09-12T14:19:31.160 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.126100.log 2024-09-12T14:19:31.161 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.53698.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.53698.log.gz 2024-09-12T14:19:31.161 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.79699.log 2024-09-12T14:19:31.162 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.126100.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.126100.log.gz 2024-09-12T14:19:31.162 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.138852.log 2024-09-12T14:19:31.162 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.79699.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.79699.log.gz 2024-09-12T14:19:31.162 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.113560.log 2024-09-12T14:19:31.163 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.138852.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.138852.log.gz 2024-09-12T14:19:31.163 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.66589.log 2024-09-12T14:19:31.164 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.113560.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.113560.log.gz 2024-09-12T14:19:31.164 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.50299.log 2024-09-12T14:19:31.165 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.66589.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.66589.log.gz 2024-09-12T14:19:31.165 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.42777.log 2024-09-12T14:19:31.165 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.50299.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.50299.log.gz 2024-09-12T14:19:31.165 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.102922.log 2024-09-12T14:19:31.166 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.42777.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.42777.log.gz 2024-09-12T14:19:31.166 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.124258.log 2024-09-12T14:19:31.167 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.102922.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.102922.log.gz 2024-09-12T14:19:31.167 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.120966.log 2024-09-12T14:19:31.167 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.124258.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.124258.log.gz 2024-09-12T14:19:31.168 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.78677.log 2024-09-12T14:19:31.168 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.120966.log: gzip 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.120966.log.gz -5 2024-09-12T14:19:31.168 INFO:teuthology.orchestra.run.smithi130.stderr: --verbose -- /var/log/ceph/cluster1-client.admin.79435.log 2024-09-12T14:19:31.169 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.78677.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.78677.log.gz 2024-09-12T14:19:31.169 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.103164.log 2024-09-12T14:19:31.169 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.79435.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.79435.log.gz 2024-09-12T14:19:31.170 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.77661.log 2024-09-12T14:19:31.170 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.103164.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.103164.log.gz 2024-09-12T14:19:31.170 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.94067.log 2024-09-12T14:19:31.171 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.77661.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.77661.log.gz 2024-09-12T14:19:31.171 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.87167.log 2024-09-12T14:19:31.172 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.94067.log: gzip 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.94067.log.gz -5 2024-09-12T14:19:31.172 INFO:teuthology.orchestra.run.smithi130.stderr: --verbose -- /var/log/ceph/cluster1-client.mirror.101509.log 2024-09-12T14:19:31.173 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.87167.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.87167.log.gz 2024-09-12T14:19:31.173 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.38957.log 2024-09-12T14:19:31.174 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.101509.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.101509.log.gz 2024-09-12T14:19:31.174 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.86137.log 2024-09-12T14:19:31.175 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.38957.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.38957.log.gz 2024-09-12T14:19:31.175 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.47058.log 2024-09-12T14:19:31.175 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.86137.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.86137.log.gz 2024-09-12T14:19:31.175 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.149076.log 2024-09-12T14:19:31.176 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.47058.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.47058.log.gz 2024-09-12T14:19:31.176 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.102263.log 2024-09-12T14:19:31.177 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.149076.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.149076.log.gz 2024-09-12T14:19:31.177 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.128819.log 2024-09-12T14:19:31.178 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.102263.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.67760.log 2024-09-12T14:19:31.178 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.102263.log.gz 2024-09-12T14:19:31.178 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.128819.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.128819.log.gz 2024-09-12T14:19:31.178 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.34836.log 2024-09-12T14:19:31.179 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.67760.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.67760.log.gz 2024-09-12T14:19:31.179 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.42216.log 2024-09-12T14:19:31.179 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.34836.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.34836.log.gz 2024-09-12T14:19:31.180 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.92239.log 2024-09-12T14:19:31.180 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.42216.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.42216.log.gz 2024-09-12T14:19:31.180 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.62728.log 2024-09-12T14:19:31.181 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.92239.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.92239.log.gz 2024-09-12T14:19:31.182 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.131926.log 2024-09-12T14:19:31.182 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.62728.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.62728.log.gz 2024-09-12T14:19:31.182 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.113414.log 2024-09-12T14:19:31.183 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.131926.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.131926.log.gz 2024-09-12T14:19:31.183 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.71955.log 2024-09-12T14:19:31.183 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.113414.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.113414.log.gz 2024-09-12T14:19:31.183 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.65202.log 2024-09-12T14:19:31.184 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.71955.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.71955.log.gz 2024-09-12T14:19:31.184 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.77451.log 2024-09-12T14:19:31.185 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.65202.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.44486.log 2024-09-12T14:19:31.185 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.65202.log.gz 2024-09-12T14:19:31.186 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.77451.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.77451.log.gz 2024-09-12T14:19:31.186 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.119792.log 2024-09-12T14:19:31.186 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.44486.log: gzip -5 0.0% --verbose -- replaced with /var/log/ceph/cluster1-client.admin.44486.log.gz -- 2024-09-12T14:19:31.187 INFO:teuthology.orchestra.run.smithi130.stderr: /var/log/ceph/cluster1-client.mirror.109190.log 2024-09-12T14:19:31.187 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.119792.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.119792.log.gz 2024-09-12T14:19:31.187 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.118741.log 2024-09-12T14:19:31.188 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.109190.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.109190.log.gz 2024-09-12T14:19:31.188 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.54239.log 2024-09-12T14:19:31.189 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.118741.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.118741.log.gz 2024-09-12T14:19:31.189 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.102056.log 2024-09-12T14:19:31.189 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.54239.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.54239.log.gz 2024-09-12T14:19:31.190 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.58545.log 2024-09-12T14:19:31.190 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.102056.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.102056.log.gzgzip 2024-09-12T14:19:31.190 INFO:teuthology.orchestra.run.smithi130.stderr: -5 --verbose -- /var/log/ceph/cluster1-client.mirror.122145.log 2024-09-12T14:19:31.191 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.58545.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.58545.log.gz 2024-09-12T14:19:31.191 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.116467.log 2024-09-12T14:19:31.192 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.122145.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.122145.log.gz 2024-09-12T14:19:31.192 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.49054.log 2024-09-12T14:19:31.193 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.116467.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.116467.log.gz 2024-09-12T14:19:31.193 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.135391.log 2024-09-12T14:19:31.193 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.49054.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.49054.log.gz 2024-09-12T14:19:31.194 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.126740.log 2024-09-12T14:19:31.195 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.135391.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.135391.log.gz 2024-09-12T14:19:31.195 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.51886.log 2024-09-12T14:19:31.195 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.126740.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.126740.log.gz 2024-09-12T14:19:31.195 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.75706.log 2024-09-12T14:19:31.196 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.51886.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.51886.log.gz 2024-09-12T14:19:31.196 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.103618.log 2024-09-12T14:19:31.197 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.75706.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.75706.log.gz 2024-09-12T14:19:31.197 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.93818.log 2024-09-12T14:19:31.198 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.103618.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.103618.log.gz 2024-09-12T14:19:31.198 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.98190.log 2024-09-12T14:19:31.198 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.93818.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.93818.log.gz 2024-09-12T14:19:31.198 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.93886.log 2024-09-12T14:19:31.199 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.98190.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.98190.log.gz 2024-09-12T14:19:31.199 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.38075.log 2024-09-12T14:19:31.199 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.93886.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.93886.log.gz 2024-09-12T14:19:31.199 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.153370.log 2024-09-12T14:19:31.200 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.38075.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.38075.log.gz 2024-09-12T14:19:31.200 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.139951.log 2024-09-12T14:19:31.201 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.153370.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.153370.log.gz 2024-09-12T14:19:31.201 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.118695.log 2024-09-12T14:19:31.202 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.139951.log: gzip -5 0.0% --verbose -- replaced with /var/log/ceph/cluster1-client.admin.139951.log.gz -- 2024-09-12T14:19:31.202 INFO:teuthology.orchestra.run.smithi130.stderr: /var/log/ceph/cluster1-client.mirror.96383.log 2024-09-12T14:19:31.202 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.118695.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.118695.log.gz 2024-09-12T14:19:31.203 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.145083.log 2024-09-12T14:19:31.203 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.96383.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.96383.log.gz 2024-09-12T14:19:31.204 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.147119.log 2024-09-12T14:19:31.204 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.145083.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.145083.log.gz 2024-09-12T14:19:31.204 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.39280.log 2024-09-12T14:19:31.205 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.147119.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.147119.log.gz 2024-09-12T14:19:31.205 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.146544.log 2024-09-12T14:19:31.206 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.39280.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.39280.log.gz 2024-09-12T14:19:31.206 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.88913.log 2024-09-12T14:19:31.206 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.146544.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.146544.log.gz 2024-09-12T14:19:31.207 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.149760.log 2024-09-12T14:19:31.207 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.88913.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.88913.log.gz 2024-09-12T14:19:31.207 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.133671.log 2024-09-12T14:19:31.208 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.149760.log: gzip 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.149760.log.gz -5 2024-09-12T14:19:31.208 INFO:teuthology.orchestra.run.smithi130.stderr: --verbose -- /var/log/ceph/cluster1-client.mirror.62898.log 2024-09-12T14:19:31.209 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.133671.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.133671.log.gz 2024-09-12T14:19:31.209 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.49957.log 2024-09-12T14:19:31.209 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.62898.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.62898.log.gz 2024-09-12T14:19:31.210 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.45788.log 2024-09-12T14:19:31.210 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.49957.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.49957.log.gz 2024-09-12T14:19:31.210 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.67694.log 2024-09-12T14:19:31.211 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.45788.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.45788.log.gz 2024-09-12T14:19:31.211 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.123620.log 2024-09-12T14:19:31.212 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.67694.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.67694.log.gz 2024-09-12T14:19:31.212 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.53292.log 2024-09-12T14:19:31.213 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.123620.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.123620.log.gz 2024-09-12T14:19:31.213 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.45879.log 2024-09-12T14:19:31.213 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.53292.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.53292.log.gz 2024-09-12T14:19:31.213 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.64702.log 2024-09-12T14:19:31.214 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.45879.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.45879.log.gz 2024-09-12T14:19:31.214 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.128990.log 2024-09-12T14:19:31.215 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.64702.log: gzip 0.0% -5 -- replaced with /var/log/ceph/cluster1-client.mirror.64702.log.gz --verbose 2024-09-12T14:19:31.215 INFO:teuthology.orchestra.run.smithi130.stderr: -- /var/log/ceph/cluster1-client.admin.78825.log 2024-09-12T14:19:31.215 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.128990.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.128990.log.gz 2024-09-12T14:19:31.216 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.126983.log 2024-09-12T14:19:31.216 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.78825.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.78825.log.gz 2024-09-12T14:19:31.216 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.118962.log 2024-09-12T14:19:31.217 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.126983.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.148778.log 2024-09-12T14:19:31.217 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.126983.log.gz 2024-09-12T14:19:31.218 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.118962.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.118962.log.gz 2024-09-12T14:19:31.218 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.149785.log 2024-09-12T14:19:31.219 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.148778.log: gzip -5 --verbose 0.0% -- -- replaced with /var/log/ceph/cluster1-client.mirror.148778.log.gz /var/log/ceph/cluster1-client.mirror.86371.log 2024-09-12T14:19:31.219 INFO:teuthology.orchestra.run.smithi130.stderr: 2024-09-12T14:19:31.219 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.149785.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.149785.log.gz 2024-09-12T14:19:31.220 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.81426.log 2024-09-12T14:19:31.220 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.86371.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.86371.log.gz 2024-09-12T14:19:31.220 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.108668.log 2024-09-12T14:19:31.221 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.81426.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.81426.log.gz 2024-09-12T14:19:31.221 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.87270.log 2024-09-12T14:19:31.222 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.108668.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.108668.log.gz 2024-09-12T14:19:31.222 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.62787.log 2024-09-12T14:19:31.222 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.87270.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.87270.log.gz 2024-09-12T14:19:31.223 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.137837.log 2024-09-12T14:19:31.223 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.62787.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.62787.log.gz 2024-09-12T14:19:31.223 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.53990.log 2024-09-12T14:19:31.224 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.137837.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.137837.log.gz 2024-09-12T14:19:31.224 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.98632.log 2024-09-12T14:19:31.225 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.53990.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.53990.log.gz 2024-09-12T14:19:31.225 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.114065.log 2024-09-12T14:19:31.225 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.98632.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.98632.log.gz 2024-09-12T14:19:31.226 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.37824.log 2024-09-12T14:19:31.226 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.114065.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.114065.log.gz 2024-09-12T14:19:31.226 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.137264.log 2024-09-12T14:19:31.227 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.37824.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.37824.log.gz 2024-09-12T14:19:31.227 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.103274.log 2024-09-12T14:19:31.228 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.137264.log: gzip -5 --verbose 0.0% -- -- replaced with /var/log/ceph/cluster1-client.mirror.137264.log.gz /var/log/ceph/cluster2-client.mirror.152103.log 2024-09-12T14:19:31.228 INFO:teuthology.orchestra.run.smithi130.stderr: 2024-09-12T14:19:31.229 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.103274.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.103274.log.gz 2024-09-12T14:19:31.229 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.36050.log 2024-09-12T14:19:31.229 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.152103.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.152103.log.gz 2024-09-12T14:19:31.230 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.91011.log 2024-09-12T14:19:31.230 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.36050.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.36050.log.gz 2024-09-12T14:19:31.230 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.154509.log 2024-09-12T14:19:31.231 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.91011.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.91011.log.gzgzip 2024-09-12T14:19:31.231 INFO:teuthology.orchestra.run.smithi130.stderr: -5 --verbose -- /var/log/ceph/cluster1-client.mirror.53882.log 2024-09-12T14:19:31.232 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.154509.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.154509.log.gz 2024-09-12T14:19:31.232 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.147499.log 2024-09-12T14:19:31.233 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.53882.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.146449.log 2024-09-12T14:19:31.233 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.53882.log.gz 2024-09-12T14:19:31.234 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.147499.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.147499.log.gz 2024-09-12T14:19:31.234 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.130859.log 2024-09-12T14:19:31.235 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.146449.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.146449.log.gz 2024-09-12T14:19:31.235 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.111104.log 2024-09-12T14:19:31.235 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.130859.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.130859.log.gz 2024-09-12T14:19:31.235 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.88300.log 2024-09-12T14:19:31.236 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.111104.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.111104.log.gz 2024-09-12T14:19:31.236 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.67135.log 2024-09-12T14:19:31.236 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.88300.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.88300.log.gz 2024-09-12T14:19:31.236 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.137433.log 2024-09-12T14:19:31.237 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.67135.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.67135.log.gz 2024-09-12T14:19:31.237 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.65181.log 2024-09-12T14:19:31.238 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.137433.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.137433.log.gz 2024-09-12T14:19:31.238 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.139288.log 2024-09-12T14:19:31.239 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.65181.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.65181.log.gz 2024-09-12T14:19:31.239 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.98126.log 2024-09-12T14:19:31.240 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.139288.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.139288.log.gz 2024-09-12T14:19:31.240 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.110411.log 2024-09-12T14:19:31.240 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.98126.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.98126.log.gz 2024-09-12T14:19:31.240 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.39594.log 2024-09-12T14:19:31.241 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.110411.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.128821.log 2024-09-12T14:19:31.242 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.110411.log.gz 2024-09-12T14:19:31.242 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.39594.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.39594.log.gz 2024-09-12T14:19:31.242 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.126925.log 2024-09-12T14:19:31.242 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.128821.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.128821.log.gz 2024-09-12T14:19:31.243 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.91604.log 2024-09-12T14:19:31.243 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.126925.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.126925.log.gz 2024-09-12T14:19:31.243 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.101025.log 2024-09-12T14:19:31.244 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.91604.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.91604.log.gz 2024-09-12T14:19:31.244 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.59406.log 2024-09-12T14:19:31.245 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.101025.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.101025.log.gz 2024-09-12T14:19:31.245 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.37239.log 2024-09-12T14:19:31.246 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.59406.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.59406.log.gz 2024-09-12T14:19:31.246 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.74491.log 2024-09-12T14:19:31.247 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.37239.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.37239.log.gz 2024-09-12T14:19:31.247 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.56323.log 2024-09-12T14:19:31.248 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.74491.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.74491.log.gz 2024-09-12T14:19:31.248 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.153577.log 2024-09-12T14:19:31.249 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.56323.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.56323.log.gz 2024-09-12T14:19:31.249 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.83040.log 2024-09-12T14:19:31.249 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.153577.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.153577.log.gz 2024-09-12T14:19:31.250 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.66547.log 2024-09-12T14:19:31.250 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.83040.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.83040.log.gz 2024-09-12T14:19:31.251 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.143297.log 2024-09-12T14:19:31.251 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.66547.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.66547.log.gz 2024-09-12T14:19:31.252 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.136860.log 2024-09-12T14:19:31.252 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.143297.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.143297.log.gz 2024-09-12T14:19:31.253 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.149873.log 2024-09-12T14:19:31.254 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.136860.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.136860.log.gz 2024-09-12T14:19:31.254 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.103416.log 2024-09-12T14:19:31.254 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.149873.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.149873.log.gz 2024-09-12T14:19:31.255 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.36264.log 2024-09-12T14:19:31.256 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.103416.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.103416.log.gz 2024-09-12T14:19:31.256 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.131643.log 2024-09-12T14:19:31.256 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.36264.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.36264.log.gz 2024-09-12T14:19:31.257 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.87630.log 2024-09-12T14:19:31.257 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.131643.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.131643.log.gz 2024-09-12T14:19:31.258 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.115938.log 2024-09-12T14:19:31.258 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.87630.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.87630.log.gz 2024-09-12T14:19:31.259 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.104519.log 2024-09-12T14:19:31.259 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.115938.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.115938.log.gz 2024-09-12T14:19:31.259 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.114886.log 2024-09-12T14:19:31.260 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.104519.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.104519.log.gz 2024-09-12T14:19:31.260 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.70184.log 2024-09-12T14:19:31.261 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.114886.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.114886.log.gz 2024-09-12T14:19:31.262 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.73166.log 2024-09-12T14:19:31.262 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.70184.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.70184.log.gz 2024-09-12T14:19:31.262 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.79523.log 2024-09-12T14:19:31.263 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.73166.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.73166.log.gz 2024-09-12T14:19:31.263 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2.audit.log 2024-09-12T14:19:31.263 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.79523.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.79523.log.gz 2024-09-12T14:19:31.263 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.151986.log 2024-09-12T14:19:31.264 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2.audit.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.101358.log 2024-09-12T14:19:31.265 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.151986.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.151986.log.gz 2024-09-12T14:19:31.265 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.121762.log 2024-09-12T14:19:31.266 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.101358.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.101358.log.gz 2024-09-12T14:19:31.266 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.116338.log 2024-09-12T14:19:31.267 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.121762.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.121762.log.gz 2024-09-12T14:19:31.267 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.74658.log 2024-09-12T14:19:31.267 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.116338.log: gzip -5 --verbose -- 0.0% /var/log/ceph/cluster2-client.mirror.46085.log 2024-09-12T14:19:31.267 INFO:teuthology.orchestra.run.smithi130.stderr: -- replaced with /var/log/ceph/cluster2-client.admin.116338.log.gz 2024-09-12T14:19:31.268 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.74658.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.105591.log 2024-09-12T14:19:31.268 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.74658.log.gz 2024-09-12T14:19:31.269 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.46085.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.46085.log.gz 2024-09-12T14:19:31.269 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.151486.log 2024-09-12T14:19:31.270 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.105591.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.105591.log.gz 2024-09-12T14:19:31.270 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.132683.log 2024-09-12T14:19:31.271 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.151486.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.151486.log.gz 2024-09-12T14:19:31.271 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.46355.log 2024-09-12T14:19:31.271 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.132683.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.132683.log.gz 2024-09-12T14:19:31.272 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.68597.log 2024-09-12T14:19:31.273 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.46355.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.46355.log.gz 2024-09-12T14:19:31.273 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.53437.log 2024-09-12T14:19:31.273 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.68597.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.68597.log.gz 2024-09-12T14:19:31.273 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.94342.log 2024-09-12T14:19:31.274 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.53437.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.53437.log.gz 2024-09-12T14:19:31.274 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.153618.log 2024-09-12T14:19:31.275 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.94342.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.94342.log.gz 2024-09-12T14:19:31.275 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.85819.log 2024-09-12T14:19:31.276 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.153618.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.153618.log.gz 2024-09-12T14:19:31.276 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.124061.log 2024-09-12T14:19:31.277 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.85819.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.85819.log.gz 2024-09-12T14:19:31.277 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.150024.log 2024-09-12T14:19:31.278 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.124061.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.124061.log.gz 2024-09-12T14:19:31.278 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.112676.log 2024-09-12T14:19:31.279 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.150024.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.150024.log.gz 2024-09-12T14:19:31.279 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.69498.log 2024-09-12T14:19:31.279 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.112676.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.112676.log.gzgzip 2024-09-12T14:19:31.280 INFO:teuthology.orchestra.run.smithi130.stderr: -5 --verbose -- /var/log/ceph/cluster1-client.admin.82201.log 2024-09-12T14:19:31.281 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.69498.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.69498.log.gz 2024-09-12T14:19:31.281 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.51195.log 2024-09-12T14:19:31.281 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.82201.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.82201.log.gz 2024-09-12T14:19:31.281 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.100475.log 2024-09-12T14:19:31.282 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.51195.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.51195.log.gz 2024-09-12T14:19:31.282 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.60690.log 2024-09-12T14:19:31.282 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.100475.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.100475.log.gz 2024-09-12T14:19:31.283 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.98334.log 2024-09-12T14:19:31.284 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.60690.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.60690.log.gz 2024-09-12T14:19:31.284 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.49407.log 2024-09-12T14:19:31.284 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.98334.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.98334.log.gz 2024-09-12T14:19:31.284 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.66491.log 2024-09-12T14:19:31.285 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.49407.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.49407.log.gz 2024-09-12T14:19:31.285 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.151634.log 2024-09-12T14:19:31.285 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.66491.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.66491.log.gz 2024-09-12T14:19:31.286 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.127372.log 2024-09-12T14:19:31.287 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.151634.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.151634.log.gz 2024-09-12T14:19:31.287 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.121152.log 2024-09-12T14:19:31.287 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.127372.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.127372.log.gz 2024-09-12T14:19:31.287 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.40831.log 2024-09-12T14:19:31.288 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.121152.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.121152.log.gz 2024-09-12T14:19:31.289 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.69936.log 2024-09-12T14:19:31.289 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.40831.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.40831.log.gz 2024-09-12T14:19:31.289 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.145461.log 2024-09-12T14:19:31.290 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.69936.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.69936.log.gz 2024-09-12T14:19:31.290 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.149075.log 2024-09-12T14:19:31.290 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.145461.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.145461.log.gz 2024-09-12T14:19:31.290 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.114998.log 2024-09-12T14:19:31.291 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.149075.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.149075.log.gz 2024-09-12T14:19:31.292 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.143249.log 2024-09-12T14:19:31.292 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.114998.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.114998.log.gz 2024-09-12T14:19:31.292 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.54848.log 2024-09-12T14:19:31.293 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.143249.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.143249.log.gz 2024-09-12T14:19:31.293 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.139454.log 2024-09-12T14:19:31.293 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.54848.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.54848.log.gz 2024-09-12T14:19:31.293 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.131443.log 2024-09-12T14:19:31.294 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.139454.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.139454.log.gz 2024-09-12T14:19:31.294 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.80261.log 2024-09-12T14:19:31.295 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.131443.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.131443.log.gz 2024-09-12T14:19:31.295 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.121048.log 2024-09-12T14:19:31.296 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.80261.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.80261.log.gz 2024-09-12T14:19:31.296 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.41440.log 2024-09-12T14:19:31.297 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.121048.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.117696.log 2024-09-12T14:19:31.297 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.121048.log.gz 2024-09-12T14:19:31.297 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.41440.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.41440.log.gz 2024-09-12T14:19:31.297 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.59911.log 2024-09-12T14:19:31.298 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.117696.log: gzip -5 --verbose -- 0.0% /var/log/ceph/cluster1-client.mirror.52349.log 2024-09-12T14:19:31.298 INFO:teuthology.orchestra.run.smithi130.stderr: -- replaced with /var/log/ceph/cluster1-client.admin.117696.log.gz 2024-09-12T14:19:31.299 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.59911.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.59911.log.gz 2024-09-12T14:19:31.299 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.136043.log 2024-09-12T14:19:31.299 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.52349.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.46105.log 2024-09-12T14:19:31.299 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.52349.log.gz 2024-09-12T14:19:31.300 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.136043.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.136043.log.gz 2024-09-12T14:19:31.300 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.76677.log 2024-09-12T14:19:31.301 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.46105.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.46105.log.gz 2024-09-12T14:19:31.301 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.70050.log 2024-09-12T14:19:31.302 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.76677.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.76677.log.gz 2024-09-12T14:19:31.302 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.46790.log 2024-09-12T14:19:31.303 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.70050.log: gzip 0.0% -5 -- replaced with /var/log/ceph/cluster1-client.mirror.70050.log.gz --verbose 2024-09-12T14:19:31.303 INFO:teuthology.orchestra.run.smithi130.stderr: -- /var/log/ceph/cluster2-client.mirror.82361.log 2024-09-12T14:19:31.303 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.46790.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.46790.log.gz 2024-09-12T14:19:31.303 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.76065.log 2024-09-12T14:19:31.304 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.82361.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.82361.log.gz 2024-09-12T14:19:31.304 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.114106.log 2024-09-12T14:19:31.304 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.76065.log: gzip 0.0% -5 -- replaced with /var/log/ceph/cluster1-client.mirror.76065.log.gz --verbose 2024-09-12T14:19:31.305 INFO:teuthology.orchestra.run.smithi130.stderr: -- /var/log/ceph/cluster2-client.admin.93198.log 2024-09-12T14:19:31.305 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.114106.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.83002.log 2024-09-12T14:19:31.305 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.114106.log.gz 2024-09-12T14:19:31.306 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.93198.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.93198.log.gz 2024-09-12T14:19:31.306 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.62727.log 2024-09-12T14:19:31.307 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.83002.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.83002.log.gz 2024-09-12T14:19:31.307 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.148459.log 2024-09-12T14:19:31.308 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.62727.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.68008.log 2024-09-12T14:19:31.308 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.62727.log.gz 2024-09-12T14:19:31.309 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.148459.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.74573.log 2024-09-12T14:19:31.309 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.148459.log.gz 2024-09-12T14:19:31.309 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.68008.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.68008.log.gz 2024-09-12T14:19:31.310 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.42299.log 2024-09-12T14:19:31.311 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.74573.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.74573.log.gz 2024-09-12T14:19:31.311 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.37995.log 2024-09-12T14:19:31.311 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.42299.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.42299.log.gz 2024-09-12T14:19:31.311 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.150650.log 2024-09-12T14:19:31.312 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.37995.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.37995.log.gz 2024-09-12T14:19:31.313 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.42967.log 2024-09-12T14:19:31.313 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.150650.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.150650.log.gz 2024-09-12T14:19:31.313 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.39995.log 2024-09-12T14:19:31.314 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.42967.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.42967.log.gz 2024-09-12T14:19:31.314 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.70526.log 2024-09-12T14:19:31.315 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.39995.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.39995.log.gz 2024-09-12T14:19:31.315 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.43830.log 2024-09-12T14:19:31.315 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.70526.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.70526.log.gz 2024-09-12T14:19:31.315 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.53569.log 2024-09-12T14:19:31.316 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.43830.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.43830.log.gz 2024-09-12T14:19:31.317 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.118325.log 2024-09-12T14:19:31.317 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.53569.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.53569.log.gz 2024-09-12T14:19:31.317 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.102252.log 2024-09-12T14:19:31.318 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.118325.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.118325.log.gz 2024-09-12T14:19:31.318 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.89518.log 2024-09-12T14:19:31.318 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.102252.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.102252.log.gz 2024-09-12T14:19:31.318 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.97686.log 2024-09-12T14:19:31.319 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.89518.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.89518.log.gz 2024-09-12T14:19:31.319 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.67430.log 2024-09-12T14:19:31.320 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.97686.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.97686.log.gz 2024-09-12T14:19:31.320 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.58747.log 2024-09-12T14:19:31.321 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.67430.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.67430.log.gz 2024-09-12T14:19:31.321 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.46219.log 2024-09-12T14:19:31.322 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.58747.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.58747.log.gz 2024-09-12T14:19:31.322 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.143424.log 2024-09-12T14:19:31.323 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.46219.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.46219.log.gz 2024-09-12T14:19:31.323 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.67496.log 2024-09-12T14:19:31.323 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.143424.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.143424.log.gz 2024-09-12T14:19:31.324 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.127966.log 2024-09-12T14:19:31.325 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.67496.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.67496.log.gz 2024-09-12T14:19:31.325 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.100353.log 2024-09-12T14:19:31.325 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.127966.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.127966.log.gz 2024-09-12T14:19:31.325 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.75664.log 2024-09-12T14:19:31.326 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.100353.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.100353.log.gz 2024-09-12T14:19:31.326 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.92888.log 2024-09-12T14:19:31.326 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.75664.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.75664.log.gz 2024-09-12T14:19:31.326 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.138917.log 2024-09-12T14:19:31.327 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.92888.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.92888.log.gz 2024-09-12T14:19:31.327 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.147288.log 2024-09-12T14:19:31.328 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.138917.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.138917.log.gz 2024-09-12T14:19:31.328 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.70770.log 2024-09-12T14:19:31.329 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.147288.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.147288.log.gz 2024-09-12T14:19:31.329 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.68727.log 2024-09-12T14:19:31.330 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.70770.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.70770.log.gz 2024-09-12T14:19:31.330 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.57703.log 2024-09-12T14:19:31.331 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.68727.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.68727.log.gz 2024-09-12T14:19:31.331 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.46811.log 2024-09-12T14:19:31.331 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.57703.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.57703.log.gz 2024-09-12T14:19:31.331 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.145752.log 2024-09-12T14:19:31.333 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.46811.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.46811.log.gz 2024-09-12T14:19:31.333 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.51406.log 2024-09-12T14:19:31.333 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.145752.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.145752.log.gz 2024-09-12T14:19:31.333 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.79083.log 2024-09-12T14:19:31.334 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.51406.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.51406.log.gz 2024-09-12T14:19:31.334 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.71782.log 2024-09-12T14:19:31.334 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.79083.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.79083.log.gz 2024-09-12T14:19:31.335 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.67363.log 2024-09-12T14:19:31.335 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.71782.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.71782.log.gz 2024-09-12T14:19:31.336 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.117427.log 2024-09-12T14:19:31.337 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.67363.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.67363.log.gz 2024-09-12T14:19:31.337 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.116974.log 2024-09-12T14:19:31.337 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.117427.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.130575.log 2024-09-12T14:19:31.338 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.117427.log.gz 2024-09-12T14:19:31.338 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.116974.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.116974.log.gz 2024-09-12T14:19:31.338 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.53277.log 2024-09-12T14:19:31.340 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.130575.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.130575.log.gz 2024-09-12T14:19:31.340 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.47601.log 2024-09-12T14:19:31.340 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.53277.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.53277.log.gz 2024-09-12T14:19:31.340 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.36265.log 2024-09-12T14:19:31.341 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.47601.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.47601.log.gz 2024-09-12T14:19:31.341 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.138409.log 2024-09-12T14:19:31.342 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.36265.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.142854.log 2024-09-12T14:19:31.342 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.36265.log.gz 2024-09-12T14:19:31.342 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.138409.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.111923.log 0.0% 2024-09-12T14:19:31.342 INFO:teuthology.orchestra.run.smithi130.stderr: -- replaced with /var/log/ceph/cluster1-client.admin.138409.log.gz 2024-09-12T14:19:31.343 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.142854.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.142854.log.gz 2024-09-12T14:19:31.344 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.105296.log 2024-09-12T14:19:31.344 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.111923.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.111923.log.gz 2024-09-12T14:19:31.344 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.91166.log 2024-09-12T14:19:31.344 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.105296.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.121765.log 2024-09-12T14:19:31.345 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.105296.log.gz 2024-09-12T14:19:31.345 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.91166.log: gzip -5 --verbose 0.0% -- -- replaced with /var/log/ceph/cluster1-client.admin.91166.log.gz /var/log/ceph/cluster1-client.mirror.86953.log 2024-09-12T14:19:31.346 INFO:teuthology.orchestra.run.smithi130.stderr: 2024-09-12T14:19:31.347 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.121765.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.121765.log.gz 2024-09-12T14:19:31.347 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.38578.log 2024-09-12T14:19:31.347 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.86953.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.86953.log.gz 2024-09-12T14:19:31.347 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.137305.log 2024-09-12T14:19:31.348 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.38578.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.38578.log.gz 2024-09-12T14:19:31.348 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.96006.log 2024-09-12T14:19:31.348 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.137305.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.137305.log.gz 2024-09-12T14:19:31.349 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.124167.log 2024-09-12T14:19:31.350 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.96006.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.96006.log.gz 2024-09-12T14:19:31.350 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.140394.log 2024-09-12T14:19:31.350 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.124167.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.124167.log.gz 2024-09-12T14:19:31.350 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.98128.log 2024-09-12T14:19:31.351 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.140394.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.140394.log.gz 2024-09-12T14:19:31.352 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.39215.log 2024-09-12T14:19:31.352 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.98128.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.98128.log.gz 2024-09-12T14:19:31.352 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.138520.log 2024-09-12T14:19:31.354 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.39215.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.39215.log.gz 2024-09-12T14:19:31.354 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.104012.log 2024-09-12T14:19:31.354 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.138520.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.138520.log.gz 2024-09-12T14:19:31.354 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.59948.log 2024-09-12T14:19:31.354 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.104012.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.104012.log.gz 2024-09-12T14:19:31.354 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.94160.log 2024-09-12T14:19:31.355 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.59948.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.59948.log.gz 2024-09-12T14:19:31.356 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.135307.log 2024-09-12T14:19:31.356 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.94160.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.94160.log.gz 2024-09-12T14:19:31.356 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.91709.log 2024-09-12T14:19:31.357 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.135307.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.135307.log.gz 2024-09-12T14:19:31.357 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.148249.log 2024-09-12T14:19:31.357 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.91709.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.91709.log.gz 2024-09-12T14:19:31.357 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.134467.log 2024-09-12T14:19:31.359 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.148249.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.148249.log.gz 2024-09-12T14:19:31.359 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.58130.log 2024-09-12T14:19:31.359 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.134467.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.134467.log.gz 2024-09-12T14:19:31.359 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.97113.log 2024-09-12T14:19:31.360 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.58130.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.58130.log.gz 2024-09-12T14:19:31.361 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.70734.log 2024-09-12T14:19:31.361 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.97113.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.97113.log.gz 2024-09-12T14:19:31.361 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.96779.log 2024-09-12T14:19:31.362 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.70734.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.70734.log.gz 2024-09-12T14:19:31.362 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.134888.log 2024-09-12T14:19:31.362 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.96779.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.96779.log.gz 2024-09-12T14:19:31.362 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.81465.log 2024-09-12T14:19:31.364 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.134888.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.134888.log.gz 2024-09-12T14:19:31.364 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.138156.log 2024-09-12T14:19:31.364 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.81465.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.81465.log.gz 2024-09-12T14:19:31.364 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.48909.log 2024-09-12T14:19:31.365 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.138156.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.138156.log.gz 2024-09-12T14:19:31.365 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.133166.log 2024-09-12T14:19:31.366 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.48909.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.48909.log.gz 2024-09-12T14:19:31.366 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.87582.log 2024-09-12T14:19:31.367 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.133166.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.133166.log.gz 2024-09-12T14:19:31.367 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.108293.log 2024-09-12T14:19:31.367 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.87582.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.87582.log.gz 2024-09-12T14:19:31.367 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.140087.log 2024-09-12T14:19:31.368 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.108293.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.108293.log.gz 2024-09-12T14:19:31.368 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.76423.log 2024-09-12T14:19:31.369 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.140087.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.40789.log 2024-09-12T14:19:31.370 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.140087.log.gz 2024-09-12T14:19:31.370 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.76423.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.76423.log.gz 2024-09-12T14:19:31.370 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.61252.log 2024-09-12T14:19:31.371 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.40789.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.40789.log.gz 2024-09-12T14:19:31.371 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.62139.log 2024-09-12T14:19:31.371 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.61252.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.61252.log.gz 2024-09-12T14:19:31.371 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.93052.log 2024-09-12T14:19:31.373 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.62139.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.62139.log.gz 2024-09-12T14:19:31.373 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.89601.log 2024-09-12T14:19:31.373 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.93052.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.93052.log.gz 2024-09-12T14:19:31.373 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.48023.log 2024-09-12T14:19:31.374 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.89601.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.89601.log.gz 2024-09-12T14:19:31.375 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.61534.log 2024-09-12T14:19:31.375 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.48023.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.48023.log.gz 2024-09-12T14:19:31.375 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.63397.log 2024-09-12T14:19:31.376 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.61534.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.38435.log 2024-09-12T14:19:31.376 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.61534.log.gz 2024-09-12T14:19:31.376 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.63397.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.106006.log 0.0% 2024-09-12T14:19:31.376 INFO:teuthology.orchestra.run.smithi130.stderr: -- replaced with /var/log/ceph/cluster1-client.mirror.63397.log.gz 2024-09-12T14:19:31.376 INFO:teuthology.orchestra.run.smithi130.stderr: 92.7% -- replaced with /var/log/ceph/cluster2.audit.log.gz 2024-09-12T14:19:31.377 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.38435.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.38435.log.gz 2024-09-12T14:19:31.377 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.152279.log 2024-09-12T14:19:31.378 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.106006.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.106006.log.gz 2024-09-12T14:19:31.378 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.117723.log 2024-09-12T14:19:31.379 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.152279.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.152279.log.gz 2024-09-12T14:19:31.379 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.43133.log 2024-09-12T14:19:31.379 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.117723.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.117723.log.gz 2024-09-12T14:19:31.380 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.64355.log 2024-09-12T14:19:31.380 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.43133.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.43133.log.gz 2024-09-12T14:19:31.380 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.88383.log 2024-09-12T14:19:31.381 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.64355.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.64355.log.gz 2024-09-12T14:19:31.381 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.96382.log 2024-09-12T14:19:31.382 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.88383.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.88383.log.gz 2024-09-12T14:19:31.383 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.153256.log 2024-09-12T14:19:31.383 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.96382.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.96382.log.gz 2024-09-12T14:19:31.383 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.44990.log 2024-09-12T14:19:31.384 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.153256.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.153256.log.gz 2024-09-12T14:19:31.384 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.101783.log 2024-09-12T14:19:31.384 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.44990.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.44990.log.gz 2024-09-12T14:19:31.384 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.78336.log 2024-09-12T14:19:31.385 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.101783.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.101783.log.gz 2024-09-12T14:19:31.385 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.71871.log 2024-09-12T14:19:31.386 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.78336.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.78336.log.gz 2024-09-12T14:19:31.386 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.57012.log 2024-09-12T14:19:31.387 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.71871.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.71871.log.gz 2024-09-12T14:19:31.387 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.78125.log 2024-09-12T14:19:31.388 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.57012.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.57012.log.gz 2024-09-12T14:19:31.388 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.41335.log 2024-09-12T14:19:31.388 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.78125.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.78125.log.gz 2024-09-12T14:19:31.388 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.138632.log 2024-09-12T14:19:31.389 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.41335.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.41335.log.gz 2024-09-12T14:19:31.389 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.103186.log 2024-09-12T14:19:31.390 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.138632.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.138632.log.gz 2024-09-12T14:19:31.390 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.142380.log 2024-09-12T14:19:31.391 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.103186.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.103186.log.gz 2024-09-12T14:19:31.391 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.83699.log 2024-09-12T14:19:31.391 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.142380.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.142380.log.gz 2024-09-12T14:19:31.392 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.134996.log 2024-09-12T14:19:31.392 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.83699.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.125518.log 2024-09-12T14:19:31.393 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.83699.log.gz 2024-09-12T14:19:31.393 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.134996.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.134996.log.gz 2024-09-12T14:19:31.394 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.47669.log 2024-09-12T14:19:31.394 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.125518.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.125518.log.gz 2024-09-12T14:19:31.395 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.89012.log 2024-09-12T14:19:31.395 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.47669.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.47669.log.gz 2024-09-12T14:19:31.395 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.118479.log 2024-09-12T14:19:31.396 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.89012.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.89012.log.gz 2024-09-12T14:19:31.396 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.150408.log 2024-09-12T14:19:31.397 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.118479.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.118479.log.gz 2024-09-12T14:19:31.397 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.54652.log 2024-09-12T14:19:31.398 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.150408.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.150408.log.gz 2024-09-12T14:19:31.398 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.110977.log 2024-09-12T14:19:31.398 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.54652.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.54652.log.gz 2024-09-12T14:19:31.398 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.77469.log 2024-09-12T14:19:31.399 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.110977.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.110977.log.gz 2024-09-12T14:19:31.399 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.58720.log 2024-09-12T14:19:31.400 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.77469.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.77469.log.gz 2024-09-12T14:19:31.400 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.73857.log 2024-09-12T14:19:31.401 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.58720.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.58720.log.gz 2024-09-12T14:19:31.401 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.109527.log 2024-09-12T14:19:31.402 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.73857.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.73857.log.gz 2024-09-12T14:19:31.402 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.90023.log 2024-09-12T14:19:31.404 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.109527.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.109527.log.gz 2024-09-12T14:19:31.404 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.95960.log 2024-09-12T14:19:31.404 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.90023.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.90023.log.gz 2024-09-12T14:19:31.404 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.57975.log 2024-09-12T14:19:31.405 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.95960.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.95960.log.gz 2024-09-12T14:19:31.405 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.49829.log 2024-09-12T14:19:31.405 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.57975.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.57975.log.gz 2024-09-12T14:19:31.405 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.74572.log 2024-09-12T14:19:31.406 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.49829.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.49829.log.gz 2024-09-12T14:19:31.407 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.134466.log 2024-09-12T14:19:31.407 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.74572.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.110244.log 2024-09-12T14:19:31.407 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.74572.log.gz 2024-09-12T14:19:31.407 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.134466.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.134466.log.gz 2024-09-12T14:19:31.407 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.133838.log 2024-09-12T14:19:31.408 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.110244.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.110244.log.gz 2024-09-12T14:19:31.409 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.136296.log 2024-09-12T14:19:31.409 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.133838.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.133838.log.gz 2024-09-12T14:19:31.409 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.118763.log 2024-09-12T14:19:31.410 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.136296.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.136296.log.gz 2024-09-12T14:19:31.410 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.124861.log 2024-09-12T14:19:31.410 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.118763.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.118763.log.gz 2024-09-12T14:19:31.411 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.149684.log 2024-09-12T14:19:31.413 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.124861.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.147660.log 2024-09-12T14:19:31.413 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.124861.log.gz 2024-09-12T14:19:31.413 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.149684.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.149684.log.gz 2024-09-12T14:19:31.413 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.67987.log 2024-09-12T14:19:31.413 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.147660.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.147660.log.gz 2024-09-12T14:19:31.413 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.94114.log 2024-09-12T14:19:31.414 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.67987.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.67987.log.gz 2024-09-12T14:19:31.414 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.142574.log 2024-09-12T14:19:31.414 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.94114.log: gzip 0.0% -5 --verbose -- -- replaced with /var/log/ceph/cluster1-client.admin.94114.log.gz /var/log/ceph/cluster1-client.mirror.137536.log 2024-09-12T14:19:31.414 INFO:teuthology.orchestra.run.smithi130.stderr: 2024-09-12T14:19:31.415 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.142574.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.110854.log 2024-09-12T14:19:31.415 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.142574.log.gz 2024-09-12T14:19:31.416 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.137536.log: 0.0%gzip -- replaced with /var/log/ceph/cluster1-client.mirror.137536.log.gz 2024-09-12T14:19:31.416 INFO:teuthology.orchestra.run.smithi130.stderr: -5 --verbose -- /var/log/ceph/cluster1-client.mirror.95784.log 2024-09-12T14:19:31.416 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.110854.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.110854.log.gz 2024-09-12T14:19:31.417 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.119852.log 2024-09-12T14:19:31.417 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.95784.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.106990.log 2024-09-12T14:19:31.418 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.95784.log.gz 2024-09-12T14:19:31.418 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.119852.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.119852.log.gz 2024-09-12T14:19:31.419 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.121656.log 2024-09-12T14:19:31.420 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.106990.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.106990.log.gz 2024-09-12T14:19:31.420 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.89832.log 2024-09-12T14:19:31.420 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.121656.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.121656.log.gz 2024-09-12T14:19:31.421 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.59642.log 2024-09-12T14:19:31.421 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.89832.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.89832.log.gz 2024-09-12T14:19:31.421 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.74091.log 2024-09-12T14:19:31.422 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.59642.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.59642.log.gz 2024-09-12T14:19:31.422 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.38752.log 2024-09-12T14:19:31.423 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.74091.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.74091.log.gz 2024-09-12T14:19:31.423 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.69511.log 2024-09-12T14:19:31.424 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.38752.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.38752.log.gz 2024-09-12T14:19:31.424 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.76338.log 2024-09-12T14:19:31.424 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.69511.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.69511.log.gz 2024-09-12T14:19:31.424 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.49784.log 2024-09-12T14:19:31.425 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.76338.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.76338.log.gz 2024-09-12T14:19:31.425 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.114602.log 2024-09-12T14:19:31.426 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.49784.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.49784.log.gz 2024-09-12T14:19:31.426 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.58586.log 2024-09-12T14:19:31.426 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.114602.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.114602.log.gz 2024-09-12T14:19:31.427 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.58907.log 2024-09-12T14:19:31.428 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.58586.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.58586.log.gz 2024-09-12T14:19:31.428 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.45498.log 2024-09-12T14:19:31.428 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.58907.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.58907.log.gz 2024-09-12T14:19:31.428 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.85318.log 2024-09-12T14:19:31.429 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.45498.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.45498.log.gz 2024-09-12T14:19:31.429 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.55899.log 2024-09-12T14:19:31.430 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.85318.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.85318.log.gz 2024-09-12T14:19:31.430 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.143857.log 2024-09-12T14:19:31.431 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.55899.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.55899.log.gz 2024-09-12T14:19:31.431 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.130264.log 2024-09-12T14:19:31.431 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.143857.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.143857.log.gz 2024-09-12T14:19:31.431 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.101675.log 2024-09-12T14:19:31.432 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.130264.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.130264.log.gz 2024-09-12T14:19:31.433 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.62181.log 2024-09-12T14:19:31.433 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.101675.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.101675.log.gz 2024-09-12T14:19:31.433 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.128597.log 2024-09-12T14:19:31.435 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.62181.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.62181.log.gz 2024-09-12T14:19:31.435 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.84624.log 2024-09-12T14:19:31.435 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.128597.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.128597.log.gz 2024-09-12T14:19:31.435 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.152706.log 2024-09-12T14:19:31.435 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.84624.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.84624.log.gz 2024-09-12T14:19:31.435 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.84343.log 2024-09-12T14:19:31.437 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.152706.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.152706.log.gz 2024-09-12T14:19:31.437 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.68873.log 2024-09-12T14:19:31.438 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.84343.log: 0.0%gzip -- replaced with /var/log/ceph/cluster1-client.mirror.84343.log.gz 2024-09-12T14:19:31.438 INFO:teuthology.orchestra.run.smithi130.stderr: -5 --verbose -- /var/log/ceph/cluster1-client.admin.137889.log 2024-09-12T14:19:31.438 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.68873.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.68873.log.gz 2024-09-12T14:19:31.438 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.55031.log 2024-09-12T14:19:31.440 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.137889.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.137889.log.gz 2024-09-12T14:19:31.440 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.70864.log 2024-09-12T14:19:31.440 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.55031.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.55031.log.gz 2024-09-12T14:19:31.440 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.72706.log 2024-09-12T14:19:31.440 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.70864.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.70864.log.gz 2024-09-12T14:19:31.440 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.122603.log 2024-09-12T14:19:31.443 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.72706.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.72706.log.gz 2024-09-12T14:19:31.443 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.108604.log 2024-09-12T14:19:31.443 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.122603.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.122603.log.gz 2024-09-12T14:19:31.443 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.35980.log 2024-09-12T14:19:31.443 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.108604.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.108604.log.gz 2024-09-12T14:19:31.443 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.101088.log 2024-09-12T14:19:31.443 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.35980.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.35980.log.gz 2024-09-12T14:19:31.444 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.58519.log 2024-09-12T14:19:31.446 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.101088.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.101088.log.gz 2024-09-12T14:19:31.446 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.144598.log 2024-09-12T14:19:31.446 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.58519.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.58519.log.gz 2024-09-12T14:19:31.446 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.75640.log 2024-09-12T14:19:31.447 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.144598.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.144598.log.gz 2024-09-12T14:19:31.447 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.123953.log 2024-09-12T14:19:31.447 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.75640.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.75640.log.gz 2024-09-12T14:19:31.447 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.47906.log 2024-09-12T14:19:31.449 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.123953.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.123953.log.gz 2024-09-12T14:19:31.449 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.86240.log 2024-09-12T14:19:31.449 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.47906.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.47906.log.gz 2024-09-12T14:19:31.450 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.76192.log 2024-09-12T14:19:31.450 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.86240.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.131654.log 2024-09-12T14:19:31.450 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.86240.log.gz 2024-09-12T14:19:31.450 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.76192.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.76192.log.gz 2024-09-12T14:19:31.450 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.123704.log 2024-09-12T14:19:31.451 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.131654.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.131654.log.gz 2024-09-12T14:19:31.451 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.111733.log 2024-09-12T14:19:31.451 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.123704.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.123704.log.gz 2024-09-12T14:19:31.452 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.55834.log 2024-09-12T14:19:31.453 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.111733.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.111733.log.gz 2024-09-12T14:19:31.453 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.150319.log 2024-09-12T14:19:31.453 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.55834.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.55834.log.gz 2024-09-12T14:19:31.453 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.118916.log 2024-09-12T14:19:31.454 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.150319.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.150319.log.gz 2024-09-12T14:19:31.455 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.66023.log 2024-09-12T14:19:31.455 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.118916.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.118916.log.gz 2024-09-12T14:19:31.455 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.50628.log 2024-09-12T14:19:31.456 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.66023.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.66023.log.gz 2024-09-12T14:19:31.456 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.43003.log 2024-09-12T14:19:31.456 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.50628.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.50628.log.gz 2024-09-12T14:19:31.456 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.83920.log 2024-09-12T14:19:31.457 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.43003.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.43003.log.gz 2024-09-12T14:19:31.458 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.75623.log 2024-09-12T14:19:31.458 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.83920.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.83920.log.gz 2024-09-12T14:19:31.458 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.61234.log 2024-09-12T14:19:31.459 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.75623.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.75623.log.gz 2024-09-12T14:19:31.459 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.111106.log 2024-09-12T14:19:31.459 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.61234.log: 0.0%gzip -- replaced with /var/log/ceph/cluster1-client.mirror.61234.log.gz 2024-09-12T14:19:31.459 INFO:teuthology.orchestra.run.smithi130.stderr: -5 --verbose -- /var/log/ceph/cluster1-client.mirror.84480.log 2024-09-12T14:19:31.460 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.111106.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.111106.log.gz 2024-09-12T14:19:31.461 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.42256.log 2024-09-12T14:19:31.461 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.84480.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.84480.log.gz 2024-09-12T14:19:31.461 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.59759.log 2024-09-12T14:19:31.462 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.42256.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.42256.log.gz 2024-09-12T14:19:31.462 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.129678.log 2024-09-12T14:19:31.462 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.59759.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.59759.log.gz 2024-09-12T14:19:31.462 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.140573.log 2024-09-12T14:19:31.463 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.129678.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.129678.log.gz 2024-09-12T14:19:31.463 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.38141.log 2024-09-12T14:19:31.464 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.140573.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.140573.log.gz 2024-09-12T14:19:31.464 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.53356.log 2024-09-12T14:19:31.465 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.38141.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.38141.log.gz 2024-09-12T14:19:31.465 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.88765.log 2024-09-12T14:19:31.466 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.53356.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.53356.log.gz 2024-09-12T14:19:31.466 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.52602.log 2024-09-12T14:19:31.467 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.88765.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.88765.log.gz 2024-09-12T14:19:31.467 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.58814.log 2024-09-12T14:19:31.467 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.52602.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.52602.log.gz 2024-09-12T14:19:31.467 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.63776.log 2024-09-12T14:19:31.468 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.58814.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.58814.log.gz 2024-09-12T14:19:31.468 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.85989.log 2024-09-12T14:19:31.469 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.63776.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.63776.log.gz 2024-09-12T14:19:31.469 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.98127.log 2024-09-12T14:19:31.470 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.85989.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.85989.log.gz 2024-09-12T14:19:31.470 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.58793.log 2024-09-12T14:19:31.470 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.98127.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.98127.log.gz 2024-09-12T14:19:31.470 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.48590.log 2024-09-12T14:19:31.471 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.58793.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.58793.log.gz 2024-09-12T14:19:31.471 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.94996.log 2024-09-12T14:19:31.471 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.48590.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.48590.log.gz 2024-09-12T14:19:31.471 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.83134.log 2024-09-12T14:19:31.472 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.94996.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.94996.log.gz 2024-09-12T14:19:31.472 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.35718.log 2024-09-12T14:19:31.473 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.83134.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.83134.log.gz 2024-09-12T14:19:31.473 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.65203.log 2024-09-12T14:19:31.474 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.35718.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.35718.log.gzgzip 2024-09-12T14:19:31.474 INFO:teuthology.orchestra.run.smithi130.stderr: -5 --verbose -- /var/log/ceph/cluster2-mgr.x.log 2024-09-12T14:19:31.474 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.65203.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.65203.log.gz 2024-09-12T14:19:31.475 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.83982.log 2024-09-12T14:19:31.475 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.90001.log 2024-09-12T14:19:31.476 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.83982.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.83982.log.gz 2024-09-12T14:19:31.476 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.140919.log 2024-09-12T14:19:31.477 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.90001.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.90001.log.gz 2024-09-12T14:19:31.477 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.66130.log 2024-09-12T14:19:31.478 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.140919.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.140919.log.gz 2024-09-12T14:19:31.478 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.139206.log 2024-09-12T14:19:31.478 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.66130.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.66130.log.gz 2024-09-12T14:19:31.479 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.99096.log 2024-09-12T14:19:31.479 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.139206.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.139206.log.gz 2024-09-12T14:19:31.479 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.105798.log 2024-09-12T14:19:31.480 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.99096.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.99096.log.gz 2024-09-12T14:19:31.480 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.51591.log 2024-09-12T14:19:31.481 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.105798.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.105798.log.gz 2024-09-12T14:19:31.481 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.125686.log 2024-09-12T14:19:31.482 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.51591.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.51591.log.gz 2024-09-12T14:19:31.482 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.65037.log 2024-09-12T14:19:31.482 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.125686.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.125686.log.gz 2024-09-12T14:19:31.483 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.144461.log 2024-09-12T14:19:31.483 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.65037.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.65037.log.gzgzip 2024-09-12T14:19:31.484 INFO:teuthology.orchestra.run.smithi130.stderr: -5 --verbose -- /var/log/ceph/cluster2-client.mirror.82133.log 2024-09-12T14:19:31.484 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.144461.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.144461.log.gz 2024-09-12T14:19:31.484 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.77727.log 2024-09-12T14:19:31.485 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.82133.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.143793.log 2024-09-12T14:19:31.485 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.82133.log.gz 2024-09-12T14:19:31.486 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.77727.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.77727.log.gz 2024-09-12T14:19:31.486 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.94755.log 2024-09-12T14:19:31.487 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.143793.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.143793.log.gz 2024-09-12T14:19:31.487 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.87793.log 2024-09-12T14:19:31.487 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.94755.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.94755.log.gz 2024-09-12T14:19:31.487 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.77638.log 2024-09-12T14:19:31.488 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.87793.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.37448.log 2024-09-12T14:19:31.488 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.87793.log.gz 2024-09-12T14:19:31.489 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.77638.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.77638.log.gz 2024-09-12T14:19:31.489 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.80765.log 2024-09-12T14:19:31.490 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.37448.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.37448.log.gz 2024-09-12T14:19:31.490 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.75119.log 2024-09-12T14:19:31.491 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.80765.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.80765.log.gz 2024-09-12T14:19:31.491 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.86895.log 2024-09-12T14:19:31.492 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.75119.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.75119.log.gz 2024-09-12T14:19:31.492 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.102338.log 2024-09-12T14:19:31.493 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.86895.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.86895.log.gz 2024-09-12T14:19:31.493 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.51381.log 2024-09-12T14:19:31.494 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.102338.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.102338.log.gz 2024-09-12T14:19:31.494 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.60587.log 2024-09-12T14:19:31.494 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.51381.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.51381.log.gz 2024-09-12T14:19:31.494 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.93798.log 2024-09-12T14:19:31.495 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.60587.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.60587.log.gz 2024-09-12T14:19:31.495 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.119895.log 2024-09-12T14:19:31.496 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.93798.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.69234.log 2024-09-12T14:19:31.496 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.93798.log.gz 2024-09-12T14:19:31.497 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.119895.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.119895.log.gz 2024-09-12T14:19:31.497 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.37031.log 2024-09-12T14:19:31.498 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.69234.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.69234.log.gz 2024-09-12T14:19:31.498 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.151945.log 2024-09-12T14:19:31.498 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.37031.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.37031.log.gz 2024-09-12T14:19:31.499 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.144535.log 2024-09-12T14:19:31.500 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.151945.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.151945.log.gz 2024-09-12T14:19:31.500 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.136737.log 2024-09-12T14:19:31.500 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.144535.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.144535.log.gz 2024-09-12T14:19:31.500 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.67297.log 2024-09-12T14:19:31.501 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.136737.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.104058.log 2024-09-12T14:19:31.501 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.136737.log.gz 2024-09-12T14:19:31.502 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.67297.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.67297.log.gz 2024-09-12T14:19:31.502 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.83502.log 2024-09-12T14:19:31.503 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.104058.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.104058.log.gz 2024-09-12T14:19:31.503 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.120149.log 2024-09-12T14:19:31.504 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.83502.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.83502.log.gz 2024-09-12T14:19:31.504 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.44370.log 2024-09-12T14:19:31.505 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.120149.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.120149.log.gz 2024-09-12T14:19:31.505 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.84931.log 2024-09-12T14:19:31.506 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.44370.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.44370.log.gz 2024-09-12T14:19:31.506 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.82956.log 2024-09-12T14:19:31.507 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.84931.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.84931.log.gz 2024-09-12T14:19:31.507 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.136249.log 2024-09-12T14:19:31.508 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.82956.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.118872.log 2024-09-12T14:19:31.508 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.82956.log.gz 2024-09-12T14:19:31.509 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.136249.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.136249.log.gz 2024-09-12T14:19:31.509 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.72042.log 2024-09-12T14:19:31.509 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.118872.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.118872.log.gz 2024-09-12T14:19:31.510 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.104631.log 2024-09-12T14:19:31.511 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.72042.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.72042.log.gz 2024-09-12T14:19:31.511 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.73818.log 2024-09-12T14:19:31.512 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.104631.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.104631.log.gz 2024-09-12T14:19:31.512 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.88632.log 2024-09-12T14:19:31.513 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.73818.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.73818.log.gz 2024-09-12T14:19:31.513 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.141965.log 2024-09-12T14:19:31.513 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.88632.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.88632.log.gz 2024-09-12T14:19:31.513 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.119730.log 2024-09-12T14:19:31.514 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.141965.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.141965.log.gz 2024-09-12T14:19:31.514 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.72106.log 2024-09-12T14:19:31.515 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.119730.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.119730.log.gz 2024-09-12T14:19:31.515 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.68168.log 2024-09-12T14:19:31.516 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.72106.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.72106.log.gz 2024-09-12T14:19:31.516 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.64154.log 2024-09-12T14:19:31.517 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.68168.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.68168.log.gz 2024-09-12T14:19:31.517 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.123410.log 2024-09-12T14:19:31.518 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.64154.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.64154.log.gz 2024-09-12T14:19:31.518 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.91342.log 2024-09-12T14:19:31.519 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.123410.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.123410.log.gz 2024-09-12T14:19:31.519 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.51763.log 2024-09-12T14:19:31.519 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.91342.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.91342.log.gz 2024-09-12T14:19:31.520 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.153645.log 2024-09-12T14:19:31.520 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.51763.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.51763.log.gz 2024-09-12T14:19:31.520 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.43233.log 2024-09-12T14:19:31.521 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.153645.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.153645.log.gz 2024-09-12T14:19:31.521 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.36392.log 2024-09-12T14:19:31.522 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.43233.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.43233.log.gz 2024-09-12T14:19:31.522 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.132745.log 2024-09-12T14:19:31.523 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.36392.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.36392.log.gz 2024-09-12T14:19:31.523 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.82227.log 2024-09-12T14:19:31.524 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.132745.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.132745.log.gz 2024-09-12T14:19:31.524 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.141664.log 2024-09-12T14:19:31.525 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.82227.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.82227.log.gz 2024-09-12T14:19:31.525 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.109254.log 2024-09-12T14:19:31.526 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.141664.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.141664.log.gz 2024-09-12T14:19:31.526 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.81376.log 2024-09-12T14:19:31.526 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.109254.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.109254.log.gz 2024-09-12T14:19:31.527 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.150016.log 2024-09-12T14:19:31.527 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.81376.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.81376.log.gz 2024-09-12T14:19:31.528 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.88550.log 2024-09-12T14:19:31.528 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.150016.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.150016.log.gz 2024-09-12T14:19:31.528 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.53147.log 2024-09-12T14:19:31.529 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.88550.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.138631.log 2024-09-12T14:19:31.530 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.88550.log.gz 2024-09-12T14:19:31.530 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.53147.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.53147.log.gz 2024-09-12T14:19:31.530 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.97956.log 2024-09-12T14:19:31.531 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.138631.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.138631.log.gz 2024-09-12T14:19:31.531 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.152914.log 2024-09-12T14:19:31.531 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.97956.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.97956.log.gz 2024-09-12T14:19:31.532 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.120691.log 2024-09-12T14:19:31.532 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.152914.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.152914.log.gz 2024-09-12T14:19:31.532 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.100605.log 2024-09-12T14:19:31.533 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.120691.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.120691.log.gz 2024-09-12T14:19:31.533 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.113139.log 2024-09-12T14:19:31.534 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.100605.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.100605.log.gz 2024-09-12T14:19:31.534 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.64322.log 2024-09-12T14:19:31.535 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.113139.log: 0.0%gzip -- replaced with /var/log/ceph/cluster1-client.mirror.113139.log.gz 2024-09-12T14:19:31.535 INFO:teuthology.orchestra.run.smithi130.stderr: -5 --verbose -- /var/log/ceph/cluster1-client.admin.105383.log 2024-09-12T14:19:31.535 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.64322.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.64322.log.gz 2024-09-12T14:19:31.535 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.112301.log 2024-09-12T14:19:31.536 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.105383.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.105383.log.gz 2024-09-12T14:19:31.536 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.33847.log 2024-09-12T14:19:31.537 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.112301.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.112301.log.gz 2024-09-12T14:19:31.537 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.56368.log 2024-09-12T14:19:31.538 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.33847.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.33847.log.gz 2024-09-12T14:19:31.538 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.99719.log 2024-09-12T14:19:31.539 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.56368.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.56368.log.gz 2024-09-12T14:19:31.539 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.60647.log 2024-09-12T14:19:31.540 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.99719.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.78338.log 2024-09-12T14:19:31.540 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.99719.log.gz 2024-09-12T14:19:31.540 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.60647.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.60647.log.gz 2024-09-12T14:19:31.541 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.106181.log 2024-09-12T14:19:31.541 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.78338.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.78338.log.gz 2024-09-12T14:19:31.541 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.96170.log 2024-09-12T14:19:31.542 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.106181.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.109234.log 2024-09-12T14:19:31.543 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.106181.log.gz 2024-09-12T14:19:31.543 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.96170.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.96170.log.gz 2024-09-12T14:19:31.543 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.40478.log 2024-09-12T14:19:31.544 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.109234.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.109234.log.gz 2024-09-12T14:19:31.544 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.152282.log 2024-09-12T14:19:31.544 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.40478.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.40478.log.gz 2024-09-12T14:19:31.544 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.104628.log 2024-09-12T14:19:31.545 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.152282.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.152282.log.gz 2024-09-12T14:19:31.545 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.132663.log 2024-09-12T14:19:31.546 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.104628.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.104628.log.gz 2024-09-12T14:19:31.546 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.129865.log 2024-09-12T14:19:31.546 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.132663.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.132663.log.gz 2024-09-12T14:19:31.546 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.110072.log 2024-09-12T14:19:31.547 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.129865.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.129865.log.gz 2024-09-12T14:19:31.548 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.77073.log 2024-09-12T14:19:31.548 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.110072.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.110072.log.gz 2024-09-12T14:19:31.549 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.44360.log 2024-09-12T14:19:31.549 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.77073.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.77073.log.gz 2024-09-12T14:19:31.549 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.119687.log 2024-09-12T14:19:31.550 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.44360.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.44360.log.gz 2024-09-12T14:19:31.550 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.87965.log 2024-09-12T14:19:31.550 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.119687.log: gzip 0.0% -5 -- replaced with /var/log/ceph/cluster1-client.mirror.119687.log.gz --verbose 2024-09-12T14:19:31.551 INFO:teuthology.orchestra.run.smithi130.stderr: -- /var/log/ceph/cluster1-client.mirror.144828.log 2024-09-12T14:19:31.551 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.87965.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.87965.log.gz 2024-09-12T14:19:31.551 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.95391.log 2024-09-12T14:19:31.552 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.144828.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.144828.log.gz 2024-09-12T14:19:31.552 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.136929.log 2024-09-12T14:19:31.553 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.95391.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.96485.log 2024-09-12T14:19:31.553 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.95391.log.gz 2024-09-12T14:19:31.554 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.136929.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.136929.log.gz 2024-09-12T14:19:31.554 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.59973.log 2024-09-12T14:19:31.555 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.96485.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.96485.log.gz 2024-09-12T14:19:31.555 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.143296.log 2024-09-12T14:19:31.556 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.59973.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.59973.log.gz 2024-09-12T14:19:31.556 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.112133.log 2024-09-12T14:19:31.557 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.143296.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.59358.log 2024-09-12T14:19:31.557 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.143296.log.gz 2024-09-12T14:19:31.558 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.112133.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.136152.log 2024-09-12T14:19:31.558 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.112133.log.gz 2024-09-12T14:19:31.559 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.59358.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.59358.log.gz 2024-09-12T14:19:31.559 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.130187.log 2024-09-12T14:19:31.559 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.136152.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.136152.log.gz 2024-09-12T14:19:31.559 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.137096.log 2024-09-12T14:19:31.560 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.130187.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.130187.log.gz 2024-09-12T14:19:31.560 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.38999.log 2024-09-12T14:19:31.561 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.137096.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.137096.log.gz 2024-09-12T14:19:31.561 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.65036.log 2024-09-12T14:19:31.562 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.38999.log: gzip 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.38999.log.gz -5 2024-09-12T14:19:31.562 INFO:teuthology.orchestra.run.smithi130.stderr: --verbose -- /var/log/ceph/cluster1-client.mirror.71128.log 2024-09-12T14:19:31.562 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.65036.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.65036.log.gz 2024-09-12T14:19:31.562 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.121453.log 2024-09-12T14:19:31.563 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.71128.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.71128.log.gz 2024-09-12T14:19:31.563 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.107258.log 2024-09-12T14:19:31.564 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.121453.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.121453.log.gz 2024-09-12T14:19:31.564 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.92268.log 2024-09-12T14:19:31.565 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.107258.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.107258.log.gz 2024-09-12T14:19:31.565 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.107163.log 2024-09-12T14:19:31.566 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.92268.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.92268.log.gz 2024-09-12T14:19:31.566 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.94388.log 2024-09-12T14:19:31.566 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.107163.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.107163.log.gz 2024-09-12T14:19:31.567 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.65332.log 2024-09-12T14:19:31.567 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.94388.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.135415.log 2024-09-12T14:19:31.567 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.94388.log.gz 2024-09-12T14:19:31.568 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.65332.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.65332.log.gz 2024-09-12T14:19:31.568 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.97373.log 2024-09-12T14:19:31.569 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.135415.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.135415.log.gz 2024-09-12T14:19:31.569 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.72393.log 2024-09-12T14:19:31.570 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.97373.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.97373.log.gz 2024-09-12T14:19:31.570 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.109148.log 2024-09-12T14:19:31.571 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.72393.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.72393.log.gz 2024-09-12T14:19:31.571 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.132641.log 2024-09-12T14:19:31.571 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.109148.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.109148.log.gz 2024-09-12T14:19:31.571 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.56618.log 2024-09-12T14:19:31.572 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.132641.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.132641.log.gz 2024-09-12T14:19:31.572 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.124379.log 2024-09-12T14:19:31.573 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.56618.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.148876.log 2024-09-12T14:19:31.573 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.56618.log.gz 2024-09-12T14:19:31.574 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.124379.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.124379.log.gz 2024-09-12T14:19:31.574 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.122227.log 2024-09-12T14:19:31.575 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.148876.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.145397.log 2024-09-12T14:19:31.575 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.148876.log.gz 2024-09-12T14:19:31.576 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.122227.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.122227.log.gz 2024-09-12T14:19:31.576 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.55967.log 2024-09-12T14:19:31.576 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.145397.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.145397.log.gz 2024-09-12T14:19:31.576 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.50711.log 2024-09-12T14:19:31.577 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.55967.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.55967.log.gz 2024-09-12T14:19:31.577 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.136988.log 2024-09-12T14:19:31.578 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.50711.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.50711.log.gz 2024-09-12T14:19:31.578 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.67907.log 2024-09-12T14:19:31.579 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.136988.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.136988.log.gz 2024-09-12T14:19:31.579 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.98233.log 2024-09-12T14:19:31.580 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.67907.log: 0.0%gzip -- replaced with /var/log/ceph/cluster1-client.admin.67907.log.gz 2024-09-12T14:19:31.580 INFO:teuthology.orchestra.run.smithi130.stderr: -5 --verbose -- /var/log/ceph/cluster1-client.mirror.96171.log 2024-09-12T14:19:31.581 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.98233.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.98233.log.gz 2024-09-12T14:19:31.581 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.82040.log 2024-09-12T14:19:31.582 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.96171.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.96171.log.gz 2024-09-12T14:19:31.582 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.35828.log 2024-09-12T14:19:31.582 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.82040.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.82040.log.gz 2024-09-12T14:19:31.583 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.67717.log 2024-09-12T14:19:31.583 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.35828.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.35828.log.gz 2024-09-12T14:19:31.584 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.65584.log 2024-09-12T14:19:31.584 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.67717.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.67717.log.gz 2024-09-12T14:19:31.585 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.48238.log 2024-09-12T14:19:31.585 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.65584.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.65584.log.gz 2024-09-12T14:19:31.585 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.118998.log 2024-09-12T14:19:31.586 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.48238.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.48238.log.gz 2024-09-12T14:19:31.586 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.80524.log 2024-09-12T14:19:31.587 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.118998.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.118998.log.gz 2024-09-12T14:19:31.587 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.130659.log 2024-09-12T14:19:31.588 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.80524.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.109149.log 2024-09-12T14:19:31.588 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.80524.log.gz 2024-09-12T14:19:31.589 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.130659.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.130659.log.gz 2024-09-12T14:19:31.589 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.56676.log 2024-09-12T14:19:31.589 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.109149.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.109149.log.gz 2024-09-12T14:19:31.590 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.145542.log 2024-09-12T14:19:31.590 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.56676.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.56676.log.gz 2024-09-12T14:19:31.590 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.128817.log 2024-09-12T14:19:31.591 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.145542.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.145542.log.gz 2024-09-12T14:19:31.591 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.37005.log 2024-09-12T14:19:31.592 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.128817.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.128817.log.gz 2024-09-12T14:19:31.592 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.62979.log 2024-09-12T14:19:31.593 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.37005.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.37005.log.gz 2024-09-12T14:19:31.593 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.152681.log 2024-09-12T14:19:31.594 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.62979.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.62979.log.gz 2024-09-12T14:19:31.594 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.140008.log 2024-09-12T14:19:31.595 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.152681.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.152681.log.gz 2024-09-12T14:19:31.595 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.132996.log 2024-09-12T14:19:31.595 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.140008.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.140008.log.gz 2024-09-12T14:19:31.596 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.97768.log 2024-09-12T14:19:31.596 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.132996.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.71670.log 2024-09-12T14:19:31.596 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.132996.log.gz 2024-09-12T14:19:31.597 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.86034.log 2024-09-12T14:19:31.598 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.97768.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.97768.log.gz 2024-09-12T14:19:31.598 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.71670.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.71670.log.gz 2024-09-12T14:19:31.598 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.92266.log 2024-09-12T14:19:31.599 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.86034.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.86034.log.gz 2024-09-12T14:19:31.599 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.109845.log 2024-09-12T14:19:31.600 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.92266.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.92266.log.gz 2024-09-12T14:19:31.600 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.111948.log 2024-09-12T14:19:31.601 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.109845.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.109845.log.gz 2024-09-12T14:19:31.601 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.144978.log 2024-09-12T14:19:31.601 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.111948.log: gzip 0.0% -5 --verbose -- replaced with /var/log/ceph/cluster1-client.mirror.111948.log.gz -- 2024-09-12T14:19:31.601 INFO:teuthology.orchestra.run.smithi130.stderr: /var/log/ceph/cluster2-client.mirror.35177.log 2024-09-12T14:19:31.602 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.144978.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.144978.log.gz 2024-09-12T14:19:31.602 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.126587.log 2024-09-12T14:19:31.603 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.35177.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.35177.log.gz 2024-09-12T14:19:31.603 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.151507.log 2024-09-12T14:19:31.604 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.126587.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.126587.log.gz 2024-09-12T14:19:31.604 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.131165.log 2024-09-12T14:19:31.605 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.151507.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.151507.log.gz 2024-09-12T14:19:31.605 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.130228.log 2024-09-12T14:19:31.606 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.131165.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.131165.log.gz 2024-09-12T14:19:31.607 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.102790.log 2024-09-12T14:19:31.607 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.130228.log: gzip 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.130228.log.gz -5 2024-09-12T14:19:31.607 INFO:teuthology.orchestra.run.smithi130.stderr: --verbose -- /var/log/ceph/cluster1-client.mirror.88321.log 2024-09-12T14:19:31.608 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.102790.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.102790.log.gz 2024-09-12T14:19:31.608 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.139234.log 2024-09-12T14:19:31.610 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.88321.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.88321.log.gz 2024-09-12T14:19:31.610 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.55548.log 2024-09-12T14:19:31.610 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.139234.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.139234.log.gz 2024-09-12T14:19:31.610 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.103570.log 2024-09-12T14:19:31.611 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.55548.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.55548.log.gzgzip 2024-09-12T14:19:31.612 INFO:teuthology.orchestra.run.smithi130.stderr: -5 --verbose -- /var/log/ceph/cluster1-client.mirror.61737.log 2024-09-12T14:19:31.612 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.103570.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.103570.log.gz 2024-09-12T14:19:31.612 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.64050.log 2024-09-12T14:19:31.616 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.61737.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.61737.log.gz 2024-09-12T14:19:31.616 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.111669.log 2024-09-12T14:19:31.616 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.64050.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.64050.log.gz 2024-09-12T14:19:31.616 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.93362.log 2024-09-12T14:19:31.616 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.111669.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.111669.log.gz 2024-09-12T14:19:31.616 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.111352.log 2024-09-12T14:19:31.616 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.93362.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.93362.log.gz 2024-09-12T14:19:31.617 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.65898.log 2024-09-12T14:19:31.617 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.111352.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.111352.log.gz 2024-09-12T14:19:31.617 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.74297.log 2024-09-12T14:19:31.617 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.65898.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.65898.log.gz 2024-09-12T14:19:31.617 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.73290.log 2024-09-12T14:19:31.617 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.74297.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.74297.log.gz 2024-09-12T14:19:31.617 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.80718.log 2024-09-12T14:19:31.619 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.73290.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.73290.log.gz 2024-09-12T14:19:31.619 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.72957.log 2024-09-12T14:19:31.619 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.80718.log: gzip 0.0% -5 -- replaced with /var/log/ceph/cluster1-client.admin.80718.log.gz --verbose 2024-09-12T14:19:31.619 INFO:teuthology.orchestra.run.smithi130.stderr: -- /var/log/ceph/cluster1-client.admin.47117.log 2024-09-12T14:19:31.619 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.72957.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.72957.log.gz 2024-09-12T14:19:31.619 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.99300.log 2024-09-12T14:19:31.619 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.47117.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.47117.log.gz 2024-09-12T14:19:31.619 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.96067.log 2024-09-12T14:19:31.621 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.99300.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.99300.log.gz 2024-09-12T14:19:31.621 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.147343.log 2024-09-12T14:19:31.621 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.96067.log: 0.0%gzip -- replaced with /var/log/ceph/cluster1-client.mirror.96067.log.gz 2024-09-12T14:19:31.621 INFO:teuthology.orchestra.run.smithi130.stderr: -5 --verbose -- /var/log/ceph/cluster1-client.mirror.39969.log 2024-09-12T14:19:31.621 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.147343.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.147343.log.gz 2024-09-12T14:19:31.621 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.94087.log 2024-09-12T14:19:31.623 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.39969.log: gzip -5 0.0% --verbose -- -- replaced with /var/log/ceph/cluster1-client.mirror.39969.log.gz /var/log/ceph/cluster1-client.mirror.100982.log 2024-09-12T14:19:31.623 INFO:teuthology.orchestra.run.smithi130.stderr: 2024-09-12T14:19:31.623 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.94087.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.94087.log.gz 2024-09-12T14:19:31.623 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.110222.log 2024-09-12T14:19:31.624 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.100982.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.100982.log.gz 2024-09-12T14:19:31.624 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.53273.log 2024-09-12T14:19:31.625 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.110222.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.110222.log.gz 2024-09-12T14:19:31.625 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.60127.log 2024-09-12T14:19:31.625 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.53273.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.53273.log.gzgzip 2024-09-12T14:19:31.625 INFO:teuthology.orchestra.run.smithi130.stderr: -5 --verbose -- /var/log/ceph/cluster1-client.mirror.110308.log 2024-09-12T14:19:31.627 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.60127.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.60127.log.gz 2024-09-12T14:19:31.627 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.49725.log 2024-09-12T14:19:31.627 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.110308.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.110308.log.gz 2024-09-12T14:19:31.627 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.34062.log 2024-09-12T14:19:31.628 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.49725.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.49725.log.gz 2024-09-12T14:19:31.628 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.98772.log 2024-09-12T14:19:31.629 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.34062.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.34062.log.gz 2024-09-12T14:19:31.629 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.135207.log 2024-09-12T14:19:31.630 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.98772.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.98772.log.gz 2024-09-12T14:19:31.630 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.123534.log 2024-09-12T14:19:31.630 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.135207.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.135207.log.gz 2024-09-12T14:19:31.630 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.69373.log 2024-09-12T14:19:31.631 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.123534.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.123534.log.gz 2024-09-12T14:19:31.631 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.128593.log 2024-09-12T14:19:31.632 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.69373.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.69373.log.gz 2024-09-12T14:19:31.632 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.37030.log 2024-09-12T14:19:31.632 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.128593.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.128593.log.gz 2024-09-12T14:19:31.632 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.62808.log 2024-09-12T14:19:31.633 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.37030.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.37030.log.gz 2024-09-12T14:19:31.633 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.119728.log 2024-09-12T14:19:31.634 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.62808.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.62808.log.gz 2024-09-12T14:19:31.634 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.149347.log 2024-09-12T14:19:31.635 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.119728.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.119728.log.gz 2024-09-12T14:19:31.635 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.106668.log 2024-09-12T14:19:31.635 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.149347.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.149347.log.gz 2024-09-12T14:19:31.635 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.118964.log 2024-09-12T14:19:31.636 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.106668.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.106668.log.gz 2024-09-12T14:19:31.636 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.82688.log 2024-09-12T14:19:31.637 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.118964.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.99214.log 2024-09-12T14:19:31.637 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.118964.log.gz 2024-09-12T14:19:31.638 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.82688.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.82688.log.gz 2024-09-12T14:19:31.638 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.41898.log 2024-09-12T14:19:31.638 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.99214.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.99214.log.gz 2024-09-12T14:19:31.639 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.146969.log 2024-09-12T14:19:31.639 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.41898.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.41898.log.gz 2024-09-12T14:19:31.639 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.72789.log 2024-09-12T14:19:31.640 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.146969.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.74362.log 2024-09-12T14:19:31.640 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.146969.log.gz 2024-09-12T14:19:31.641 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.72789.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.72789.log.gz 2024-09-12T14:19:31.641 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.57099.log 2024-09-12T14:19:31.642 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.74362.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.74362.log.gz 2024-09-12T14:19:31.642 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.93590.log 2024-09-12T14:19:31.642 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.57099.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.57099.log.gz 2024-09-12T14:19:31.643 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.81517.log 2024-09-12T14:19:31.643 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.93590.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.93590.log.gz 2024-09-12T14:19:31.644 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.89666.log 2024-09-12T14:19:31.644 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.81517.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.81517.log.gz 2024-09-12T14:19:31.644 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.46032.log 2024-09-12T14:19:31.645 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.89666.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.89666.log.gz 2024-09-12T14:19:31.645 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.129108.log 2024-09-12T14:19:31.645 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.46032.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.46032.log.gz 2024-09-12T14:19:31.646 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.63441.log 2024-09-12T14:19:31.646 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.129108.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.129108.log.gz 2024-09-12T14:19:31.646 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.42193.log 2024-09-12T14:19:31.647 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.63441.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.143919.log 2024-09-12T14:19:31.648 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.63441.log.gz 2024-09-12T14:19:31.648 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.42193.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.42193.log.gz 2024-09-12T14:19:31.648 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.85925.log 2024-09-12T14:19:31.648 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.143919.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.143919.log.gz 2024-09-12T14:19:31.649 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.153826.log 2024-09-12T14:19:31.649 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.85925.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.44339.log 2024-09-12T14:19:31.650 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.85925.log.gz 2024-09-12T14:19:31.650 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.153826.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.153826.log.gz 2024-09-12T14:19:31.651 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.35032.log 2024-09-12T14:19:31.651 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.44339.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.44339.log.gz 2024-09-12T14:19:31.651 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.96694.log 2024-09-12T14:19:31.652 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.35032.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.35032.log.gz 2024-09-12T14:19:31.652 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.42052.log 2024-09-12T14:19:31.653 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.96694.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.96694.log.gz 2024-09-12T14:19:31.653 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.65941.log 2024-09-12T14:19:31.653 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.42052.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.42052.log.gz 2024-09-12T14:19:31.654 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.73543.log 2024-09-12T14:19:31.655 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.65941.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.65941.log.gz 2024-09-12T14:19:31.655 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.105497.log 2024-09-12T14:19:31.655 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.73543.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.73543.log.gz 2024-09-12T14:19:31.655 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.39529.log 2024-09-12T14:19:31.656 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.105497.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.105497.log.gz 2024-09-12T14:19:31.656 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.105570.log 2024-09-12T14:19:31.657 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.39529.log: gzip 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.39529.log.gz -5 2024-09-12T14:19:31.657 INFO:teuthology.orchestra.run.smithi130.stderr: --verbose -- /var/log/ceph/cluster1-client.mirror.73586.log 2024-09-12T14:19:31.657 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.105570.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.105570.log.gz 2024-09-12T14:19:31.657 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.65401.log 2024-09-12T14:19:31.658 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.73586.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.73586.log.gzgzip 2024-09-12T14:19:31.658 INFO:teuthology.orchestra.run.smithi130.stderr: -5 --verbose -- /var/log/ceph/cluster1-client.mirror.122792.log 2024-09-12T14:19:31.659 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.65401.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.65401.log.gz 2024-09-12T14:19:31.659 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.42300.log 2024-09-12T14:19:31.660 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.122792.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.108646.log 2024-09-12T14:19:31.660 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.122792.log.gz 2024-09-12T14:19:31.661 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.42300.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.42300.log.gz 2024-09-12T14:19:31.661 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.152616.log 2024-09-12T14:19:31.661 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.108646.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.108646.log.gz 2024-09-12T14:19:31.662 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.83809.log 2024-09-12T14:19:31.663 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.152616.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.152616.log.gz 2024-09-12T14:19:31.663 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.148992.log 2024-09-12T14:19:31.663 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.83809.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.83809.log.gz 2024-09-12T14:19:31.663 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.113815.log 2024-09-12T14:19:31.664 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.148992.log: gzip 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.148992.log.gz -5 2024-09-12T14:19:31.664 INFO:teuthology.orchestra.run.smithi130.stderr: --verbose -- /var/log/ceph/cluster1-client.mirror.75536.log 2024-09-12T14:19:31.665 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.113815.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.113815.log.gz 2024-09-12T14:19:31.665 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.56245.log 2024-09-12T14:19:31.666 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.75536.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.75536.log.gz 2024-09-12T14:19:31.666 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.42658.log 2024-09-12T14:19:31.666 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.56245.log: 0.0%gzip -- replaced with /var/log/ceph/cluster2-client.admin.56245.log.gz 2024-09-12T14:19:31.666 INFO:teuthology.orchestra.run.smithi130.stderr: -5 --verbose -- /var/log/ceph/cluster1-client.mirror.134366.log 2024-09-12T14:19:31.667 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.42658.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.42658.log.gz 2024-09-12T14:19:31.667 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.153303.log 2024-09-12T14:19:31.668 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.134366.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.134366.log.gz 2024-09-12T14:19:31.668 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.136843.log 2024-09-12T14:19:31.669 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.153303.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.153303.log.gz 2024-09-12T14:19:31.670 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.50898.log 2024-09-12T14:19:31.670 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.136843.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.136843.log.gz 2024-09-12T14:19:31.670 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.148150.log 2024-09-12T14:19:31.671 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.50898.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.50898.log.gz 2024-09-12T14:19:31.671 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.99658.log 2024-09-12T14:19:31.671 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.148150.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.148150.log.gz 2024-09-12T14:19:31.672 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.112805.log 2024-09-12T14:19:31.673 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.99658.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.99658.log.gz 2024-09-12T14:19:31.673 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.73375.log 2024-09-12T14:19:31.673 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.112805.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.112805.log.gz 2024-09-12T14:19:31.673 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.41544.log 2024-09-12T14:19:31.675 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.73375.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.73375.log.gz 2024-09-12T14:19:31.675 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.132795.log 2024-09-12T14:19:31.675 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.41544.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.41544.log.gz 2024-09-12T14:19:31.675 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.43675.log 2024-09-12T14:19:31.676 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.132795.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.132795.log.gz 2024-09-12T14:19:31.676 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.85622.log 2024-09-12T14:19:31.676 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.43675.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.43675.log.gz 2024-09-12T14:19:31.676 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.144135.log 2024-09-12T14:19:31.678 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.85622.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.85622.log.gz 2024-09-12T14:19:31.678 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.128360.log 2024-09-12T14:19:31.678 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.144135.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.144135.log.gz 2024-09-12T14:19:31.678 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.121698.log 2024-09-12T14:19:31.680 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.128360.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.145544.log 2024-09-12T14:19:31.680 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.128360.log.gz 2024-09-12T14:19:31.680 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.121698.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.121698.log.gz 2024-09-12T14:19:31.680 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.39045.log 2024-09-12T14:19:31.680 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.145544.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.145544.log.gz 2024-09-12T14:19:31.681 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.112134.log 2024-09-12T14:19:31.681 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.39045.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.39045.log.gz 2024-09-12T14:19:31.681 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.104269.log 2024-09-12T14:19:31.682 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.112134.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.112134.log.gz 2024-09-12T14:19:31.682 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.38537.log 2024-09-12T14:19:31.683 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.104269.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.104269.log.gz 2024-09-12T14:19:31.683 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.59321.log 2024-09-12T14:19:31.684 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.38537.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.134260.log 2024-09-12T14:19:31.685 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.38537.log.gz 2024-09-12T14:19:31.685 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.59321.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.59321.log.gz 2024-09-12T14:19:31.685 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.108812.log 2024-09-12T14:19:31.685 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.134260.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.134260.log.gz 2024-09-12T14:19:31.686 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.90569.log 2024-09-12T14:19:31.686 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.108812.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.69146.log 2024-09-12T14:19:31.686 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.108812.log.gz 2024-09-12T14:19:31.687 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.90569.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.90569.log.gz 2024-09-12T14:19:31.687 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.72998.log 2024-09-12T14:19:31.688 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.69146.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.69146.log.gz 2024-09-12T14:19:31.688 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.105104.log 2024-09-12T14:19:31.688 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.72998.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.72998.log.gz 2024-09-12T14:19:31.689 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.52978.log 2024-09-12T14:19:31.690 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.105104.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.37888.log 2024-09-12T14:19:31.690 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.105104.log.gz 2024-09-12T14:19:31.691 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.52978.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.52978.log.gz 2024-09-12T14:19:31.691 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.148210.log 2024-09-12T14:19:31.691 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.37888.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.37888.log.gz 2024-09-12T14:19:31.692 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.54788.log 2024-09-12T14:19:31.693 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.148210.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.100647.log 2024-09-12T14:19:31.693 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.148210.log.gz 2024-09-12T14:19:31.694 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.54788.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.54788.log.gz 2024-09-12T14:19:31.694 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.79172.log 2024-09-12T14:19:31.694 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.100647.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.100647.log.gz 2024-09-12T14:19:31.694 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.134826.log 2024-09-12T14:19:31.695 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.79172.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.79172.log.gz 2024-09-12T14:19:31.695 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.118038.log 2024-09-12T14:19:31.696 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.134826.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.134826.log.gz 2024-09-12T14:19:31.696 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.86997.log 2024-09-12T14:19:31.696 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.118038.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.118038.log.gz 2024-09-12T14:19:31.697 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.134995.log 2024-09-12T14:19:31.698 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.86997.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.121740.log 2024-09-12T14:19:31.698 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.86997.log.gz 2024-09-12T14:19:31.698 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.134995.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.134995.log.gz 2024-09-12T14:19:31.699 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.52222.log 2024-09-12T14:19:31.699 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.121740.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.121740.log.gz 2024-09-12T14:19:31.699 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.53859.log 2024-09-12T14:19:31.700 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.52222.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.52222.log.gz 2024-09-12T14:19:31.700 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.109064.log 2024-09-12T14:19:31.701 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.53859.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.53859.log.gz 2024-09-12T14:19:31.701 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.152198.log 2024-09-12T14:19:31.702 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.109064.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.109064.log.gz 2024-09-12T14:19:31.702 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.94865.log 2024-09-12T14:19:31.703 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.152198.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.152198.log.gz 2024-09-12T14:19:31.703 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.121763.log 2024-09-12T14:19:31.704 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.94865.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.94865.log.gz 2024-09-12T14:19:31.704 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.55011.log 2024-09-12T14:19:31.705 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.121763.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.121763.log.gz 2024-09-12T14:19:31.705 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.71448.log 2024-09-12T14:19:31.706 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.55011.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.55011.log.gz 2024-09-12T14:19:31.706 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.109718.log 2024-09-12T14:19:31.706 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.71448.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.71448.log.gz 2024-09-12T14:19:31.707 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.119115.log 2024-09-12T14:19:31.707 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.109718.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.48968.log 2024-09-12T14:19:31.707 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.109718.log.gz 2024-09-12T14:19:31.708 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.119115.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.119115.log.gz 2024-09-12T14:19:31.708 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.153869.log 2024-09-12T14:19:31.709 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.48968.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.48968.log.gz 2024-09-12T14:19:31.709 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.109317.log 2024-09-12T14:19:31.710 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.153869.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.153869.log.gz 2024-09-12T14:19:31.710 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.64190.log 2024-09-12T14:19:31.711 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.109317.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.109317.log.gz 2024-09-12T14:19:31.711 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.107475.log 2024-09-12T14:19:31.712 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.64190.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.64190.log.gz 2024-09-12T14:19:31.712 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.52433.log 2024-09-12T14:19:31.713 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.107475.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.107475.log.gz 2024-09-12T14:19:31.713 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.116251.log 2024-09-12T14:19:31.713 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.52433.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.52433.log.gz 2024-09-12T14:19:31.714 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.40540.log 2024-09-12T14:19:31.715 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.116251.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.116251.log.gz 2024-09-12T14:19:31.715 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.51929.log 2024-09-12T14:19:31.715 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.40540.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.40540.log.gz 2024-09-12T14:19:31.716 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.54156.log 2024-09-12T14:19:31.716 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.51929.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.51929.log.gz 2024-09-12T14:19:31.717 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.82382.log 2024-09-12T14:19:31.717 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.54156.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.54156.log.gz 2024-09-12T14:19:31.718 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.65856.log 2024-09-12T14:19:31.718 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.82382.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.82382.log.gz 2024-09-12T14:19:31.718 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.142042.log 2024-09-12T14:19:31.719 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.65856.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.65856.log.gz 2024-09-12T14:19:31.719 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.122266.log 2024-09-12T14:19:31.720 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.142042.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.142042.log.gz 2024-09-12T14:19:31.720 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.112956.log 2024-09-12T14:19:31.721 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.122266.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.122266.log.gz 2024-09-12T14:19:31.722 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.145188.log 2024-09-12T14:19:31.722 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.112956.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.112956.log.gz 2024-09-12T14:19:31.722 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.93006.log 2024-09-12T14:19:31.723 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.145188.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.145188.log.gz 2024-09-12T14:19:31.723 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.70579.log 2024-09-12T14:19:31.723 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.93006.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.93006.log.gz 2024-09-12T14:19:31.723 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.36683.log 2024-09-12T14:19:31.724 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.70579.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.70579.log.gz 2024-09-12T14:19:31.724 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.98817.log 2024-09-12T14:19:31.725 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.36683.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.36683.log.gz 2024-09-12T14:19:31.725 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.57292.log 2024-09-12T14:19:31.726 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.98817.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.98817.log.gz 2024-09-12T14:19:31.726 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.109656.log 2024-09-12T14:19:31.727 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.57292.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.57292.log.gz 2024-09-12T14:19:31.727 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.123787.log 2024-09-12T14:19:31.727 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.109656.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.109656.log.gz 2024-09-12T14:19:31.728 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.132285.log 2024-09-12T14:19:31.729 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.123787.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.123787.log.gz 2024-09-12T14:19:31.729 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.100812.log 2024-09-12T14:19:31.729 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.132285.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.132285.log.gz 2024-09-12T14:19:31.730 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.142770.log 2024-09-12T14:19:31.730 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.100812.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.100812.log.gz 2024-09-12T14:19:31.730 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.93277.log 2024-09-12T14:19:31.731 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.142770.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.142770.log.gz 2024-09-12T14:19:31.731 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.139233.log 2024-09-12T14:19:31.732 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.93277.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.93277.log.gz 2024-09-12T14:19:31.732 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.38793.log 2024-09-12T14:19:31.733 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.139233.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.139233.log.gz 2024-09-12T14:19:31.733 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.81604.log 2024-09-12T14:19:31.734 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.38793.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.38793.log.gz 2024-09-12T14:19:31.734 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.150892.log 2024-09-12T14:19:31.735 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.81604.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.81604.log.gz 2024-09-12T14:19:31.735 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.123619.log 2024-09-12T14:19:31.736 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.150892.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.150892.log.gz 2024-09-12T14:19:31.737 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.57552.log 2024-09-12T14:19:31.737 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.123619.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.123619.log.gz 2024-09-12T14:19:31.737 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.120922.log 2024-09-12T14:19:31.738 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.57552.log: gzip -5 0.0% --verbose -- -- replaced with /var/log/ceph/cluster2-client.mirror.57552.log.gz /var/log/ceph/cluster1-client.admin.138091.log 2024-09-12T14:19:31.738 INFO:teuthology.orchestra.run.smithi130.stderr: 2024-09-12T14:19:31.738 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.120922.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.120922.log.gz 2024-09-12T14:19:31.738 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.49662.log 2024-09-12T14:19:31.740 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.138091.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.138091.log.gz 2024-09-12T14:19:31.740 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.96065.log 2024-09-12T14:19:31.740 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.49662.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.49662.log.gz 2024-09-12T14:19:31.740 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.46448.log 2024-09-12T14:19:31.742 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.96065.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.119306.log 2024-09-12T14:19:31.742 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.96065.log.gz 2024-09-12T14:19:31.742 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.46448.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.46448.log.gz 2024-09-12T14:19:31.742 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.46858.log 2024-09-12T14:19:31.742 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.119306.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.119306.log.gz 2024-09-12T14:19:31.742 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.97917.log 2024-09-12T14:19:31.743 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.46858.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.46858.log.gz 2024-09-12T14:19:31.744 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.148338.log 2024-09-12T14:19:31.744 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.97917.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.97917.log.gz 2024-09-12T14:19:31.744 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.121490.log 2024-09-12T14:19:31.746 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.148338.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.148338.log.gz 2024-09-12T14:19:31.746 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.69869.log 2024-09-12T14:19:31.746 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.121490.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.121490.log.gz 2024-09-12T14:19:31.746 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.39400.log 2024-09-12T14:19:31.746 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.69869.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.105022.log 2024-09-12T14:19:31.747 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.69869.log.gz 2024-09-12T14:19:31.748 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.39400.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.39400.log.gz 2024-09-12T14:19:31.748 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.149616.log 2024-09-12T14:19:31.748 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.105022.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.105022.log.gz 2024-09-12T14:19:31.748 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.109865.log 2024-09-12T14:19:31.749 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.149616.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.149616.log.gz 2024-09-12T14:19:31.749 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.49176.log 2024-09-12T14:19:31.750 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.109865.log: gzip 0.0% -5 -- replaced with /var/log/ceph/cluster1-client.mirror.109865.log.gz --verbose 2024-09-12T14:19:31.750 INFO:teuthology.orchestra.run.smithi130.stderr: -- /var/log/ceph/cluster1-client.admin.102634.log 2024-09-12T14:19:31.751 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.49176.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.49176.log.gz 2024-09-12T14:19:31.751 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.102900.log 2024-09-12T14:19:31.751 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.102634.log: gzip 0.0% -5 -- replaced with /var/log/ceph/cluster1-client.admin.102634.log.gz --verbose 2024-09-12T14:19:31.751 INFO:teuthology.orchestra.run.smithi130.stderr: -- /var/log/ceph/cluster1-client.mirror.62054.log 2024-09-12T14:19:31.752 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.102900.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.102900.log.gz 2024-09-12T14:19:31.752 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.51280.log 2024-09-12T14:19:31.753 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.62054.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.127883.log 2024-09-12T14:19:31.753 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.62054.log.gz 2024-09-12T14:19:31.753 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.51280.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.24642.log 2024-09-12T14:19:31.753 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.51280.log.gz 2024-09-12T14:19:31.754 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.127883.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.127883.log.gz 2024-09-12T14:19:31.755 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.77094.log 2024-09-12T14:19:31.755 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/ceph.tmp-client.admin.24642.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.24642.log.gz 2024-09-12T14:19:31.755 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.145922.log 2024-09-12T14:19:31.756 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.77094.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.77094.log.gz 2024-09-12T14:19:31.756 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.123788.log 2024-09-12T14:19:31.756 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.145922.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.145922.log.gz 2024-09-12T14:19:31.757 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.148388.log 2024-09-12T14:19:31.757 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.123788.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.123788.log.gz 2024-09-12T14:19:31.758 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.68943.log 2024-09-12T14:19:31.758 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.148388.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.148388.log.gz 2024-09-12T14:19:31.759 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.143185.log 2024-09-12T14:19:31.759 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.68943.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.68943.log.gz 2024-09-12T14:19:31.760 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.52601.log 2024-09-12T14:19:31.760 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.143185.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.143185.log.gz 2024-09-12T14:19:31.760 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.143076.log 2024-09-12T14:19:31.761 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.52601.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.67495.log 2024-09-12T14:19:31.761 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.52601.log.gz 2024-09-12T14:19:31.762 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.143076.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.143076.log.gz 2024-09-12T14:19:31.762 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.140599.log 2024-09-12T14:19:31.763 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.67495.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.67495.log.gz 2024-09-12T14:19:31.763 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.100016.log 2024-09-12T14:19:31.764 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.140599.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.140599.log.gz 2024-09-12T14:19:31.764 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.60236.log 2024-09-12T14:19:31.765 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.100016.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.93264.log 2024-09-12T14:19:31.765 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.100016.log.gz 2024-09-12T14:19:31.766 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.60236.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.60236.log.gz 2024-09-12T14:19:31.766 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.106802.log 2024-09-12T14:19:31.766 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.93264.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.93264.log.gz 2024-09-12T14:19:31.767 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.80310.log 2024-09-12T14:19:31.767 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.106802.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.106802.log.gz 2024-09-12T14:19:31.768 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.122355.log 2024-09-12T14:19:31.768 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.80310.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.80310.log.gz 2024-09-12T14:19:31.768 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.139679.log 2024-09-12T14:19:31.769 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.122355.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.122355.log.gz 2024-09-12T14:19:31.769 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.141717.log 2024-09-12T14:19:31.770 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.139679.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.40475.log 2024-09-12T14:19:31.770 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.139679.log.gz 2024-09-12T14:19:31.771 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.141717.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.141717.log.gz 2024-09-12T14:19:31.771 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.91710.log 2024-09-12T14:19:31.772 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.40475.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.40475.log.gz 2024-09-12T14:19:31.772 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.75121.log 2024-09-12T14:19:31.772 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.91710.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.91710.log.gz 2024-09-12T14:19:31.773 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.85621.log 2024-09-12T14:19:31.773 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.75121.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.75121.log.gz 2024-09-12T14:19:31.773 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.119940.log 2024-09-12T14:19:31.774 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.85621.log: gzip 0.0% -5 --verbose -- replaced with /var/log/ceph/cluster1-client.mirror.85621.log.gz -- 2024-09-12T14:19:31.774 INFO:teuthology.orchestra.run.smithi130.stderr: /var/log/ceph/cluster1-client.admin.43948.log 2024-09-12T14:19:31.775 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.119940.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.119940.log.gz 2024-09-12T14:19:31.775 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.86012.log 2024-09-12T14:19:31.775 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.43948.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.43948.log.gz 2024-09-12T14:19:31.776 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.97496.log 2024-09-12T14:19:31.776 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.86012.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.86012.log.gz 2024-09-12T14:19:31.777 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.93745.log 2024-09-12T14:19:31.777 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.97496.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.97496.log.gz 2024-09-12T14:19:31.777 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.90469.log 2024-09-12T14:19:31.778 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.93745.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.93745.log.gz 2024-09-12T14:19:31.778 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.106442.log 2024-09-12T14:19:31.779 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.90469.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.136621.log 2024-09-12T14:19:31.779 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.90469.log.gz 2024-09-12T14:19:31.780 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.106442.log: 0.0%gzip -- replaced with /var/log/ceph/cluster1-client.mirror.106442.log.gz -5 2024-09-12T14:19:31.780 INFO:teuthology.orchestra.run.smithi130.stderr: --verbose -- /var/log/ceph/cluster1-client.admin.43783.log 2024-09-12T14:19:31.781 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.136621.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.136621.log.gz 2024-09-12T14:19:31.781 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.41268.log 2024-09-12T14:19:31.781 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.43783.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.43783.log.gz 2024-09-12T14:19:31.782 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.116468.log 2024-09-12T14:19:31.782 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.41268.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.41268.log.gz 2024-09-12T14:19:31.782 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.112677.log 2024-09-12T14:19:31.783 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.116468.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.116468.log.gz 2024-09-12T14:19:31.783 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.71780.log 2024-09-12T14:19:31.784 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.112677.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.138941.log 2024-09-12T14:19:31.784 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.112677.log.gz 2024-09-12T14:19:31.785 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.71780.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.71780.log.gz 2024-09-12T14:19:31.785 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.133205.log 2024-09-12T14:19:31.785 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.138941.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.138941.log.gz 2024-09-12T14:19:31.785 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.47300.log 2024-09-12T14:19:31.786 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.133205.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.133205.log.gz 2024-09-12T14:19:31.786 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.95671.log 2024-09-12T14:19:31.787 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.47300.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.47300.log.gz 2024-09-12T14:19:31.787 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.70465.log 2024-09-12T14:19:31.788 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.95671.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.95671.log.gz 2024-09-12T14:19:31.788 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.61632.log 2024-09-12T14:19:31.789 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.70465.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.70465.log.gz 2024-09-12T14:19:31.789 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.101359.log 2024-09-12T14:19:31.793 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.61632.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.61632.log.gz 2024-09-12T14:19:31.793 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.85020.log 2024-09-12T14:19:31.793 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.101359.log: gzip -5 0.0% --verbose -- -- replaced with /var/log/ceph/cluster1-client.mirror.101359.log.gz /var/log/ceph/cluster1-client.admin.151191.log 2024-09-12T14:19:31.793 INFO:teuthology.orchestra.run.smithi130.stderr: 2024-09-12T14:19:31.794 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.85020.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.85020.log.gz 2024-09-12T14:19:31.794 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.141074.log 2024-09-12T14:19:31.794 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.151191.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.151191.log.gz 2024-09-12T14:19:31.794 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.142115.log 2024-09-12T14:19:31.794 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.141074.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.141074.log.gzgzip 2024-09-12T14:19:31.794 INFO:teuthology.orchestra.run.smithi130.stderr: -5 --verbose -- /var/log/ceph/cluster1-client.mirror.64874.log 2024-09-12T14:19:31.794 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.142115.log: 0.0%gzip -- replaced with /var/log/ceph/cluster1-client.mirror.142115.log.gz 2024-09-12T14:19:31.794 INFO:teuthology.orchestra.run.smithi130.stderr: -5 --verbose -- /var/log/ceph/cluster1-client.mirror.135940.log 2024-09-12T14:19:31.796 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.64874.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.64874.log.gz 2024-09-12T14:19:31.796 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.56892.log 2024-09-12T14:19:31.796 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.135940.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.135940.log.gz 2024-09-12T14:19:31.796 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.49056.log 2024-09-12T14:19:31.796 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.56892.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.56892.log.gz 2024-09-12T14:19:31.796 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.59453.log 2024-09-12T14:19:31.797 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.49056.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.49056.log.gzgzip 2024-09-12T14:19:31.797 INFO:teuthology.orchestra.run.smithi130.stderr: -5 --verbose -- /var/log/ceph/cluster1-client.mirror.123284.log 2024-09-12T14:19:31.798 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.59453.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.59453.log.gz 2024-09-12T14:19:31.798 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.69895.log 2024-09-12T14:19:31.798 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.123284.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.123284.log.gz 2024-09-12T14:19:31.798 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.76235.log 2024-09-12T14:19:31.799 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.69895.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.69895.log.gz 2024-09-12T14:19:31.799 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.42616.log 2024-09-12T14:19:31.799 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.76235.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.76235.log.gz 2024-09-12T14:19:31.799 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.101066.log 2024-09-12T14:19:31.800 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.42616.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.42616.log.gz 2024-09-12T14:19:31.800 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.81885.log 2024-09-12T14:19:31.802 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.101066.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.101066.log.gz 2024-09-12T14:19:31.802 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.52263.log 2024-09-12T14:19:31.802 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.81885.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.81885.log.gz 2024-09-12T14:19:31.802 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.134155.log 2024-09-12T14:19:31.803 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.52263.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.52263.log.gz 2024-09-12T14:19:31.803 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.87880.log 2024-09-12T14:19:31.804 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.134155.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.134155.log.gz 2024-09-12T14:19:31.804 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.46831.log 2024-09-12T14:19:31.805 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.87880.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.87880.log.gz 2024-09-12T14:19:31.805 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.95123.log 2024-09-12T14:19:31.805 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.46831.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.46831.log.gz 2024-09-12T14:19:31.805 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.137960.log 2024-09-12T14:19:31.806 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.95123.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.95123.log.gz 2024-09-12T14:19:31.806 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.69239.log 2024-09-12T14:19:31.806 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.137960.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.137960.log.gz 2024-09-12T14:19:31.806 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.137476.log 2024-09-12T14:19:31.807 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.69239.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.69239.log.gz 2024-09-12T14:19:31.807 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.34486.log 2024-09-12T14:19:31.808 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.137476.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.137476.log.gz 2024-09-12T14:19:31.809 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.113623.log 2024-09-12T14:19:31.809 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.34486.log: gzip 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.34486.log.gz -5 2024-09-12T14:19:31.809 INFO:teuthology.orchestra.run.smithi130.stderr: --verbose -- /var/log/ceph/cluster1-client.mirror.111862.log 2024-09-12T14:19:31.810 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.113623.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.113623.log.gz 2024-09-12T14:19:31.810 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.131298.log 2024-09-12T14:19:31.810 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.111862.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.111862.log.gz 2024-09-12T14:19:31.810 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.120065.log 2024-09-12T14:19:31.811 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.131298.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.131298.log.gz 2024-09-12T14:19:31.811 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.133250.log 2024-09-12T14:19:31.812 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.120065.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.135697.log 2024-09-12T14:19:31.812 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.120065.log.gz 2024-09-12T14:19:31.812 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.133250.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.133250.log.gz 2024-09-12T14:19:31.813 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.69484.log 2024-09-12T14:19:31.814 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.135697.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.135697.log.gz 2024-09-12T14:19:31.814 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.61403.log 2024-09-12T14:19:31.814 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.69484.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.69484.log.gz 2024-09-12T14:19:31.814 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.98314.log 2024-09-12T14:19:31.815 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.61403.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.61403.log.gz 2024-09-12T14:19:31.815 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.66027.log 2024-09-12T14:19:31.815 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.98314.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.98314.log.gz 2024-09-12T14:19:31.816 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.75849.log 2024-09-12T14:19:31.816 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.66027.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.66027.log.gz 2024-09-12T14:19:31.817 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.75096.log 2024-09-12T14:19:31.817 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.75849.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.75849.log.gz 2024-09-12T14:19:31.817 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.99219.log 2024-09-12T14:19:31.818 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.75096.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.128609.log 2024-09-12T14:19:31.818 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.75096.log.gz 2024-09-12T14:19:31.819 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.99219.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.99219.log.gz 2024-09-12T14:19:31.819 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.46717.log 2024-09-12T14:19:31.820 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.128609.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.128609.log.gz 2024-09-12T14:19:31.820 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.152987.log 2024-09-12T14:19:31.821 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.46717.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.46717.log.gz 2024-09-12T14:19:31.821 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.57201.log 2024-09-12T14:19:31.822 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.152987.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.152987.log.gz 2024-09-12T14:19:31.822 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.121213.log 2024-09-12T14:19:31.822 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.57201.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.57201.log.gz 2024-09-12T14:19:31.822 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.39888.log 2024-09-12T14:19:31.823 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.121213.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.121213.log.gz 2024-09-12T14:19:31.823 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.64191.log 2024-09-12T14:19:31.824 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.39888.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.39888.log.gz 2024-09-12T14:19:31.824 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.152285.log 2024-09-12T14:19:31.825 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.64191.log: 0.0%gzip -- replaced with /var/log/ceph/cluster1-client.mirror.64191.log.gz 2024-09-12T14:19:31.825 INFO:teuthology.orchestra.run.smithi130.stderr: -5 --verbose -- /var/log/ceph/cluster1-client.mirror.36719.log 2024-09-12T14:19:31.826 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.152285.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.152285.log.gz 2024-09-12T14:19:31.826 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.70553.log 2024-09-12T14:19:31.826 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.36719.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.36719.log.gz 2024-09-12T14:19:31.827 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.86744.log 2024-09-12T14:19:31.827 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.70553.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.70553.log.gz 2024-09-12T14:19:31.827 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.73123.log 2024-09-12T14:19:31.828 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.86744.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.86744.log.gz 2024-09-12T14:19:31.828 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.123321.log 2024-09-12T14:19:31.829 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.73123.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.73123.log.gz 2024-09-12T14:19:31.829 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.69550.log 2024-09-12T14:19:31.830 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.123321.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.64242.log 2024-09-12T14:19:31.830 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.123321.log.gz 2024-09-12T14:19:31.830 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.69550.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.67341.log 2024-09-12T14:19:31.831 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.69550.log.gz 2024-09-12T14:19:31.831 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.64242.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.64242.log.gz 2024-09-12T14:19:31.832 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.35827.log 2024-09-12T14:19:31.833 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.67341.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.65707.log 2024-09-12T14:19:31.833 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.67341.log.gz 2024-09-12T14:19:31.833 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.35827.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.35827.log.gz 2024-09-12T14:19:31.833 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.120148.log 2024-09-12T14:19:31.834 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.65707.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.131781.log 2024-09-12T14:19:31.834 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.65707.log.gz 2024-09-12T14:19:31.834 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.120148.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.120148.log.gz 2024-09-12T14:19:31.834 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.115608.log 2024-09-12T14:19:31.835 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.131781.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.139125.log 2024-09-12T14:19:31.835 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.131781.log.gz 2024-09-12T14:19:31.836 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.115608.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.115608.log.gz 2024-09-12T14:19:31.836 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.146344.log 2024-09-12T14:19:31.837 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.139125.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.139125.log.gz 2024-09-12T14:19:31.837 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.152961.log 2024-09-12T14:19:31.838 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.146344.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.146344.log.gz 2024-09-12T14:19:31.838 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.62896.log 2024-09-12T14:19:31.839 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.152961.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.152961.log.gz 2024-09-12T14:19:31.839 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.61631.log 2024-09-12T14:19:31.839 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.136888.log 2024-09-12T14:19:31.840 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.62896.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.62896.log.gz 2024-09-12T14:19:31.840 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.61631.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.61631.log.gz 2024-09-12T14:19:31.840 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.41609.log 2024-09-12T14:19:31.841 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.136888.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.118395.log 2024-09-12T14:19:31.841 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.136888.log.gz 2024-09-12T14:19:31.842 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.41609.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.41609.log.gz 2024-09-12T14:19:31.843 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.121828.log 2024-09-12T14:19:31.843 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.118395.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.118395.log.gz 2024-09-12T14:19:31.843 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.65583.log 2024-09-12T14:19:31.844 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.121828.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.121828.log.gz 2024-09-12T14:19:31.844 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-osd.0.log 2024-09-12T14:19:31.845 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.65583.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.65583.log.gz 2024-09-12T14:19:31.845 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.75663.log 2024-09-12T14:19:31.846 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-osd.0.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.61006.log 2024-09-12T14:19:31.846 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.75663.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.75663.log.gz 2024-09-12T14:19:31.846 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.84792.log 2024-09-12T14:19:31.847 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.61006.log: 0.0%gzip -- replaced with /var/log/ceph/cluster1-client.mirror.61006.log.gz 2024-09-12T14:19:31.847 INFO:teuthology.orchestra.run.smithi130.stderr: -5 --verbose -- /var/log/ceph/cluster2-client.mirror.58773.log 2024-09-12T14:19:31.847 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.84792.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.84792.log.gz 2024-09-12T14:19:31.848 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.53104.log 2024-09-12T14:19:31.849 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.58773.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.77393.log 2024-09-12T14:19:31.849 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.58773.log.gz 2024-09-12T14:19:31.849 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.53104.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.53104.log.gz 2024-09-12T14:19:31.849 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.85904.log 2024-09-12T14:19:31.850 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.77393.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.61951.log 2024-09-12T14:19:31.850 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.77393.log.gz 2024-09-12T14:19:31.850 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.85904.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.85904.log.gz 2024-09-12T14:19:31.851 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.108854.log 2024-09-12T14:19:31.852 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.61951.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.61951.log.gz 2024-09-12T14:19:31.852 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.150180.log 2024-09-12T14:19:31.853 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.108854.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.99159.log 2024-09-12T14:19:31.853 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.108854.log.gz 2024-09-12T14:19:31.853 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.150180.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.150180.log.gz 2024-09-12T14:19:31.853 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.128071.log 2024-09-12T14:19:31.854 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.99159.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.99159.log.gz 2024-09-12T14:19:31.855 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.119508.log 2024-09-12T14:19:31.855 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.128071.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.128071.log.gz 2024-09-12T14:19:31.855 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.32106.log 2024-09-12T14:19:31.855 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.119508.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.133880.log 2024-09-12T14:19:31.856 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.119508.log.gz 2024-09-12T14:19:31.857 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.32106.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.32106.log.gz 2024-09-12T14:19:31.857 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.72873.log 2024-09-12T14:19:31.857 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.133880.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.133880.log.gz 2024-09-12T14:19:31.857 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.108498.log 2024-09-12T14:19:31.858 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.72873.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.72873.log.gz 2024-09-12T14:19:31.858 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.129275.log 2024-09-12T14:19:31.859 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.108498.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.108498.log.gz 2024-09-12T14:19:31.859 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.132973.log 2024-09-12T14:19:31.860 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.129275.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.129275.log.gz 2024-09-12T14:19:31.860 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.131864.log 2024-09-12T14:19:31.860 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.132973.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.132973.log.gz 2024-09-12T14:19:31.860 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.112467.log 2024-09-12T14:19:31.861 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.131864.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.131864.log.gz 2024-09-12T14:19:31.862 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.90447.log 2024-09-12T14:19:31.862 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.112467.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.112467.log.gz 2024-09-12T14:19:31.862 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.143362.log 2024-09-12T14:19:31.863 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.90447.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.90447.log.gz 2024-09-12T14:19:31.863 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.33982.log 2024-09-12T14:19:31.863 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.143362.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.143362.log.gz 2024-09-12T14:19:31.863 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.71629.log 2024-09-12T14:19:31.865 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.33982.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.33982.log.gz 2024-09-12T14:19:31.865 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.88720.log 2024-09-12T14:19:31.865 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.71629.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.71629.log.gz 2024-09-12T14:19:31.865 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.39339.log 2024-09-12T14:19:31.866 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.88720.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.88720.log.gz 2024-09-12T14:19:31.866 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.129202.log 2024-09-12T14:19:31.867 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.39339.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.39339.log.gz 2024-09-12T14:19:31.867 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.93912.log 2024-09-12T14:19:31.867 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.129202.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.129202.log.gz 2024-09-12T14:19:31.867 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.63820.log 2024-09-12T14:19:31.868 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.93912.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.93912.log.gz 2024-09-12T14:19:31.868 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.51476.log 2024-09-12T14:19:31.869 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.63820.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.63820.log.gz 2024-09-12T14:19:31.869 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.36881.log 2024-09-12T14:19:31.870 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.51476.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.51476.log.gz 2024-09-12T14:19:31.870 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.42530.log 2024-09-12T14:19:31.871 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.36881.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.36881.log.gz 2024-09-12T14:19:31.871 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.99508.log 2024-09-12T14:19:31.871 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.42530.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.42530.log.gz 2024-09-12T14:19:31.872 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.112446.log 2024-09-12T14:19:31.872 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.99508.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.99508.log.gz 2024-09-12T14:19:31.872 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.114689.log 2024-09-12T14:19:31.873 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.112446.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.112446.log.gz 2024-09-12T14:19:31.873 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.116044.log 2024-09-12T14:19:31.874 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.114689.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.114689.log.gz 2024-09-12T14:19:31.875 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.146759.log 2024-09-12T14:19:31.875 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.116044.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.116044.log.gz 2024-09-12T14:19:31.875 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.149908.log 2024-09-12T14:19:31.876 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.146759.log: 0.0%gzip -- replaced with /var/log/ceph/cluster1-client.mirror.146759.log.gz 2024-09-12T14:19:31.876 INFO:teuthology.orchestra.run.smithi130.stderr: -5 --verbose -- /var/log/ceph/cluster1-client.admin.54618.log 2024-09-12T14:19:31.876 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.149908.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.149908.log.gz 2024-09-12T14:19:31.876 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.78275.log 2024-09-12T14:19:31.877 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.54618.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.54618.log.gz 2024-09-12T14:19:31.877 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.120064.log 2024-09-12T14:19:31.878 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.78275.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.78275.log.gz 2024-09-12T14:19:31.878 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.129544.log 2024-09-12T14:19:31.879 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.120064.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.120064.log.gz 2024-09-12T14:19:31.879 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.151798.log 2024-09-12T14:19:31.880 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.129544.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.129544.log.gz 2024-09-12T14:19:31.880 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.53189.log 2024-09-12T14:19:31.880 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.151798.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.151798.log.gz 2024-09-12T14:19:31.880 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.60064.log 2024-09-12T14:19:31.881 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.53189.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.53189.log.gz 2024-09-12T14:19:31.881 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.75202.log 2024-09-12T14:19:31.882 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.60064.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.60064.log.gz 2024-09-12T14:19:31.882 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.89747.log 2024-09-12T14:19:31.883 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.75202.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.75202.log.gz 2024-09-12T14:19:31.884 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.129295.log 2024-09-12T14:19:31.884 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.89747.log: gzip 0.0% -5 -- replaced with /var/log/ceph/cluster1-client.mirror.89747.log.gz --verbose 2024-09-12T14:19:31.884 INFO:teuthology.orchestra.run.smithi130.stderr: -- /var/log/ceph/cluster1-client.mirror.71410.log 2024-09-12T14:19:31.885 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.129295.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.36112.log 2024-09-12T14:19:31.885 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.129295.log.gz 2024-09-12T14:19:31.885 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.71410.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.94228.log 2024-09-12T14:19:31.885 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.71410.log.gz 2024-09-12T14:19:31.886 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.36112.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.36112.log.gz 2024-09-12T14:19:31.886 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.74996.log 2024-09-12T14:19:31.886 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.94228.log: gzip 0.0% -5 --verbose -- replaced with /var/log/ceph/cluster1-client.admin.94228.log.gz -- 2024-09-12T14:19:31.886 INFO:teuthology.orchestra.run.smithi130.stderr: /var/log/ceph/cluster1-client.mirror.108038.log 2024-09-12T14:19:31.887 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.74996.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.74996.log.gz 2024-09-12T14:19:31.887 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.85483.log 2024-09-12T14:19:31.888 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.108038.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.108038.log.gz 2024-09-12T14:19:31.888 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.135142.log 2024-09-12T14:19:31.889 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.85483.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.85483.log.gz 2024-09-12T14:19:31.889 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.152820.log 2024-09-12T14:19:31.889 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.135142.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.135142.log.gz 2024-09-12T14:19:31.890 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.102105.log 2024-09-12T14:19:31.890 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.152820.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.36308.log 2024-09-12T14:19:31.890 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.152820.log.gz 2024-09-12T14:19:31.891 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.102105.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.102105.log.gz 2024-09-12T14:19:31.891 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.92808.log 2024-09-12T14:19:31.892 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.36308.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.36308.log.gz 2024-09-12T14:19:31.892 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.110074.log 2024-09-12T14:19:31.893 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.92808.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.92808.log.gz 2024-09-12T14:19:31.893 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.140866.log 2024-09-12T14:19:31.894 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.110074.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.77726.log 2024-09-12T14:19:31.894 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.110074.log.gz 2024-09-12T14:19:31.894 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.140866.log: gzip -5 --verbose 0.0% -- -- replaced with /var/log/ceph/cluster1-client.admin.140866.log.gz /var/log/ceph/cluster1-client.mirror.143334.log 2024-09-12T14:19:31.894 INFO:teuthology.orchestra.run.smithi130.stderr: 2024-09-12T14:19:31.895 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.77726.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.77726.log.gz 2024-09-12T14:19:31.895 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.43036.log 2024-09-12T14:19:31.896 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.143334.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.143334.log.gz 2024-09-12T14:19:31.896 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.113477.log 2024-09-12T14:19:31.897 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.43036.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.43036.log.gz 2024-09-12T14:19:31.897 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.44223.log 2024-09-12T14:19:31.897 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.113477.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.113477.log.gz 2024-09-12T14:19:31.897 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.66658.log 2024-09-12T14:19:31.898 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.44223.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.44223.log.gz 2024-09-12T14:19:31.898 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.137975.log 2024-09-12T14:19:31.899 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.66658.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.66658.log.gz 2024-09-12T14:19:31.899 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.100937.log 2024-09-12T14:19:31.900 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.137975.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.137975.log.gz 2024-09-12T14:19:31.900 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.42405.log 2024-09-12T14:19:31.901 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.100937.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.100937.log.gzgzip 2024-09-12T14:19:31.901 INFO:teuthology.orchestra.run.smithi130.stderr: -5 --verbose -- /var/log/ceph/cluster2-client.mirror.116435.log 2024-09-12T14:19:31.901 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.42405.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.38916.log 2024-09-12T14:19:31.902 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.42405.log.gz 2024-09-12T14:19:31.903 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.116435.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.116435.log.gz 2024-09-12T14:19:31.903 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.42095.log 2024-09-12T14:19:31.903 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.38916.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.38916.log.gz 2024-09-12T14:19:31.903 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.100855.log 2024-09-12T14:19:31.904 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.42095.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.42095.log.gz 2024-09-12T14:19:31.904 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.43183.log 2024-09-12T14:19:31.905 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.100855.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.100855.log.gz 2024-09-12T14:19:31.905 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.75412.log 2024-09-12T14:19:31.905 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.43183.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.104531.log 2024-09-12T14:19:31.906 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.43183.log.gz 2024-09-12T14:19:31.906 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.75412.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.75412.log.gz 2024-09-12T14:19:31.906 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.115635.log 2024-09-12T14:19:31.907 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.104531.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.104531.log.gz 2024-09-12T14:19:31.908 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.123366.log 2024-09-12T14:19:31.908 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.115635.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.115635.log.gz 2024-09-12T14:19:31.908 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.150017.log 2024-09-12T14:19:31.909 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.123366.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.123366.log.gz 2024-09-12T14:19:31.909 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.84832.log 2024-09-12T14:19:31.910 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.150017.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.150017.log.gz 2024-09-12T14:19:31.910 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.144063.log 2024-09-12T14:19:31.910 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.84832.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.84832.log.gz 2024-09-12T14:19:31.910 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.124757.log 2024-09-12T14:19:31.911 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.144063.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.144063.log.gz 2024-09-12T14:19:31.912 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.77391.log 2024-09-12T14:19:31.912 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.124757.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.148568.log 2024-09-12T14:19:31.912 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.124757.log.gz 2024-09-12T14:19:31.913 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.77391.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.77391.log.gz 2024-09-12T14:19:31.913 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.78488.log 2024-09-12T14:19:31.913 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.148568.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.148568.log.gz 2024-09-12T14:19:31.914 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.81076.log 2024-09-12T14:19:31.915 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.78488.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.78488.log.gz 2024-09-12T14:19:31.915 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.127606.log 2024-09-12T14:19:31.915 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.81076.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.81076.log.gz 2024-09-12T14:19:31.915 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.69641.log 2024-09-12T14:19:31.916 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.127606.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.105363.log 2024-09-12T14:19:31.916 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.127606.log.gz 2024-09-12T14:19:31.916 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.69641.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.69641.log.gz 2024-09-12T14:19:31.916 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.54596.log 2024-09-12T14:19:31.917 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.105363.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.105363.log.gz 2024-09-12T14:19:31.917 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.57293.log 2024-09-12T14:19:31.918 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.54596.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.54596.log.gz 2024-09-12T14:19:31.918 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.127203.log 2024-09-12T14:19:31.919 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.57293.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.57293.log.gz 2024-09-12T14:19:31.919 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.57208.log 2024-09-12T14:19:31.919 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.127203.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.89095.log 2024-09-12T14:19:31.920 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.127203.log.gz 2024-09-12T14:19:31.920 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.57208.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.57208.log.gz 2024-09-12T14:19:31.920 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.125182.log 2024-09-12T14:19:31.921 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.89095.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.89095.log.gz 2024-09-12T14:19:31.921 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.37805.log 2024-09-12T14:19:31.922 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.125182.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.125182.log.gz 2024-09-12T14:19:31.922 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.118589.log 2024-09-12T14:19:31.923 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.37805.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.37805.log.gz 2024-09-12T14:19:31.923 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.147598.log 2024-09-12T14:19:31.923 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.118589.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.151688.log 2024-09-12T14:19:31.924 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.118589.log.gz 2024-09-12T14:19:31.924 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.147598.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.147598.log.gz 2024-09-12T14:19:31.924 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.119810.log 2024-09-12T14:19:31.925 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.151688.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.113603.log 2024-09-12T14:19:31.925 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.151688.log.gz 2024-09-12T14:19:31.926 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.119810.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.119810.log.gz 2024-09-12T14:19:31.926 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.93476.log 2024-09-12T14:19:31.927 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.113603.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.113603.log.gz 2024-09-12T14:19:31.927 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.80831.log 2024-09-12T14:19:31.928 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.93476.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.93476.log.gz 2024-09-12T14:19:31.928 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.138063.log 2024-09-12T14:19:31.928 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.80831.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.129450.log 2024-09-12T14:19:31.928 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.80831.log.gz 2024-09-12T14:19:31.929 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.138063.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.129477.log 2024-09-12T14:19:31.929 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.138063.log.gz 2024-09-12T14:19:31.930 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.129450.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.129450.log.gz 2024-09-12T14:19:31.930 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.54450.log 2024-09-12T14:19:31.931 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.129477.log: gzip -5 --verbose -- 0.0% /var/log/ceph/cluster1-client.admin.139071.log 2024-09-12T14:19:31.931 INFO:teuthology.orchestra.run.smithi130.stderr: -- replaced with /var/log/ceph/cluster1-client.admin.129477.log.gz 2024-09-12T14:19:31.931 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.54450.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.54450.log.gz 2024-09-12T14:19:31.932 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.110888.log 2024-09-12T14:19:31.932 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.139071.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.138698.log 2024-09-12T14:19:31.932 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.139071.log.gz 2024-09-12T14:19:31.933 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.110888.log: gzip -5 0.0% --verbose -- replaced with /var/log/ceph/cluster1-client.mirror.110888.log.gz 2024-09-12T14:19:31.933 INFO:teuthology.orchestra.run.smithi130.stderr: -- /var/log/ceph/cluster1-client.mirror.147811.log 2024-09-12T14:19:31.934 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.138698.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.138698.log.gz 2024-09-12T14:19:31.934 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.105778.log 2024-09-12T14:19:31.935 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.147811.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.147811.log.gz 2024-09-12T14:19:31.936 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.87521.log 2024-09-12T14:19:31.936 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.105778.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.105778.log.gz 2024-09-12T14:19:31.936 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.120565.log 2024-09-12T14:19:31.936 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.87521.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.87521.log.gz 2024-09-12T14:19:31.937 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.135772.log 2024-09-12T14:19:31.937 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.120565.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.120565.log.gz 2024-09-12T14:19:31.937 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.47628.log 2024-09-12T14:19:31.939 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.135772.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.135772.log.gz 2024-09-12T14:19:31.939 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.97811.log 2024-09-12T14:19:31.939 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.47628.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.47628.log.gz 2024-09-12T14:19:31.939 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.104722.log 2024-09-12T14:19:31.940 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.97811.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.97811.log.gz 2024-09-12T14:19:31.940 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.87835.log 2024-09-12T14:19:31.940 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.104722.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.104722.log.gz 2024-09-12T14:19:31.941 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.154257.log 2024-09-12T14:19:31.941 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.87835.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.87835.log.gz 2024-09-12T14:19:31.941 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.34064.log 2024-09-12T14:19:31.942 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.154257.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.109568.log 2024-09-12T14:19:31.943 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.154257.log.gz 2024-09-12T14:19:31.943 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.34064.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.34064.log.gz 2024-09-12T14:19:31.943 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.148110.log 2024-09-12T14:19:31.943 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.109568.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.148611.log 2024-09-12T14:19:31.943 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.109568.log.gz 2024-09-12T14:19:31.944 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.148110.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.148110.log.gz 2024-09-12T14:19:31.944 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.135306.log 2024-09-12T14:19:31.945 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.148611.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.148611.log.gz 2024-09-12T14:19:31.945 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.65082.log 2024-09-12T14:19:31.946 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.135306.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.135306.log.gz 2024-09-12T14:19:31.946 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.40999.log 2024-09-12T14:19:31.947 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.65082.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.65082.log.gz 2024-09-12T14:19:31.947 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.65770.log 2024-09-12T14:19:31.947 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.40999.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.40999.log.gz 2024-09-12T14:19:31.948 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.74804.log 2024-09-12T14:19:31.948 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.65770.log: 0.0%gzip -- replaced with /var/log/ceph/cluster1-client.mirror.65770.log.gz -5 2024-09-12T14:19:31.948 INFO:teuthology.orchestra.run.smithi130.stderr: --verbose -- /var/log/ceph/cluster1-client.mirror.86239.log 2024-09-12T14:19:31.949 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.74804.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.144441.log 2024-09-12T14:19:31.949 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.74804.log.gz 2024-09-12T14:19:31.950 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.86239.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.86239.log.gz 2024-09-12T14:19:31.950 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.76105.log 2024-09-12T14:19:31.950 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.144441.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.94934.log 2024-09-12T14:19:31.951 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.144441.log.gz 2024-09-12T14:19:31.951 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.76105.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.40599.log 2024-09-12T14:19:31.951 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.76105.log.gz 2024-09-12T14:19:31.952 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.94934.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.94934.log.gz 2024-09-12T14:19:31.952 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.121638.log 2024-09-12T14:19:31.953 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.40599.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.40599.log.gz 2024-09-12T14:19:31.953 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.130707.log 2024-09-12T14:19:31.954 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.121638.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.77849.log 2024-09-12T14:19:31.954 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.121638.log.gz 2024-09-12T14:19:31.955 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.130707.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.130707.log.gz 2024-09-12T14:19:31.955 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.151393.log 2024-09-12T14:19:31.955 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.77849.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.77849.log.gz 2024-09-12T14:19:31.956 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.77137.log 2024-09-12T14:19:31.956 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.151393.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.151393.log.gz 2024-09-12T14:19:31.956 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.124565.log 2024-09-12T14:19:31.957 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.77137.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.77137.log.gz 2024-09-12T14:19:31.957 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.93316.log 2024-09-12T14:19:31.958 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.124565.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.124565.log.gz 2024-09-12T14:19:31.958 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.69842.log 2024-09-12T14:19:31.959 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.93316.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.93316.log.gz 2024-09-12T14:19:31.959 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.137915.log 2024-09-12T14:19:31.959 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.69842.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.120107.log 2024-09-12T14:19:31.959 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.69842.log.gz 2024-09-12T14:19:31.960 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.137915.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.137915.log.gz 2024-09-12T14:19:31.960 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.72540.log 2024-09-12T14:19:31.961 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.120107.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.120107.log.gz 2024-09-12T14:19:31.962 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.58700.log 2024-09-12T14:19:31.962 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.72540.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.72540.log.gz 2024-09-12T14:19:31.962 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.128400.log 2024-09-12T14:19:31.963 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.58700.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.37195.log 2024-09-12T14:19:31.963 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.58700.log.gz 2024-09-12T14:19:31.964 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.128400.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.128400.log.gz 2024-09-12T14:19:31.964 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.72820.log 2024-09-12T14:19:31.965 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.37195.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.37195.log.gz 2024-09-12T14:19:31.965 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.133294.log 2024-09-12T14:19:31.965 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.72820.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.72820.log.gz 2024-09-12T14:19:31.966 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.92651.log 2024-09-12T14:19:31.966 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.133294.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.133294.log.gz 2024-09-12T14:19:31.966 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.147851.log 2024-09-12T14:19:31.967 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.92651.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.92651.log.gz 2024-09-12T14:19:31.967 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.98292.log 2024-09-12T14:19:31.968 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.36529.log 2024-09-12T14:19:31.969 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.147851.log: /var/log/ceph/cluster1-client.mirror.98292.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.147851.log.gz 2024-09-12T14:19:31.969 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.98292.log.gz 2024-09-12T14:19:31.969 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.45827.log 2024-09-12T14:19:31.969 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.36529.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.40725.log 2024-09-12T14:19:31.970 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.36529.log.gz 2024-09-12T14:19:31.970 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.45827.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.45827.log.gz 2024-09-12T14:19:31.970 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.39137.log 2024-09-12T14:19:31.971 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.40725.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.40725.log.gz 2024-09-12T14:19:31.971 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.86954.log 2024-09-12T14:19:31.972 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.39137.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.39137.log.gz 2024-09-12T14:19:31.972 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.74122.log 2024-09-12T14:19:31.973 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.86954.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.86954.log.gz 2024-09-12T14:19:31.973 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.97293.log 2024-09-12T14:19:31.973 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.74122.log: gzip 0.0% -5 -- replaced with /var/log/ceph/cluster1-client.mirror.74122.log.gz --verbose -- 2024-09-12T14:19:31.974 INFO:teuthology.orchestra.run.smithi130.stderr: /var/log/ceph/cluster1-client.admin.117810.log 2024-09-12T14:19:31.974 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.97293.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.131579.log 2024-09-12T14:19:31.974 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.97293.log.gz 2024-09-12T14:19:31.975 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.117810.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.117810.log.gzgzip 2024-09-12T14:19:31.975 INFO:teuthology.orchestra.run.smithi130.stderr: -5 --verbose -- /var/log/ceph/cluster1-client.mirror.135600.log 2024-09-12T14:19:31.976 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.131579.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.131579.log.gz 2024-09-12T14:19:31.976 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.115661.log 2024-09-12T14:19:31.977 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.135600.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.135600.log.gz 2024-09-12T14:19:31.977 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.45229.log 2024-09-12T14:19:31.978 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.115661.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.115661.log.gz 2024-09-12T14:19:31.978 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.66680.log 2024-09-12T14:19:31.978 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.45229.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.45229.log.gz 2024-09-12T14:19:31.979 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.103859.log 2024-09-12T14:19:31.979 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.66680.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.66680.log.gz 2024-09-12T14:19:31.979 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.139379.log 2024-09-12T14:19:31.981 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.103859.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.103859.log.gz 2024-09-12T14:19:31.981 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.100435.log 2024-09-12T14:19:31.981 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.139379.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.139379.log.gz 2024-09-12T14:19:31.981 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.42799.log 2024-09-12T14:19:31.982 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.100435.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.100435.log.gz 2024-09-12T14:19:31.982 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.72810.log 2024-09-12T14:19:31.982 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.42799.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.42799.log.gz 2024-09-12T14:19:31.982 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.50207.log 2024-09-12T14:19:31.983 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.72810.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.72810.log.gz 2024-09-12T14:19:31.983 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.62095.log 2024-09-12T14:19:31.984 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.50207.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.105664.log 2024-09-12T14:19:31.984 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.50207.log.gz 2024-09-12T14:19:31.985 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.62095.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.62095.log.gz 2024-09-12T14:19:31.985 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.89453.log 2024-09-12T14:19:31.985 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.74657.log 2024-09-12T14:19:31.985 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.105664.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.105664.log.gz 2024-09-12T14:19:31.986 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.89453.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.89453.log.gz 2024-09-12T14:19:31.986 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.63066.log 2024-09-12T14:19:31.987 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.74657.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.74657.log.gz 2024-09-12T14:19:31.987 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.105065.log 2024-09-12T14:19:31.988 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.63066.log: gzip 0.0% -5 -- replaced with /var/log/ceph/cluster1-client.mirror.63066.log.gz --verbose 2024-09-12T14:19:31.988 INFO:teuthology.orchestra.run.smithi130.stderr: -- /var/log/ceph/cluster1-client.mirror.42255.log 2024-09-12T14:19:31.989 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.105065.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.105065.log.gz 2024-09-12T14:19:31.989 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.132536.log 2024-09-12T14:19:31.989 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.42255.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.42255.log.gz 2024-09-12T14:19:31.989 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.65500.log 2024-09-12T14:19:31.990 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.132536.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.132536.log.gz 2024-09-12T14:19:31.990 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.86651.log 2024-09-12T14:19:31.991 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.65500.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.65500.log.gz 2024-09-12T14:19:31.991 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.93517.log 2024-09-12T14:19:31.992 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.73349.log 2024-09-12T14:19:31.992 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.86651.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.86651.log.gz 2024-09-12T14:19:31.992 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.93517.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.93517.log.gz 2024-09-12T14:19:31.992 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.103362.log 2024-09-12T14:19:31.993 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.73349.log: gzip 0.0% -5 -- replaced with /var/log/ceph/cluster1-client.mirror.73349.log.gz 2024-09-12T14:19:31.993 INFO:teuthology.orchestra.run.smithi130.stderr: --verbose -- /var/log/ceph/cluster1-client.mirror.72541.log 2024-09-12T14:19:31.994 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.103362.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.103362.log.gz 2024-09-12T14:19:31.994 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.138786.log 2024-09-12T14:19:31.995 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.72541.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.72541.log.gz 2024-09-12T14:19:31.995 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.93016.log 2024-09-12T14:19:31.996 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.138786.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.138786.log.gz 2024-09-12T14:19:31.996 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.65139.log 2024-09-12T14:19:31.996 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.93016.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.93016.log.gz 2024-09-12T14:19:31.997 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.122011.log 2024-09-12T14:19:31.997 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.65139.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.65139.log.gz 2024-09-12T14:19:31.997 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.69156.log 2024-09-12T14:19:31.998 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.122011.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.122011.log.gz 2024-09-12T14:19:31.998 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.94887.log 2024-09-12T14:19:31.999 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.69156.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.69156.log.gz 2024-09-12T14:19:31.999 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.143795.log 2024-09-12T14:19:32.000 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.94887.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.94887.log.gz 2024-09-12T14:19:32.000 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.125114.log 2024-09-12T14:19:32.000 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.143795.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.143795.log.gz 2024-09-12T14:19:32.001 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.89140.log 2024-09-12T14:19:32.001 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.125114.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.125114.log.gz 2024-09-12T14:19:32.001 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.85694.log 2024-09-12T14:19:32.002 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.89140.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.89140.log.gz 2024-09-12T14:19:32.002 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.151686.log 2024-09-12T14:19:32.003 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.85694.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.85694.log.gz 2024-09-12T14:19:32.003 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.127093.log 2024-09-12T14:19:32.004 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.151686.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.151686.log.gz 2024-09-12T14:19:32.004 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.99047.log 2024-09-12T14:19:32.005 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.127093.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.127093.log.gz 2024-09-12T14:19:32.005 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.54241.log 2024-09-12T14:19:32.005 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.99047.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.99047.log.gz 2024-09-12T14:19:32.006 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.129705.log 2024-09-12T14:19:32.006 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.54241.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.91960.log 2024-09-12T14:19:32.006 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.54241.log.gz 2024-09-12T14:19:32.007 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.129705.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.129705.log.gz 2024-09-12T14:19:32.007 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.55687.log 2024-09-12T14:19:32.008 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.91960.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.91960.log.gz 2024-09-12T14:19:32.008 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.141805.log 2024-09-12T14:19:32.009 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.55687.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.55687.log.gz 2024-09-12T14:19:32.009 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.110178.log 2024-09-12T14:19:32.010 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.141805.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.141805.log.gz 2024-09-12T14:19:32.010 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.145962.log 2024-09-12T14:19:32.010 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.110178.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.110178.log.gz 2024-09-12T14:19:32.011 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.109928.log 2024-09-12T14:19:32.011 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.145962.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.145962.log.gz 2024-09-12T14:19:32.011 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.113375.log 2024-09-12T14:19:32.012 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.109928.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.109928.log.gz 2024-09-12T14:19:32.012 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.35530.log 2024-09-12T14:19:32.013 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.113375.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.113375.log.gz 2024-09-12T14:19:32.013 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.144704.log 2024-09-12T14:19:32.014 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.35530.log: 0.0%gzip -- replaced with /var/log/ceph/cluster1-client.mirror.35530.log.gz 2024-09-12T14:19:32.014 INFO:teuthology.orchestra.run.smithi130.stderr: -5 --verbose -- /var/log/ceph/cluster1-client.mirror.109569.log 2024-09-12T14:19:32.014 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.144704.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.144704.log.gz 2024-09-12T14:19:32.014 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.115218.log 2024-09-12T14:19:32.015 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.109569.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.145439.log 2024-09-12T14:19:32.015 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.109569.log.gz 2024-09-12T14:19:32.016 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.115218.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.115218.log.gz 2024-09-12T14:19:32.016 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.104523.log 2024-09-12T14:19:32.017 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.145439.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.145439.log.gz 2024-09-12T14:19:32.017 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.123175.log 2024-09-12T14:19:32.017 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.104523.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.104523.log.gz 2024-09-12T14:19:32.018 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.53879.log 2024-09-12T14:19:32.018 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.123175.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.123175.log.gz 2024-09-12T14:19:32.018 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.94409.log 2024-09-12T14:19:32.019 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.53879.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.53879.log.gz 2024-09-12T14:19:32.019 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.134177.log 2024-09-12T14:19:32.020 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.94409.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.100895.log 2024-09-12T14:19:32.020 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.94409.log.gz 2024-09-12T14:19:32.021 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.134177.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.134177.log.gz 2024-09-12T14:19:32.021 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.87578.log 2024-09-12T14:19:32.022 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.100895.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.100895.log.gz 2024-09-12T14:19:32.022 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.91077.log 2024-09-12T14:19:32.022 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.87578.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.87578.log.gz 2024-09-12T14:19:32.023 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.38226.log 2024-09-12T14:19:32.023 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.91077.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.91077.log.gz 2024-09-12T14:19:32.023 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.52181.log 2024-09-12T14:19:32.024 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.38226.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.38226.log.gz 2024-09-12T14:19:32.024 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.136885.log 2024-09-12T14:19:32.025 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.52181.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.136822.log 2024-09-12T14:19:32.025 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.52181.log.gz 2024-09-12T14:19:32.026 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.136885.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.136885.log.gz 2024-09-12T14:19:32.026 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.135286.log 2024-09-12T14:19:32.026 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.136822.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.136822.log.gz 2024-09-12T14:19:32.027 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.148188.log 2024-09-12T14:19:32.027 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.135286.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.88847.log 2024-09-12T14:19:32.027 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.135286.log.gz 2024-09-12T14:19:32.028 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.148188.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.133396.log 2024-09-12T14:19:32.028 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.148188.log.gz 2024-09-12T14:19:32.029 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.88847.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.88847.log.gz 2024-09-12T14:19:32.029 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.146847.log 2024-09-12T14:19:32.030 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.133396.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.133396.log.gz 2024-09-12T14:19:32.030 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.75388.log 2024-09-12T14:19:32.031 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.146847.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.146847.log.gz 2024-09-12T14:19:32.031 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.88130.log 2024-09-12T14:19:32.031 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.75388.log: gzip -5 --verbose 0.0% -- /var/log/ceph/cluster1-client.mirror.76798.log -- replaced with /var/log/ceph/cluster1-client.mirror.75388.log.gz 2024-09-12T14:19:32.031 INFO:teuthology.orchestra.run.smithi130.stderr: 2024-09-12T14:19:32.032 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.88130.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.54425.log 2024-09-12T14:19:32.032 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.88130.log.gz 2024-09-12T14:19:32.033 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.76798.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.76798.log.gz 2024-09-12T14:19:32.033 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.131928.log 2024-09-12T14:19:32.034 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.54425.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.54425.log.gz 2024-09-12T14:19:32.034 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.117293.log 2024-09-12T14:19:32.035 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.131928.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.131928.log.gz 2024-09-12T14:19:32.035 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.38956.log 2024-09-12T14:19:32.036 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.117293.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.117293.log.gz 2024-09-12T14:19:32.036 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.47934.log 2024-09-12T14:19:32.036 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.38956.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.38956.log.gz 2024-09-12T14:19:32.037 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.50383.log 2024-09-12T14:19:32.037 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.47934.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.47934.log.gz 2024-09-12T14:19:32.038 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.127318.log 2024-09-12T14:19:32.038 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.50383.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.50383.log.gz 2024-09-12T14:19:32.038 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.124755.log 2024-09-12T14:19:32.039 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.127318.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.127318.log.gz 2024-09-12T14:19:32.039 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.107035.log 2024-09-12T14:19:32.040 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.124755.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.124755.log.gz 2024-09-12T14:19:32.040 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.146802.log 2024-09-12T14:19:32.041 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.107035.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.107035.log.gz 2024-09-12T14:19:32.041 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.109695.log 2024-09-12T14:19:32.042 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.146802.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.146802.log.gz 2024-09-12T14:19:32.042 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.153007.log 2024-09-12T14:19:32.043 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.109695.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.109695.log.gz 2024-09-12T14:19:32.043 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.82087.log 2024-09-12T14:19:32.044 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.153007.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.153007.log.gz 2024-09-12T14:19:32.044 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.70298.log 2024-09-12T14:19:32.044 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.82087.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.82087.log.gz 2024-09-12T14:19:32.045 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.128673.log 2024-09-12T14:19:32.045 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.70298.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.70298.log.gz 2024-09-12T14:19:32.045 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.50352.log 2024-09-12T14:19:32.046 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.128673.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.128673.log.gz 2024-09-12T14:19:32.046 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.60689.log 2024-09-12T14:19:32.047 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.50352.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.50352.log.gz 2024-09-12T14:19:32.047 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.0.34559.log 2024-09-12T14:19:32.048 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.60689.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.60689.log.gz 2024-09-12T14:19:32.048 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.123493.log 2024-09-12T14:19:32.049 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.0.34559.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.47469.log 2024-09-12T14:19:32.050 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.123493.log: 0.0%gzip -- replaced with /var/log/ceph/cluster1-client.mirror.123493.log.gz -5 2024-09-12T14:19:32.050 INFO:teuthology.orchestra.run.smithi130.stderr: --verbose -- /var/log/ceph/cluster1-client.admin.57808.log 2024-09-12T14:19:32.050 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.47469.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.47469.log.gz 2024-09-12T14:19:32.051 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.100393.log 2024-09-12T14:19:32.051 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.57808.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.57808.log.gz 2024-09-12T14:19:32.052 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.89960.log 2024-09-12T14:19:32.052 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.100393.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.100393.log.gz 2024-09-12T14:19:32.053 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.138516.log 2024-09-12T14:19:32.053 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.89960.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.89960.log.gz 2024-09-12T14:19:32.054 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.62013.log 2024-09-12T14:19:32.054 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.138516.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.138516.log.gz 2024-09-12T14:19:32.054 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.140090.log 2024-09-12T14:19:32.055 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.62013.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.135116.log 2024-09-12T14:19:32.055 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.62013.log.gz 2024-09-12T14:19:32.056 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.140090.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.140090.log.gz 2024-09-12T14:19:32.056 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.91253.log 2024-09-12T14:19:32.057 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.135116.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.135116.log.gz 2024-09-12T14:19:32.057 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.44028.log 2024-09-12T14:19:32.058 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.91253.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.91253.log.gz 2024-09-12T14:19:32.058 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.37635.log 2024-09-12T14:19:32.059 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.44028.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.44028.log.gz 2024-09-12T14:19:32.059 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.45991.log 2024-09-12T14:19:32.059 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.37635.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.37635.log.gz 2024-09-12T14:19:32.060 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.73480.log 2024-09-12T14:19:32.060 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.45991.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.47253.log 2024-09-12T14:19:32.060 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.45991.log.gz 2024-09-12T14:19:32.061 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.73480.log: gzip 0.0% -5 --verbose -- replaced with /var/log/ceph/cluster1-client.mirror.73480.log.gz -- 2024-09-12T14:19:32.061 INFO:teuthology.orchestra.run.smithi130.stderr: /var/log/ceph/cluster1-client.mirror.107015.log 2024-09-12T14:19:32.062 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.47253.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.47253.log.gz 2024-09-12T14:19:32.062 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.86577.log 2024-09-12T14:19:32.063 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.107015.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.107015.log.gz 2024-09-12T14:19:32.063 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.148524.log 2024-09-12T14:19:32.064 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.86577.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.86577.log.gz 2024-09-12T14:19:32.064 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.82646.log 2024-09-12T14:19:32.065 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.148524.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.79895.log 2024-09-12T14:19:32.065 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.148524.log.gz 2024-09-12T14:19:32.065 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.82646.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.82646.log.gz 2024-09-12T14:19:32.066 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.117977.log 2024-09-12T14:19:32.066 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.79895.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.79895.log.gz 2024-09-12T14:19:32.066 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.95323.log 2024-09-12T14:19:32.067 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.117977.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.117977.log.gz 2024-09-12T14:19:32.067 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.53671.log 2024-09-12T14:19:32.068 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.95323.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.95323.log.gz 2024-09-12T14:19:32.068 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.39217.log 2024-09-12T14:19:32.069 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.53671.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.53671.log.gz 2024-09-12T14:19:32.069 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.101153.log 2024-09-12T14:19:32.070 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.39217.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.129564.log 2024-09-12T14:19:32.070 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.39217.log.gz 2024-09-12T14:19:32.071 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.101153.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.145760.log 2024-09-12T14:19:32.071 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.101153.log.gz 2024-09-12T14:19:32.071 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.129564.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.129564.log.gz 2024-09-12T14:19:32.071 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.91055.log 2024-09-12T14:19:32.072 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.145760.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.145760.log.gz 2024-09-12T14:19:32.072 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.69375.log 2024-09-12T14:19:32.073 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.91055.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.108353.log 2024-09-12T14:19:32.073 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.91055.log.gz 2024-09-12T14:19:32.074 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.69375.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.69375.log.gz 2024-09-12T14:19:32.074 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.124257.log 2024-09-12T14:19:32.074 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.108353.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.55659.log 2024-09-12T14:19:32.074 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.108353.log.gz 2024-09-12T14:19:32.075 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.124257.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.124257.log.gz 2024-09-12T14:19:32.075 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.91012.log 2024-09-12T14:19:32.076 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.55659.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.55659.log.gzgzip 2024-09-12T14:19:32.076 INFO:teuthology.orchestra.run.smithi130.stderr: -5 --verbose -- /var/log/ceph/cluster1-client.mirror.63483.log 2024-09-12T14:19:32.077 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.91012.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.91012.log.gz 2024-09-12T14:19:32.077 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.47715.log 2024-09-12T14:19:32.078 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.63483.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.63483.log.gz 2024-09-12T14:19:32.078 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.36988.log 2024-09-12T14:19:32.079 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.47715.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.47715.log.gz 2024-09-12T14:19:32.079 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.65624.log 2024-09-12T14:19:32.079 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.36988.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.36988.log.gz 2024-09-12T14:19:32.079 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.146024.log 2024-09-12T14:19:32.080 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.65624.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.65624.log.gz 2024-09-12T14:19:32.080 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.35695.log 2024-09-12T14:19:32.081 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.146024.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.98860.log 2024-09-12T14:19:32.081 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.146024.log.gz 2024-09-12T14:19:32.082 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.35695.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.35695.log.gz 2024-09-12T14:19:32.082 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.84689.log 2024-09-12T14:19:32.082 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.98860.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.98860.log.gz 2024-09-12T14:19:32.083 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.63735.log 2024-09-12T14:19:32.083 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.84689.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.84689.log.gz 2024-09-12T14:19:32.084 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.125116.log 2024-09-12T14:19:32.084 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.63735.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.63735.log.gz 2024-09-12T14:19:32.085 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.62391.log 2024-09-12T14:19:32.085 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.125116.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.125116.log.gz 2024-09-12T14:19:32.086 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.134551.log 2024-09-12T14:19:32.086 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.62391.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.62391.log.gz 2024-09-12T14:19:32.086 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.136779.log 2024-09-12T14:19:32.087 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.134551.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.134551.log.gz 2024-09-12T14:19:32.087 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.120649.log 2024-09-12T14:19:32.088 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.136779.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.106404.log 2024-09-12T14:19:32.088 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.136779.log.gz 2024-09-12T14:19:32.089 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.120649.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.120649.log.gz 2024-09-12T14:19:32.089 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.63902.log 2024-09-12T14:19:32.089 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.106404.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.106404.log.gz 2024-09-12T14:19:32.090 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.150452.log 2024-09-12T14:19:32.090 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.63902.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.40119.log 2024-09-12T14:19:32.090 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.63902.log.gz 2024-09-12T14:19:32.091 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.150452.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.120714.log 0.0% 2024-09-12T14:19:32.091 INFO:teuthology.orchestra.run.smithi130.stderr: -- replaced with /var/log/ceph/cluster1-client.admin.150452.log.gz 2024-09-12T14:19:32.092 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.40119.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.40119.log.gz 2024-09-12T14:19:32.092 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.80101.log 2024-09-12T14:19:32.092 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.120714.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.120714.log.gz 2024-09-12T14:19:32.093 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.115433.log 2024-09-12T14:19:32.094 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.80101.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.80101.log.gz 2024-09-12T14:19:32.094 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.134004.log 2024-09-12T14:19:32.094 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.115433.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.115433.log.gz 2024-09-12T14:19:32.095 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.65665.log 2024-09-12T14:19:32.095 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.134004.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.134004.log.gz 2024-09-12T14:19:32.095 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.137053.log 2024-09-12T14:19:32.096 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.65665.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.52119.log 2024-09-12T14:19:32.096 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.65665.log.gz 2024-09-12T14:19:32.097 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.137053.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.54631.log 2024-09-12T14:19:32.097 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.137053.log.gz 2024-09-12T14:19:32.098 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.52119.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.52119.log.gz 2024-09-12T14:19:32.098 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.103967.log 2024-09-12T14:19:32.099 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.54631.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.54631.log.gz 2024-09-12T14:19:32.099 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.51005.log 2024-09-12T14:19:32.099 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.103967.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.103967.log.gz 2024-09-12T14:19:32.099 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.81021.log 2024-09-12T14:19:32.100 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.51005.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.51005.log.gz 2024-09-12T14:19:32.100 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.48212.log 2024-09-12T14:19:32.101 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.81021.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.81021.log.gz 2024-09-12T14:19:32.101 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.77679.log 2024-09-12T14:19:32.102 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.48212.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.48212.log.gz 2024-09-12T14:19:32.102 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.100894.log 2024-09-12T14:19:32.103 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.77679.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.77679.log.gz 2024-09-12T14:19:32.103 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.37553.log 2024-09-12T14:19:32.104 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.100894.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.100894.log.gz 2024-09-12T14:19:32.104 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.142443.log 2024-09-12T14:19:32.105 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.37553.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.92924.log 2024-09-12T14:19:32.105 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.37553.log.gz 2024-09-12T14:19:32.106 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.142443.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.142443.log.gz 2024-09-12T14:19:32.106 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.54732.log 2024-09-12T14:19:32.106 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.92924.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.92924.log.gz 2024-09-12T14:19:32.106 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.107651.log 2024-09-12T14:19:32.107 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.54732.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.101463.log 2024-09-12T14:19:32.107 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.54732.log.gz 2024-09-12T14:19:32.108 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.107651.log: gzip 0.0% -5 --verbose -- replaced with /var/log/ceph/cluster1-client.mirror.107651.log.gz -- 2024-09-12T14:19:32.108 INFO:teuthology.orchestra.run.smithi130.stderr: /var/log/ceph/cluster2-client.admin.80773.log 2024-09-12T14:19:32.108 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.101463.log: gzip -5 --verbose 0.0% -- /var/log/ceph/cluster1-client.admin.57949.log -- replaced with /var/log/ceph/cluster1-client.mirror.101463.log.gz 2024-09-12T14:19:32.108 INFO:teuthology.orchestra.run.smithi130.stderr: 2024-09-12T14:19:32.109 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.80773.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.80773.log.gz 2024-09-12T14:19:32.109 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.141831.log 2024-09-12T14:19:32.110 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.57949.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.57949.log.gz 2024-09-12T14:19:32.110 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.104166.log 2024-09-12T14:19:32.111 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.141831.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.141831.log.gz 2024-09-12T14:19:32.111 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.149595.log 2024-09-12T14:19:32.112 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.104166.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.104166.log.gz 2024-09-12T14:19:32.112 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.94732.log 2024-09-12T14:19:32.113 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.149595.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.149595.log.gz 2024-09-12T14:19:32.113 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.112303.log 2024-09-12T14:19:32.114 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.94732.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.94732.log.gz 2024-09-12T14:19:32.114 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.49117.log 2024-09-12T14:19:32.115 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.112303.log: gzip 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.112303.log.gz -5 2024-09-12T14:19:32.115 INFO:teuthology.orchestra.run.smithi130.stderr: --verbose -- /var/log/ceph/cluster1-client.mirror.142816.log 2024-09-12T14:19:32.116 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.49117.log: gzip -5 --verbose -- 0.0% /var/log/ceph/cluster1-client.mirror.94445.log 2024-09-12T14:19:32.116 INFO:teuthology.orchestra.run.smithi130.stderr: -- replaced with /var/log/ceph/cluster1-client.mirror.49117.log.gz 2024-09-12T14:19:32.116 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.142816.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.146198.log 2024-09-12T14:19:32.116 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.142816.log.gz 2024-09-12T14:19:32.117 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.94445.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.94445.log.gz 2024-09-12T14:19:32.117 INFO:teuthology.orchestra.run.smithi130.stderr: 82.4% -- replaced with /var/log/ceph/cluster1-client.mirror.0.34559.log.gz 2024-09-12T14:19:32.118 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.149810.log 2024-09-12T14:19:32.118 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.146198.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.146198.log.gz 2024-09-12T14:19:32.118 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.152588.log 2024-09-12T14:19:32.119 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.149810.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.149810.log.gz 2024-09-12T14:19:32.119 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.107485.log 2024-09-12T14:19:32.120 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.152588.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.105080.log 2024-09-12T14:19:32.120 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.152588.log.gz 2024-09-12T14:19:32.121 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.107485.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.107485.log.gz 2024-09-12T14:19:32.121 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.61528.log 2024-09-12T14:19:32.122 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.105080.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.105080.log.gz 2024-09-12T14:19:32.122 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.73860.log 2024-09-12T14:19:32.122 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.61528.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.61528.log.gz 2024-09-12T14:19:32.123 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.84204.log 2024-09-12T14:19:32.124 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.73860.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.73860.log.gz 2024-09-12T14:19:32.124 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.147369.log 2024-09-12T14:19:32.124 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.84204.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.84204.log.gz 2024-09-12T14:19:32.124 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.84331.log 2024-09-12T14:19:32.125 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.147369.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.141623.log 2024-09-12T14:19:32.125 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.147369.log.gz 2024-09-12T14:19:32.126 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.84331.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.84331.log.gz 2024-09-12T14:19:32.126 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.61320.log 2024-09-12T14:19:32.127 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.141623.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.141623.log.gz 2024-09-12T14:19:32.127 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.141872.log 2024-09-12T14:19:32.128 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.61320.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.61320.log.gzgzip 2024-09-12T14:19:32.128 INFO:teuthology.orchestra.run.smithi130.stderr: -5 --verbose -- /var/log/ceph/cluster1-client.mirror.72916.log 2024-09-12T14:19:32.128 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.141872.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.141872.log.gz 2024-09-12T14:19:32.129 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.84852.log 2024-09-12T14:19:32.129 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.72916.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.72916.log.gz 2024-09-12T14:19:32.129 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.68414.log 2024-09-12T14:19:32.130 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.84852.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.84852.log.gz 2024-09-12T14:19:32.130 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.35380.log 2024-09-12T14:19:32.131 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.68414.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.68414.log.gz 2024-09-12T14:19:32.131 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.40118.log 2024-09-12T14:19:32.131 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.35380.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.35380.log.gz 2024-09-12T14:19:32.132 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.64511.log 2024-09-12T14:19:32.132 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.40118.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.40118.log.gz 2024-09-12T14:19:32.132 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.46542.log 2024-09-12T14:19:32.133 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.64511.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.64511.log.gz 2024-09-12T14:19:32.133 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.36565.log 2024-09-12T14:19:32.134 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.46542.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.122969.log 2024-09-12T14:19:32.134 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.46542.log.gz 2024-09-12T14:19:32.135 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.36565.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.36565.log.gz 2024-09-12T14:19:32.135 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.68640.log 2024-09-12T14:19:32.135 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.122969.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.122969.log.gz 2024-09-12T14:19:32.136 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.107650.log 2024-09-12T14:19:32.136 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.68640.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.68640.log.gz 2024-09-12T14:19:32.137 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.118563.log 2024-09-12T14:19:32.137 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.107650.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.107650.log.gz 2024-09-12T14:19:32.137 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.140854.log 2024-09-12T14:19:32.138 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.118563.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.118563.log.gz 2024-09-12T14:19:32.138 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.91137.log 2024-09-12T14:19:32.139 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.140854.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.140854.log.gz 2024-09-12T14:19:32.139 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.88510.log 2024-09-12T14:19:32.139 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.91137.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.91137.log.gz 2024-09-12T14:19:32.139 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.149572.log 2024-09-12T14:19:32.140 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.88510.log: 0.0%gzip -- replaced with /var/log/ceph/cluster1-client.mirror.88510.log.gz -5 2024-09-12T14:19:32.140 INFO:teuthology.orchestra.run.smithi130.stderr: --verbose -- /var/log/ceph/cluster1-client.mirror.132787.log 2024-09-12T14:19:32.141 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.149572.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.149572.log.gz 2024-09-12T14:19:32.141 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.125674.log 2024-09-12T14:19:32.142 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.132787.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.111060.log 2024-09-12T14:19:32.142 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.132787.log.gz 2024-09-12T14:19:32.142 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.125674.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.125674.log.gz 2024-09-12T14:19:32.143 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.53233.log 2024-09-12T14:19:32.143 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.111060.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.111060.log.gz 2024-09-12T14:19:32.143 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.39738.log 2024-09-12T14:19:32.144 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.53233.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.58291.log 2024-09-12T14:19:32.144 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.53233.log.gz 2024-09-12T14:19:32.145 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.39738.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.39738.log.gz 2024-09-12T14:19:32.145 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.101424.log 2024-09-12T14:19:32.146 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.58291.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.58291.log.gz 2024-09-12T14:19:32.146 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.81286.log 2024-09-12T14:19:32.147 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.101424.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.101424.log.gz 2024-09-12T14:19:32.147 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.148860.log 2024-09-12T14:19:32.148 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.81286.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.81286.log.gz 2024-09-12T14:19:32.148 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.113645.log 2024-09-12T14:19:32.149 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.148860.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.84582.log 2024-09-12T14:19:32.149 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.148860.log.gz 2024-09-12T14:19:32.150 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.113645.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.113645.log.gz 2024-09-12T14:19:32.150 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.83873.log 2024-09-12T14:19:32.150 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.84582.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.84582.log.gz 2024-09-12T14:19:32.151 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.89831.log 2024-09-12T14:19:32.151 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.83873.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.76398.log 2024-09-12T14:19:32.151 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.83873.log.gz 2024-09-12T14:19:32.152 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.89831.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.89831.log.gz 2024-09-12T14:19:32.152 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.57835.log 2024-09-12T14:19:32.153 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.76398.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.76398.log.gz 2024-09-12T14:19:32.153 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.136610.log 2024-09-12T14:19:32.153 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.57835.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.57835.log.gz 2024-09-12T14:19:32.154 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.131865.log 2024-09-12T14:19:32.154 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.136610.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.136610.log.gz 2024-09-12T14:19:32.154 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.45445.log 2024-09-12T14:19:32.155 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.131865.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.131865.log.gz 2024-09-12T14:19:32.155 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.148968.log 2024-09-12T14:19:32.156 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.45445.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.45445.log.gz 2024-09-12T14:19:32.156 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.42051.log 2024-09-12T14:19:32.157 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.148968.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.148968.log.gz 2024-09-12T14:19:32.157 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.129222.log 2024-09-12T14:19:32.158 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.42051.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.42051.log.gz 2024-09-12T14:19:32.158 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.81859.log 2024-09-12T14:19:32.159 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.129222.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.129222.log.gz 2024-09-12T14:19:32.159 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.93570.log 2024-09-12T14:19:32.159 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.81859.log: gzip 0.0% -5 --verbose -- -- replaced with /var/log/ceph/cluster1-client.admin.81859.log.gz /var/log/ceph/cluster1-client.mirror.36201.log 2024-09-12T14:19:32.159 INFO:teuthology.orchestra.run.smithi130.stderr: 2024-09-12T14:19:32.160 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.93570.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.93570.log.gz 2024-09-12T14:19:32.160 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.147309.log 2024-09-12T14:19:32.161 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.36201.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.36201.log.gz 2024-09-12T14:19:32.161 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.92612.log 2024-09-12T14:19:32.162 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.147309.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.147309.log.gz 2024-09-12T14:19:32.162 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.135519.log 2024-09-12T14:19:32.163 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.92612.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.92612.log.gz 2024-09-12T14:19:32.163 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.144126.log 2024-09-12T14:19:32.163 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.135519.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.135519.log.gz 2024-09-12T14:19:32.164 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.142701.log 2024-09-12T14:19:32.165 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.144126.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.144126.log.gz 2024-09-12T14:19:32.165 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.34853.log 2024-09-12T14:19:32.166 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.142701.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.146824.log 2024-09-12T14:19:32.166 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.142701.log.gz 2024-09-12T14:19:32.166 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.34853.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.34853.log.gz 2024-09-12T14:19:32.167 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.45227.log 2024-09-12T14:19:32.167 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.146824.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.146824.log.gz 2024-09-12T14:19:32.167 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.137772.log 2024-09-12T14:19:32.168 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.45227.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.115219.log 2024-09-12T14:19:32.168 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.45227.log.gz 2024-09-12T14:19:32.169 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.137772.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.137772.log.gz 2024-09-12T14:19:32.169 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.29063.log 2024-09-12T14:19:32.170 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.115219.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.115219.log.gz 2024-09-12T14:19:32.170 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.140335.log 2024-09-12T14:19:32.170 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.29063.log: gzip 0.0% -5 -- replaced with /var/log/ceph/cluster2-client.admin.29063.log.gz --verbose 2024-09-12T14:19:32.170 INFO:teuthology.orchestra.run.smithi130.stderr: -- /var/log/ceph/cluster1-client.mirror.54079.log 2024-09-12T14:19:32.171 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.140335.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.81156.log 2024-09-12T14:19:32.171 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.140335.log.gz 2024-09-12T14:19:32.172 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.54079.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.54079.log.gz 2024-09-12T14:19:32.172 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.151300.log 2024-09-12T14:19:32.173 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.81156.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.81156.log.gz 2024-09-12T14:19:32.173 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.62308.log 2024-09-12T14:19:32.174 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.151300.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.151300.log.gz 2024-09-12T14:19:32.174 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.114150.log 2024-09-12T14:19:32.174 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.62308.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.62308.log.gz 2024-09-12T14:19:32.174 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.150322.log 2024-09-12T14:19:32.175 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.114150.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.114150.log.gz 2024-09-12T14:19:32.175 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.84969.log 2024-09-12T14:19:32.176 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.150322.log: gzip -5 0.0% --verbose -- -- replaced with /var/log/ceph/cluster1-client.admin.150322.log.gz 2024-09-12T14:19:32.176 INFO:teuthology.orchestra.run.smithi130.stderr: /var/log/ceph/cluster1-client.mirror.48348.log 2024-09-12T14:19:32.177 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.84969.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.65391.log 2024-09-12T14:19:32.177 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.84969.log.gz 2024-09-12T14:19:32.177 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.48348.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.48348.log.gz 2024-09-12T14:19:32.178 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.36854.log 2024-09-12T14:19:32.178 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.65391.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.65391.log.gz 2024-09-12T14:19:32.178 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.59711.log 2024-09-12T14:19:32.179 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.36854.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.36854.log.gz 2024-09-12T14:19:32.179 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.51700.log 2024-09-12T14:19:32.180 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.59711.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.38792.log 2024-09-12T14:19:32.180 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.59711.log.gz 2024-09-12T14:19:32.180 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.51700.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.51700.log.gz 2024-09-12T14:19:32.181 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.152642.log 2024-09-12T14:19:32.182 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.38792.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.38792.log.gz 2024-09-12T14:19:32.182 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.87312.log 2024-09-12T14:19:32.183 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.152642.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.152642.log.gz 2024-09-12T14:19:32.183 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.148716.log 2024-09-12T14:19:32.183 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.87312.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.87312.log.gz 2024-09-12T14:19:32.184 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.149199.log 2024-09-12T14:19:32.184 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.148716.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.148716.log.gz 2024-09-12T14:19:32.184 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.147538.log 2024-09-12T14:19:32.185 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.149199.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.149199.log.gz 2024-09-12T14:19:32.185 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.62475.log 2024-09-12T14:19:32.186 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.147538.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.141121.log 2024-09-12T14:19:32.186 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.147538.log.gz 2024-09-12T14:19:32.187 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.62475.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.35006.log 2024-09-12T14:19:32.187 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.62475.log.gz 2024-09-12T14:19:32.187 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.141121.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.141121.log.gz 2024-09-12T14:19:32.188 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.48695.log 2024-09-12T14:19:32.188 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.35006.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.35006.log.gz 2024-09-12T14:19:32.188 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.112623.log 2024-09-12T14:19:32.189 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.48695.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.48695.log.gz 2024-09-12T14:19:32.190 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.100098.log 2024-09-12T14:19:32.190 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.112623.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.112623.log.gz 2024-09-12T14:19:32.190 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.110307.log 2024-09-12T14:19:32.190 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.100098.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.100098.log.gz 2024-09-12T14:19:32.191 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.78016.log 2024-09-12T14:19:32.191 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.110307.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.110307.log.gz 2024-09-12T14:19:32.191 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.93953.log 2024-09-12T14:19:32.192 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.78016.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.78016.log.gz 2024-09-12T14:19:32.192 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.102703.log 2024-09-12T14:19:32.193 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.93953.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.93953.log.gz 2024-09-12T14:19:32.193 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.71754.log 2024-09-12T14:19:32.194 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.102703.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.102703.log.gz 2024-09-12T14:19:32.194 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.76528.log 2024-09-12T14:19:32.195 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.71754.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.71754.log.gz 2024-09-12T14:19:32.195 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.45139.log 2024-09-12T14:19:32.195 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.76528.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.76528.log.gz 2024-09-12T14:19:32.195 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.139542.log 2024-09-12T14:19:32.196 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.45139.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.45139.log.gz 2024-09-12T14:19:32.196 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.125048.log 2024-09-12T14:19:32.197 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.139542.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.139542.log.gz 2024-09-12T14:19:32.197 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.65394.log 2024-09-12T14:19:32.198 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.125048.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.125048.log.gz 2024-09-12T14:19:32.198 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.121970.log 2024-09-12T14:19:32.198 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.65394.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.65394.log.gz 2024-09-12T14:19:32.198 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.62664.log 2024-09-12T14:19:32.199 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.121970.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.121970.log.gz 2024-09-12T14:19:32.199 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.95276.log 2024-09-12T14:19:32.200 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.62664.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.62664.log.gz 2024-09-12T14:19:32.200 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.100983.log 2024-09-12T14:19:32.201 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.95276.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.95276.log.gz 2024-09-12T14:19:32.201 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.111103.log 2024-09-12T14:19:32.202 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.100983.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.100983.log.gz 2024-09-12T14:19:32.202 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.97327.log 2024-09-12T14:19:32.203 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.111103.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.111103.log.gzgzip 2024-09-12T14:19:32.203 INFO:teuthology.orchestra.run.smithi130.stderr: -5 --verbose -- /var/log/ceph/cluster1-client.mirror.96969.log 2024-09-12T14:19:32.204 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.97327.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.122646.log 2024-09-12T14:19:32.204 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.97327.log.gz 2024-09-12T14:19:32.204 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.96969.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.96969.log.gz 2024-09-12T14:19:32.205 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.50501.log 2024-09-12T14:19:32.205 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.122646.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.85612.log 2024-09-12T14:19:32.205 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.122646.log.gz 2024-09-12T14:19:32.206 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.50501.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.50501.log.gz 2024-09-12T14:19:32.206 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.104059.log 2024-09-12T14:19:32.207 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.85612.log: gzip -5 --verbose -- 0.0% /var/log/ceph/cluster1-client.mirror.136589.log -- replaced with /var/log/ceph/cluster1-client.mirror.85612.log.gz 2024-09-12T14:19:32.207 INFO:teuthology.orchestra.run.smithi130.stderr: 2024-09-12T14:19:32.208 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.104059.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.104059.log.gz 2024-09-12T14:19:32.208 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.128418.log 2024-09-12T14:19:32.209 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.136589.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.124731.log 2024-09-12T14:19:32.209 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.136589.log.gz 2024-09-12T14:19:32.209 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.128418.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.128418.log.gz 2024-09-12T14:19:32.209 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.113038.log 2024-09-12T14:19:32.210 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.124731.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.85484.log 2024-09-12T14:19:32.210 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.124731.log.gz 2024-09-12T14:19:32.211 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.113038.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.113038.log.gz 2024-09-12T14:19:32.211 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.76843.log 2024-09-12T14:19:32.212 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.85484.log: 0.0%gzip -- replaced with /var/log/ceph/cluster1-client.mirror.85484.log.gz -5 2024-09-12T14:19:32.212 INFO:teuthology.orchestra.run.smithi130.stderr: --verbose -- /var/log/ceph/cluster1-client.mirror.50669.log 2024-09-12T14:19:32.212 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.76843.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.76843.log.gz 2024-09-12T14:19:32.212 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.144293.log 2024-09-12T14:19:32.213 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.50669.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.119181.log 2024-09-12T14:19:32.213 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.50669.log.gz 2024-09-12T14:19:32.214 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.144293.log: 0.0%gzip -- replaced with /var/log/ceph/cluster1-client.mirror.144293.log.gz -5 2024-09-12T14:19:32.214 INFO:teuthology.orchestra.run.smithi130.stderr: --verbose -- /var/log/ceph/cluster1-client.mirror.107105.log 2024-09-12T14:19:32.215 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.119181.log: gzip 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.119181.log.gz -5 2024-09-12T14:19:32.215 INFO:teuthology.orchestra.run.smithi130.stderr: --verbose -- /var/log/ceph/cluster1-client.mirror.86893.log 2024-09-12T14:19:32.215 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.107105.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.107105.log.gz 2024-09-12T14:19:32.216 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.82516.log 2024-09-12T14:19:32.216 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.86893.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.86893.log.gz 2024-09-12T14:19:32.216 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.102834.log 2024-09-12T14:19:32.217 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.82516.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.82516.log.gz 2024-09-12T14:19:32.217 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.152225.log 2024-09-12T14:19:32.218 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.102834.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.102834.log.gz 2024-09-12T14:19:32.218 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.45811.log 2024-09-12T14:19:32.219 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.152225.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.152225.log.gz 2024-09-12T14:19:32.219 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.112091.log 2024-09-12T14:19:32.220 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.45811.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.45811.log.gz 2024-09-12T14:19:32.220 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.77871.log 2024-09-12T14:19:32.221 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.112091.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.132621.log 2024-09-12T14:19:32.221 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.112091.log.gz 2024-09-12T14:19:32.222 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.77871.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.77871.log.gz 2024-09-12T14:19:32.222 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.96589.log 2024-09-12T14:19:32.222 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.132621.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.132621.log.gz 2024-09-12T14:19:32.222 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.101803.log 2024-09-12T14:19:32.223 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.96589.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.137054.log 2024-09-12T14:19:32.223 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.96589.log.gz 2024-09-12T14:19:32.224 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.101803.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.101803.log.gz 2024-09-12T14:19:32.224 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.94839.log 2024-09-12T14:19:32.225 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.137054.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.137054.log.gz 2024-09-12T14:19:32.225 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.42659.log 2024-09-12T14:19:32.225 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.94839.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.94839.log.gz 2024-09-12T14:19:32.226 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.111629.log 2024-09-12T14:19:32.226 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.42659.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.42659.log.gz 2024-09-12T14:19:32.226 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.123073.log 2024-09-12T14:19:32.227 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.111629.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.111629.log.gz 2024-09-12T14:19:32.227 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.121489.log 2024-09-12T14:19:32.228 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.123073.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.123073.log.gz 2024-09-12T14:19:32.228 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.84688.log 2024-09-12T14:19:32.229 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.121489.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.121489.log.gz 2024-09-12T14:19:32.229 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.87544.log 2024-09-12T14:19:32.230 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.84688.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.112512.log 2024-09-12T14:19:32.230 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.84688.log.gz 2024-09-12T14:19:32.231 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.87544.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.87544.log.gz 2024-09-12T14:19:32.231 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.143007.log 2024-09-12T14:19:32.232 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.112512.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.112512.log.gz 2024-09-12T14:19:32.232 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.75055.log 2024-09-12T14:19:32.233 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.143007.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.143007.log.gz 2024-09-12T14:19:32.233 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.130907.log 2024-09-12T14:19:32.234 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.75055.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.75055.log.gz 2024-09-12T14:19:32.234 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.107260.log 2024-09-12T14:19:32.234 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.130907.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.130907.log.gz 2024-09-12T14:19:32.234 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.87460.log 2024-09-12T14:19:32.236 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.107260.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.107260.log.gz 2024-09-12T14:19:32.236 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.119069.log 2024-09-12T14:19:32.236 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.87460.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.87460.log.gz 2024-09-12T14:19:32.236 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.103768.log 2024-09-12T14:19:32.237 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.119069.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.77574.log 2024-09-12T14:19:32.237 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.119069.log.gz 2024-09-12T14:19:32.238 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.103768.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.103768.log.gzgzip 2024-09-12T14:19:32.238 INFO:teuthology.orchestra.run.smithi130.stderr: -5 --verbose -- /var/log/ceph/cluster1-client.mirror.135962.log 2024-09-12T14:19:32.238 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.77574.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.77574.log.gz 2024-09-12T14:19:32.239 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.78335.log 2024-09-12T14:19:32.239 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.135962.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.54072.log 2024-09-12T14:19:32.239 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.135962.log.gz 2024-09-12T14:19:32.240 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.78335.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.78335.log.gz 2024-09-12T14:19:32.240 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.112386.log 2024-09-12T14:19:32.241 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.54072.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.51046.log 2024-09-12T14:19:32.241 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.54072.log.gz 2024-09-12T14:19:32.242 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.112386.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.112386.log.gz 2024-09-12T14:19:32.242 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.117924.log 2024-09-12T14:19:32.243 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.51046.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.99764.log 2024-09-12T14:19:32.243 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.51046.log.gz 2024-09-12T14:19:32.243 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.117924.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.117924.log.gz 2024-09-12T14:19:32.244 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.142745.log 2024-09-12T14:19:32.244 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.99764.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.99764.log.gz 2024-09-12T14:19:32.245 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.86155.log 2024-09-12T14:19:32.245 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.142745.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.142745.log.gz 2024-09-12T14:19:32.245 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.43298.log 2024-09-12T14:19:32.246 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.86155.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.86155.log.gz 2024-09-12T14:19:32.246 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.88468.log 2024-09-12T14:19:32.247 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.43298.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.85043.log 2024-09-12T14:19:32.247 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.43298.log.gz 2024-09-12T14:19:32.248 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.88468.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.56500.log 2024-09-12T14:19:32.248 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.88468.log.gz 2024-09-12T14:19:32.248 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.85043.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.85043.log.gz 2024-09-12T14:19:32.249 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.100227.log 2024-09-12T14:19:32.249 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.56500.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.143581.log 2024-09-12T14:19:32.249 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.56500.log.gz 2024-09-12T14:19:32.250 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.100227.log: gzip -5 --verbose -- 0.0% /var/log/ceph/cluster1-client.mirror.101955.log 2024-09-12T14:19:32.250 INFO:teuthology.orchestra.run.smithi130.stderr: -- replaced with /var/log/ceph/cluster1-client.mirror.100227.log.gz 2024-09-12T14:19:32.251 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.143581.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.143581.log.gz 2024-09-12T14:19:32.251 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.146928.log 2024-09-12T14:19:32.252 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.101955.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.101955.log.gz 2024-09-12T14:19:32.252 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.153691.log 2024-09-12T14:19:32.253 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.146928.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.146928.log.gz 2024-09-12T14:19:32.253 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.47868.log 2024-09-12T14:19:32.253 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.153691.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.153691.log.gz 2024-09-12T14:19:32.254 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.55437.log 2024-09-12T14:19:32.254 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.47868.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.47868.log.gz 2024-09-12T14:19:32.254 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.141261.log 2024-09-12T14:19:32.255 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.55437.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.55437.log.gz 2024-09-12T14:19:32.255 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.101598.log 2024-09-12T14:19:32.256 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.141261.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.126766.log 2024-09-12T14:19:32.256 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.141261.log.gz 2024-09-12T14:19:32.257 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.101598.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.101598.log.gz 2024-09-12T14:19:32.257 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.123745.log 2024-09-12T14:19:32.258 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.126766.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.126766.log.gz 2024-09-12T14:19:32.258 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.112615.log 2024-09-12T14:19:32.259 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.123745.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.123745.log.gz 2024-09-12T14:19:32.259 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.56565.log 2024-09-12T14:19:32.259 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.112615.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.112615.log.gz 2024-09-12T14:19:32.260 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.105839.log 2024-09-12T14:19:32.260 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.56565.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.56565.log.gz 2024-09-12T14:19:32.260 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.43444.log 2024-09-12T14:19:32.261 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.105839.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.149410.log 2024-09-12T14:19:32.261 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.105839.log.gz 2024-09-12T14:19:32.262 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.43444.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.43444.log.gz 2024-09-12T14:19:32.262 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.110620.log 2024-09-12T14:19:32.262 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.149410.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.149410.log.gz 2024-09-12T14:19:32.263 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.130621.log 2024-09-12T14:19:32.263 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.110620.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.110620.log.gz 2024-09-12T14:19:32.264 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.77515.log 2024-09-12T14:19:32.264 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.130621.log: 0.0%gzip -- replaced with /var/log/ceph/cluster1-client.mirror.130621.log.gz -5 2024-09-12T14:19:32.264 INFO:teuthology.orchestra.run.smithi130.stderr: --verbose -- /var/log/ceph/cluster1-client.mirror.142022.log 2024-09-12T14:19:32.265 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.77515.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.117000.log 2024-09-12T14:19:32.265 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.77515.log.gz 2024-09-12T14:19:32.266 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.142022.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.142022.log.gz 2024-09-12T14:19:32.266 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.97030.log 2024-09-12T14:19:32.267 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.117000.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.87166.log 2024-09-12T14:19:32.267 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.117000.log.gz 2024-09-12T14:19:32.267 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.97030.log: gzip 0.0% -5 --verbose -- replaced with /var/log/ceph/cluster1-client.mirror.97030.log.gz -- 2024-09-12T14:19:32.268 INFO:teuthology.orchestra.run.smithi130.stderr: /var/log/ceph/cluster1-client.mirror.121049.log 2024-09-12T14:19:32.268 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.87166.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.87166.log.gz 2024-09-12T14:19:32.268 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.133296.log 2024-09-12T14:19:32.269 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.121049.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.145503.log 2024-09-12T14:19:32.269 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.121049.log.gz 2024-09-12T14:19:32.270 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.133296.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.146589.log 2024-09-12T14:19:32.270 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.133296.log.gz 2024-09-12T14:19:32.271 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.145503.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.145503.log.gz 2024-09-12T14:19:32.271 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.59252.log 2024-09-12T14:19:32.271 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.146589.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.146589.log.gz 2024-09-12T14:19:32.272 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.100057.log 2024-09-12T14:19:32.272 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.59252.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.59252.log.gz 2024-09-12T14:19:32.272 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.65541.log 2024-09-12T14:19:32.273 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.100057.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.100057.log.gz 2024-09-12T14:19:32.273 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.80030.log 2024-09-12T14:19:32.274 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.65541.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.65541.log.gz 2024-09-12T14:19:32.274 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.117153.log 2024-09-12T14:19:32.275 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.80030.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.80030.log.gz 2024-09-12T14:19:32.275 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.64354.log 2024-09-12T14:19:32.276 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.117153.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.117153.log.gz 2024-09-12T14:19:32.276 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.135412.log 2024-09-12T14:19:32.276 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.64354.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.64354.log.gz 2024-09-12T14:19:32.276 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.118111.log 2024-09-12T14:19:32.277 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.135412.log: gzip -5 --verbose 0.0% -- /var/log/ceph/cluster1-client.admin.126764.log -- replaced with /var/log/ceph/cluster1-client.mirror.135412.log.gz 2024-09-12T14:19:32.277 INFO:teuthology.orchestra.run.smithi130.stderr: 2024-09-12T14:19:32.278 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.118111.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.118111.log.gz 2024-09-12T14:19:32.278 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.47931.log 2024-09-12T14:19:32.279 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.126764.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.126764.log.gz 2024-09-12T14:19:32.279 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-mon.a.log 2024-09-12T14:19:32.279 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.47931.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.47931.log.gz 2024-09-12T14:19:32.280 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.99576.log 2024-09-12T14:19:32.280 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-mon.a.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.109927.log 2024-09-12T14:19:32.281 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.99576.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.99576.log.gz 2024-09-12T14:19:32.281 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.60274.log 2024-09-12T14:19:32.282 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.109927.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.109927.log.gz 2024-09-12T14:19:32.282 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.128270.log 2024-09-12T14:19:32.283 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.60274.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.60274.log.gz 2024-09-12T14:19:32.283 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.138476.log 2024-09-12T14:19:32.284 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.128270.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.134993.log 2024-09-12T14:19:32.284 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.128270.log.gz 2024-09-12T14:19:32.284 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.138476.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.48845.log 2024-09-12T14:19:32.284 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.138476.log.gz 2024-09-12T14:19:32.285 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.134993.log: gzip 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.134993.log.gz -5 2024-09-12T14:19:32.285 INFO:teuthology.orchestra.run.smithi130.stderr: --verbose -- /var/log/ceph/cluster2-client.admin.140275.log 2024-09-12T14:19:32.286 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.48845.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.48845.log.gz 2024-09-12T14:19:32.286 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.108621.log 2024-09-12T14:19:32.287 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.140275.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.140275.log.gz 2024-09-12T14:19:32.287 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.87500.log 2024-09-12T14:19:32.287 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.108621.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.108621.log.gz 2024-09-12T14:19:32.287 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.99614.log 2024-09-12T14:19:32.288 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.87500.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.87500.log.gz 2024-09-12T14:19:32.288 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.40391.log 2024-09-12T14:19:32.289 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.99614.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.99614.log.gz 2024-09-12T14:19:32.289 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.90403.log 2024-09-12T14:19:32.290 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.40391.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.40391.log.gz 2024-09-12T14:19:32.290 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.56427.log 2024-09-12T14:19:32.290 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.90403.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.90403.log.gz 2024-09-12T14:19:32.291 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.94670.log 2024-09-12T14:19:32.291 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.56427.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.56427.log.gz 2024-09-12T14:19:32.292 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.62307.log 2024-09-12T14:19:32.292 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.94670.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.94670.log.gz 2024-09-12T14:19:32.292 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.124128.log 2024-09-12T14:19:32.293 INFO:teuthology.orchestra.run.smithi060.stderr: 91.6% -- replaced with /var/log/ceph/cluster1-mgr.x.log.gz 2024-09-12T14:19:32.294 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.62307.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.62307.log.gz 2024-09-12T14:19:32.294 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.80083.log 2024-09-12T14:19:32.294 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.124128.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.124128.log.gz 2024-09-12T14:19:32.294 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.96424.log 2024-09-12T14:19:32.295 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.80083.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.80083.log.gz 2024-09-12T14:19:32.295 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.83088.log 2024-09-12T14:19:32.296 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.96424.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.96424.log.gz 2024-09-12T14:19:32.296 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.62182.log 2024-09-12T14:19:32.297 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.83088.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.83088.log.gz 2024-09-12T14:19:32.297 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.133605.log 2024-09-12T14:19:32.297 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.62182.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.62182.log.gz 2024-09-12T14:19:32.297 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.75708.log 2024-09-12T14:19:32.298 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.133605.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.133605.log.gz 2024-09-12T14:19:32.298 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.34921.log 2024-09-12T14:19:32.299 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.75708.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.75708.log.gz 2024-09-12T14:19:32.299 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.75939.log 2024-09-12T14:19:32.300 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.34921.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.34921.log.gz 2024-09-12T14:19:32.300 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.106881.log 2024-09-12T14:19:32.301 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.75939.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.75939.log.gz 2024-09-12T14:19:32.301 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.55475.log 2024-09-12T14:19:32.301 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.106881.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.106881.log.gz 2024-09-12T14:19:32.301 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.38708.log 2024-09-12T14:19:32.302 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.55475.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.55475.log.gz 2024-09-12T14:19:32.302 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.137307.log 2024-09-12T14:19:32.303 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.38708.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.38708.log.gz 2024-09-12T14:19:32.303 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.82019.log 2024-09-12T14:19:32.304 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.137307.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.137307.log.gz 2024-09-12T14:19:32.304 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.52180.log 2024-09-12T14:19:32.305 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.82019.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.82019.log.gz 2024-09-12T14:19:32.305 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.40536.log 2024-09-12T14:19:32.305 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.52180.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.52180.log.gz 2024-09-12T14:19:32.305 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.140253.log 2024-09-12T14:19:32.306 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.40536.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.40536.log.gz 2024-09-12T14:19:32.306 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.48908.log 2024-09-12T14:19:32.307 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.140253.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.140253.log.gz 2024-09-12T14:19:32.307 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.62809.log 2024-09-12T14:19:32.308 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.48908.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.48908.log.gz 2024-09-12T14:19:32.308 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.90575.log 2024-09-12T14:19:32.309 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.62809.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.62809.log.gz 2024-09-12T14:19:32.309 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.99848.log 2024-09-12T14:19:32.309 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.90575.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.90575.log.gz 2024-09-12T14:19:32.310 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.29468.log 2024-09-12T14:19:32.310 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.99848.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.99848.log.gz 2024-09-12T14:19:32.310 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.34560.log 2024-09-12T14:19:32.311 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.29468.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.29468.log.gz 2024-09-12T14:19:32.311 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.109106.log 2024-09-12T14:19:32.312 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.34560.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.56893.log 2024-09-12T14:19:32.312 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.109106.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.109106.log.gz 2024-09-12T14:19:32.313 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.117220.log 2024-09-12T14:19:32.314 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.56893.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.56893.log.gz 2024-09-12T14:19:32.314 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.50373.log 2024-09-12T14:19:32.314 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.117220.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.117220.log.gz 2024-09-12T14:19:32.315 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.87480.log 2024-09-12T14:19:32.315 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.50373.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.50373.log.gz 2024-09-12T14:19:32.316 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.145923.log 2024-09-12T14:19:32.316 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.87480.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.83393.log 2024-09-12T14:19:32.316 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.87480.log.gz 2024-09-12T14:19:32.317 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.145923.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.73606.log 2024-09-12T14:19:32.317 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.145923.log.gz 2024-09-12T14:19:32.318 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.83393.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.83393.log.gz 2024-09-12T14:19:32.318 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.146616.log 2024-09-12T14:19:32.319 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.73606.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.73606.log.gz 2024-09-12T14:19:32.319 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.72747.log 2024-09-12T14:19:32.319 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.146616.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.146616.log.gzgzip 2024-09-12T14:19:32.319 INFO:teuthology.orchestra.run.smithi130.stderr: -5 --verbose -- /var/log/ceph/cluster1-client.admin.67650.log 2024-09-12T14:19:32.320 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.72747.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.72747.log.gz 2024-09-12T14:19:32.320 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.118065.log 2024-09-12T14:19:32.321 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.67650.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.67650.log.gzgzip 2024-09-12T14:19:32.321 INFO:teuthology.orchestra.run.smithi130.stderr: -5 --verbose -- /var/log/ceph/cluster2-client.mirror.117655.log 2024-09-12T14:19:32.322 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.118065.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.118065.log.gz 2024-09-12T14:19:32.322 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.35870.log 2024-09-12T14:19:32.322 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.117655.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.149766.log 2024-09-12T14:19:32.322 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.117655.log.gz 2024-09-12T14:19:32.323 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.35870.log: 89.3% -- replaced with /var/log/ceph/cluster1-client.mirror.34560.log.gz 2024-09-12T14:19:32.323 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.35870.log.gz 2024-09-12T14:19:32.323 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.111944.log 2024-09-12T14:19:32.324 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.149766.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.149766.log.gz 2024-09-12T14:19:32.324 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.81359.log 2024-09-12T14:19:32.325 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.111944.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.111944.log.gz 2024-09-12T14:19:32.325 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.34303.log 2024-09-12T14:19:32.326 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.81359.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.81359.log.gz 2024-09-12T14:19:32.326 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.53820.log 2024-09-12T14:19:32.326 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.34303.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.34303.log.gz 2024-09-12T14:19:32.327 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.51338.log 2024-09-12T14:19:32.327 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.53820.log: 0.0%gzip -- replaced with /var/log/ceph/cluster1-client.mirror.53820.log.gz 2024-09-12T14:19:32.327 INFO:teuthology.orchestra.run.smithi130.stderr: -5 --verbose -- /var/log/ceph/cluster1-client.admin.79303.log 2024-09-12T14:19:32.328 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.51338.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.51338.log.gz 2024-09-12T14:19:32.328 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.75981.log 2024-09-12T14:19:32.329 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.79303.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.71057.log 2024-09-12T14:19:32.329 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.79303.log.gz 2024-09-12T14:19:32.329 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.75981.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.75981.log.gz 2024-09-12T14:19:32.329 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.112385.log 2024-09-12T14:19:32.330 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.71057.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.107827.log 2024-09-12T14:19:32.330 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.71057.log.gz 2024-09-12T14:19:32.331 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.112385.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.112385.log.gz 2024-09-12T14:19:32.331 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.98987.log 2024-09-12T14:19:32.332 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.107827.log: gzip 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.107827.log.gz -5 2024-09-12T14:19:32.332 INFO:teuthology.orchestra.run.smithi130.stderr: --verbose -- /var/log/ceph/cluster1-client.mirror.134720.log 2024-09-12T14:19:32.332 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.98987.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.98987.log.gz 2024-09-12T14:19:32.333 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.120225.log 2024-09-12T14:19:32.334 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.134720.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.134720.log.gz 2024-09-12T14:19:32.334 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.134786.log 2024-09-12T14:19:32.334 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.120225.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.120225.log.gz 2024-09-12T14:19:32.334 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.69073.log 2024-09-12T14:19:32.335 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.134786.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.134786.log.gz 2024-09-12T14:19:32.335 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.77029.log 2024-09-12T14:19:32.336 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.69073.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.123365.log 2024-09-12T14:19:32.336 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.69073.log.gz 2024-09-12T14:19:32.337 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.77029.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.77029.log.gz 2024-09-12T14:19:32.337 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.123997.log 2024-09-12T14:19:32.338 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.123365.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.123365.log.gz 2024-09-12T14:19:32.338 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.66531.log 2024-09-12T14:19:32.338 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.123997.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.123997.log.gz 2024-09-12T14:19:32.338 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.51213.log 2024-09-12T14:19:32.339 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.66531.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.117790.log 2024-09-12T14:19:32.339 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.66531.log.gz 2024-09-12T14:19:32.340 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.51213.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.51213.log.gz 2024-09-12T14:19:32.340 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.81791.log 2024-09-12T14:19:32.341 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.117790.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.117790.log.gz 2024-09-12T14:19:32.341 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.80512.log 2024-09-12T14:19:32.342 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.81791.log: 0.0%gzip -- replaced with /var/log/ceph/cluster2-client.mirror.81791.log.gz -5 2024-09-12T14:19:32.342 INFO:teuthology.orchestra.run.smithi130.stderr: --verbose -- /var/log/ceph/cluster1-client.mirror.146970.log 2024-09-12T14:19:32.342 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.80512.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.80512.log.gz 2024-09-12T14:19:32.342 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.150430.log 2024-09-12T14:19:32.343 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.146970.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.59190.log 2024-09-12T14:19:32.343 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.146970.log.gz 2024-09-12T14:19:32.344 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.150430.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.56322.log 2024-09-12T14:19:32.344 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.150430.log.gz 2024-09-12T14:19:32.345 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.59190.log: gzip 0.0% -5 --verbose -- replaced with /var/log/ceph/cluster1-client.mirror.59190.log.gz -- 2024-09-12T14:19:32.345 INFO:teuthology.orchestra.run.smithi130.stderr: /var/log/ceph/cluster1-client.mirror.64281.log 2024-09-12T14:19:32.345 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.56322.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.56322.log.gz 2024-09-12T14:19:32.346 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.94000.log 2024-09-12T14:19:32.346 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.64281.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.64281.log.gz 2024-09-12T14:19:32.347 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.49573.log 2024-09-12T14:19:32.347 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.94000.log: gzip 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.94000.log.gz -5 2024-09-12T14:19:32.347 INFO:teuthology.orchestra.run.smithi130.stderr: --verbose -- /var/log/ceph/cluster1-client.mirror.65455.log 2024-09-12T14:19:32.348 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.49573.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.59795.log 2024-09-12T14:19:32.348 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.49573.log.gz 2024-09-12T14:19:32.349 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.65455.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.65455.log.gz 2024-09-12T14:19:32.349 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.107758.log 2024-09-12T14:19:32.350 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.59795.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.59795.log.gz 2024-09-12T14:19:32.350 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.65981.log 2024-09-12T14:19:32.351 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.107758.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.107758.log.gz 2024-09-12T14:19:32.351 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.144556.log 2024-09-12T14:19:32.352 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.65981.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.65981.log.gz 2024-09-12T14:19:32.352 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.120627.log 2024-09-12T14:19:32.352 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.144556.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.144556.log.gz 2024-09-12T14:19:32.352 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.111230.log 2024-09-12T14:19:32.353 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.120627.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.120627.log.gz 2024-09-12T14:19:32.353 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.141986.log 2024-09-12T14:19:32.354 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.111230.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.111230.log.gz 2024-09-12T14:19:32.354 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.55747.log 2024-09-12T14:19:32.355 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.141986.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.141986.log.gz 2024-09-12T14:19:32.355 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.58834.log 2024-09-12T14:19:32.356 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.55747.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.55747.log.gz 2024-09-12T14:19:32.356 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.117562.log 2024-09-12T14:19:32.356 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.58834.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.58834.log.gz 2024-09-12T14:19:32.357 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.130968.log 2024-09-12T14:19:32.357 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.117562.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.117562.log.gz 2024-09-12T14:19:32.357 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.124647.log 2024-09-12T14:19:32.358 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.130968.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.130968.log.gz 2024-09-12T14:19:32.358 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.40260.log 2024-09-12T14:19:32.359 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.124647.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.124647.log.gz 2024-09-12T14:19:32.359 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.69240.log 2024-09-12T14:19:32.360 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.40260.log: gzip 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.40260.log.gz -5 2024-09-12T14:19:32.360 INFO:teuthology.orchestra.run.smithi130.stderr: --verbose -- /var/log/ceph/cluster1-client.admin.79479.log 2024-09-12T14:19:32.361 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.69240.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.69240.log.gz 2024-09-12T14:19:32.361 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.108146.log 2024-09-12T14:19:32.362 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.79479.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.79479.log.gz 2024-09-12T14:19:32.362 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.99299.log 2024-09-12T14:19:32.362 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.108146.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.108146.log.gz 2024-09-12T14:19:32.363 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.77307.log 2024-09-12T14:19:32.363 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.99299.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.99299.log.gz 2024-09-12T14:19:32.363 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.57559.log 2024-09-12T14:19:32.364 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.77307.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.77307.log.gz 2024-09-12T14:19:32.364 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.50709.log 2024-09-12T14:19:32.365 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.57559.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.57559.log.gz 2024-09-12T14:19:32.365 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.48634.log 2024-09-12T14:19:32.366 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.50709.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.50709.log.gz 2024-09-12T14:19:32.366 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.42215.log 2024-09-12T14:19:32.366 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.48634.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.71474.log 2024-09-12T14:19:32.366 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.48634.log.gz 2024-09-12T14:19:32.367 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.42215.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.57861.log 2024-09-12T14:19:32.367 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.42215.log.gz 2024-09-12T14:19:32.368 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.71474.log: gzip 0.0% -5 -- replaced with /var/log/ceph/cluster1-client.mirror.71474.log.gz --verbose 2024-09-12T14:19:32.368 INFO:teuthology.orchestra.run.smithi130.stderr: -- /var/log/ceph/cluster2-client.mirror.80991.log 2024-09-12T14:19:32.369 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.57861.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.57861.log.gz 2024-09-12T14:19:32.369 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.79923.log 2024-09-12T14:19:32.370 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.80991.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.80991.log.gz 2024-09-12T14:19:32.370 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.98629.log 2024-09-12T14:19:32.371 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.79923.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.79923.log.gz 2024-09-12T14:19:32.371 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.50921.log 2024-09-12T14:19:32.372 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.98629.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.98629.log.gz 2024-09-12T14:19:32.372 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.82589.log 2024-09-12T14:19:32.372 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.50921.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.50921.log.gz 2024-09-12T14:19:32.372 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.101715.log 2024-09-12T14:19:32.373 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.82589.log: gzip -5 0.0% --verbose -- replaced with /var/log/ceph/cluster2-client.mirror.82589.log.gz -- 2024-09-12T14:19:32.373 INFO:teuthology.orchestra.run.smithi130.stderr: /var/log/ceph/cluster1-client.mirror.41039.log 2024-09-12T14:19:32.374 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.101715.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.101715.log.gz 2024-09-12T14:19:32.375 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.136569.log 2024-09-12T14:19:32.375 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.41039.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.41039.log.gz 2024-09-12T14:19:32.375 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.125007.log 2024-09-12T14:19:32.376 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.136569.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.136569.log.gz 2024-09-12T14:19:32.376 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.152629.log 2024-09-12T14:19:32.376 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.125007.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.125007.log.gz 2024-09-12T14:19:32.376 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.34972.log 2024-09-12T14:19:32.377 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.152629.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.152629.log.gz 2024-09-12T14:19:32.377 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.57729.log 2024-09-12T14:19:32.378 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.34972.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.34972.log.gz 2024-09-12T14:19:32.378 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.71627.log 2024-09-12T14:19:32.378 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.57729.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.57729.log.gz 2024-09-12T14:19:32.379 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.140841.log 2024-09-12T14:19:32.379 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.71627.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.71627.log.gz 2024-09-12T14:19:32.379 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.96066.log 2024-09-12T14:19:32.380 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.140841.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.140841.log.gz 2024-09-12T14:19:32.380 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.123494.log 2024-09-12T14:19:32.382 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.96066.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.96066.log.gz 2024-09-12T14:19:32.382 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.51676.log 2024-09-12T14:19:32.382 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.123494.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.123494.log.gz 2024-09-12T14:19:32.382 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.137138.log 2024-09-12T14:19:32.383 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.51676.log: 0.0%gzip -- replaced with /var/log/ceph/cluster1-client.mirror.51676.log.gz 2024-09-12T14:19:32.383 INFO:teuthology.orchestra.run.smithi130.stderr: -5 --verbose -- /var/log/ceph/cluster1-client.mirror.97283.log 2024-09-12T14:19:32.383 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.137138.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.137138.log.gz 2024-09-12T14:19:32.383 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.151394.log 2024-09-12T14:19:32.384 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.97283.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.97283.log.gz 2024-09-12T14:19:32.384 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.151083.log 2024-09-12T14:19:32.385 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.151394.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.151394.log.gz 2024-09-12T14:19:32.385 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.148610.log 2024-09-12T14:19:32.386 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.151083.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.151083.log.gz 2024-09-12T14:19:32.386 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.96212.log 2024-09-12T14:19:32.387 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.148610.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.148610.log.gz 2024-09-12T14:19:32.387 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.141644.log 2024-09-12T14:19:32.388 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.96212.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.96212.log.gz 2024-09-12T14:19:32.388 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.58063.log 2024-09-12T14:19:32.388 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.141644.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.141644.log.gz 2024-09-12T14:19:32.388 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.85397.log 2024-09-12T14:19:32.389 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.58063.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.58063.log.gz 2024-09-12T14:19:32.389 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.43653.log 2024-09-12T14:19:32.390 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.85397.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.85397.log.gz 2024-09-12T14:19:32.390 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.110700.log 2024-09-12T14:19:32.390 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.43653.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.43653.log.gz 2024-09-12T14:19:32.391 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.83783.log 2024-09-12T14:19:32.391 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.110700.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.110700.log.gz 2024-09-12T14:19:32.392 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.77916.log 2024-09-12T14:19:32.392 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.83783.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.83783.log.gz 2024-09-12T14:19:32.392 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.95087.log 2024-09-12T14:19:32.393 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.77916.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.77916.log.gz 2024-09-12T14:19:32.393 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.49055.log 2024-09-12T14:19:32.394 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.95087.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.95087.log.gz 2024-09-12T14:19:32.394 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.97202.log 2024-09-12T14:19:32.394 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.49055.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.49055.log.gz 2024-09-12T14:19:32.394 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.131249.log 2024-09-12T14:19:32.395 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.97202.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.97202.log.gz 2024-09-12T14:19:32.395 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.98376.log 2024-09-12T14:19:32.396 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.131249.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.131249.log.gz 2024-09-12T14:19:32.396 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.131269.log 2024-09-12T14:19:32.397 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.98376.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.98376.log.gz 2024-09-12T14:19:32.397 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.107360.log 2024-09-12T14:19:32.398 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.131269.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.131269.log.gz 2024-09-12T14:19:32.398 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.98861.log 2024-09-12T14:19:32.398 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.107360.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.107360.log.gz 2024-09-12T14:19:32.398 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.70211.log 2024-09-12T14:19:32.399 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.98861.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.64241.log 2024-09-12T14:19:32.399 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.98861.log.gz 2024-09-12T14:19:32.400 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.70211.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.70211.log.gz 2024-09-12T14:19:32.400 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.49366.log 2024-09-12T14:19:32.401 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.64241.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.64241.log.gz 2024-09-12T14:19:32.401 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.148861.log 2024-09-12T14:19:32.402 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.49366.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.49366.log.gz 2024-09-12T14:19:32.402 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.86408.log 2024-09-12T14:19:32.402 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.148861.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.148861.log.gz 2024-09-12T14:19:32.402 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.105550.log 2024-09-12T14:19:32.403 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.86408.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.72043.log 2024-09-12T14:19:32.403 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.86408.log.gz 2024-09-12T14:19:32.404 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.105550.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.105550.log.gz 2024-09-12T14:19:32.404 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.113266.log 2024-09-12T14:19:32.405 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.72043.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.72043.log.gz 2024-09-12T14:19:32.405 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.130247.log 2024-09-12T14:19:32.406 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.113266.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.113266.log.gz 2024-09-12T14:19:32.406 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.97452.log 2024-09-12T14:19:32.406 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.130247.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.130247.log.gz 2024-09-12T14:19:32.406 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.84919.log 2024-09-12T14:19:32.407 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.97452.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.97452.log.gz 2024-09-12T14:19:32.407 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.131060.log 2024-09-12T14:19:32.408 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.84919.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.84919.log.gz 2024-09-12T14:19:32.408 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.111437.log 2024-09-12T14:19:32.409 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.131060.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.131060.log.gz 2024-09-12T14:19:32.409 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.39739.log 2024-09-12T14:19:32.409 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.111437.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.111437.log.gz 2024-09-12T14:19:32.409 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.38667.log 2024-09-12T14:19:32.410 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.39739.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.39739.log.gz 2024-09-12T14:19:32.410 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.83003.log 2024-09-12T14:19:32.411 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.38667.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.38667.log.gz 2024-09-12T14:19:32.412 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.132492.log 2024-09-12T14:19:32.412 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.83003.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.83003.log.gz 2024-09-12T14:19:32.412 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.88676.log 2024-09-12T14:19:32.413 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.132492.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.132492.log.gz 2024-09-12T14:19:32.413 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.120313.log 2024-09-12T14:19:32.414 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.88676.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.55058.log 2024-09-12T14:19:32.414 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.88676.log.gz 2024-09-12T14:19:32.414 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.120313.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.120313.log.gz 2024-09-12T14:19:32.415 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.90157.log 2024-09-12T14:19:32.415 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.55058.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.55058.log.gz 2024-09-12T14:19:32.415 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.109408.log 2024-09-12T14:19:32.416 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.90157.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.90157.log.gz 2024-09-12T14:19:32.416 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.106906.log 2024-09-12T14:19:32.417 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.109408.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.109408.log.gz 2024-09-12T14:19:32.417 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.57340.log 2024-09-12T14:19:32.418 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.106906.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.106906.log.gz 2024-09-12T14:19:32.418 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.100560.log 2024-09-12T14:19:32.419 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.57340.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.57340.log.gz 2024-09-12T14:19:32.419 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.86492.log 2024-09-12T14:19:32.420 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.100560.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.100560.log.gz 2024-09-12T14:19:32.420 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.63065.log 2024-09-12T14:19:32.420 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.86492.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.86492.log.gz 2024-09-12T14:19:32.421 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.132286.log 2024-09-12T14:19:32.421 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.63065.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.55680.log 2024-09-12T14:19:32.421 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.63065.log.gz 2024-09-12T14:19:32.422 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.132286.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.132286.log.gz 2024-09-12T14:19:32.422 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.151245.log 2024-09-12T14:19:32.423 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.55680.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.55680.log.gz 2024-09-12T14:19:32.423 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.35917.log 2024-09-12T14:19:32.424 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.151245.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.151245.log.gz 2024-09-12T14:19:32.424 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.66299.log 2024-09-12T14:19:32.425 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.35917.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.35917.log.gz 2024-09-12T14:19:32.425 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.55746.log 2024-09-12T14:19:32.426 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.66299.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.92176.log 2024-09-12T14:19:32.426 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.66299.log.gz 2024-09-12T14:19:32.426 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.55746.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.55746.log.gz 2024-09-12T14:19:32.427 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.108208.log 2024-09-12T14:19:32.427 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.92176.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.92176.log.gz 2024-09-12T14:19:32.428 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.86533.log 2024-09-12T14:19:32.428 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.108208.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.108208.log.gz 2024-09-12T14:19:32.428 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.123935.log 2024-09-12T14:19:32.429 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.86533.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.86533.log.gz 2024-09-12T14:19:32.429 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.69728.log 2024-09-12T14:19:32.430 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.123935.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.123935.log.gz 2024-09-12T14:19:32.430 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.148993.log 2024-09-12T14:19:32.431 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.69728.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.69728.log.gz 2024-09-12T14:19:32.431 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.120796.log 2024-09-12T14:19:32.432 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.148993.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.148993.log.gz 2024-09-12T14:19:32.432 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.139179.log 2024-09-12T14:19:32.433 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.120796.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.120796.log.gz 2024-09-12T14:19:32.433 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.146660.log 2024-09-12T14:19:32.434 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.139179.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.139179.log.gz 2024-09-12T14:19:32.434 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.61067.log 2024-09-12T14:19:32.434 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.146660.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.146660.log.gz 2024-09-12T14:19:32.434 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.63024.log 2024-09-12T14:19:32.435 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.61067.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.61067.log.gz 2024-09-12T14:19:32.435 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.149811.log 2024-09-12T14:19:32.435 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.63024.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.63024.log.gz 2024-09-12T14:19:32.436 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.108082.log 2024-09-12T14:19:32.436 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.149811.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.149811.log.gz 2024-09-12T14:19:32.437 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.57000.log 2024-09-12T14:19:32.437 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.108082.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.108082.log.gz 2024-09-12T14:19:32.438 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.115110.log 2024-09-12T14:19:32.438 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.57000.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.57000.log.gz 2024-09-12T14:19:32.438 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.40893.log 2024-09-12T14:19:32.439 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.115110.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.115110.log.gz 2024-09-12T14:19:32.439 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.92650.log 2024-09-12T14:19:32.440 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.40893.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.40893.log.gz 2024-09-12T14:19:32.440 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.53148.log 2024-09-12T14:19:32.440 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.92650.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.92650.log.gz 2024-09-12T14:19:32.441 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.95918.log 2024-09-12T14:19:32.441 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.53148.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.53148.log.gz 2024-09-12T14:19:32.441 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.137757.log 2024-09-12T14:19:32.442 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.95918.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.95918.log.gz 2024-09-12T14:19:32.442 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.68142.log 2024-09-12T14:19:32.443 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.137757.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.137757.log.gz 2024-09-12T14:19:32.443 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.61717.log 2024-09-12T14:19:32.444 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.68142.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.68142.log.gz 2024-09-12T14:19:32.444 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.74385.log 2024-09-12T14:19:32.445 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.61717.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.61717.log.gz 2024-09-12T14:19:32.445 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.83655.log 2024-09-12T14:19:32.446 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.74385.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.74385.log.gz 2024-09-12T14:19:32.446 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.88324.log 2024-09-12T14:19:32.446 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.83655.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.108770.log 2024-09-12T14:19:32.447 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.83655.log.gz 2024-09-12T14:19:32.447 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.88324.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.88324.log.gz 2024-09-12T14:19:32.447 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.150518.log 2024-09-12T14:19:32.448 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.108770.log: gzip 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.108770.log.gz -5 2024-09-12T14:19:32.448 INFO:teuthology.orchestra.run.smithi130.stderr: --verbose -- /var/log/ceph/cluster1-client.mirror.35363.log 2024-09-12T14:19:32.449 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.150518.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.150518.log.gz 2024-09-12T14:19:32.450 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.118347.log 2024-09-12T14:19:32.451 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.35363.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.35363.log.gz 2024-09-12T14:19:32.451 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.71718.log 2024-09-12T14:19:32.451 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.118347.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.118347.log.gz 2024-09-12T14:19:32.451 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.41100.log 2024-09-12T14:19:32.452 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.71718.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.71718.log.gz 2024-09-12T14:19:32.452 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.150321.log 2024-09-12T14:19:32.453 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.41100.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.41100.log.gz 2024-09-12T14:19:32.453 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.94295.log 2024-09-12T14:19:32.454 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.150321.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.150321.log.gz 2024-09-12T14:19:32.454 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.106331.log 2024-09-12T14:19:32.455 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.94295.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.94295.log.gz 2024-09-12T14:19:32.455 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.34812.log 2024-09-12T14:19:32.455 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.106331.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.122521.log 2024-09-12T14:19:32.455 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.106331.log.gz 2024-09-12T14:19:32.457 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.34812.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.110157.log 2024-09-12T14:19:32.457 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.34812.log.gz 2024-09-12T14:19:32.457 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.122521.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.122521.log.gz 2024-09-12T14:19:32.457 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.49305.log 2024-09-12T14:19:32.458 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.110157.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.110157.log.gz 2024-09-12T14:19:32.458 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.39839.log 2024-09-12T14:19:32.458 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.49305.log: gzip 0.0% -5 -- replaced with /var/log/ceph/cluster1-client.mirror.49305.log.gz --verbose 2024-09-12T14:19:32.458 INFO:teuthology.orchestra.run.smithi130.stderr: -- /var/log/ceph/cluster1-client.admin.68805.log 2024-09-12T14:19:32.459 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.39839.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.39839.log.gz 2024-09-12T14:19:32.459 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.80421.log 2024-09-12T14:19:32.460 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.68805.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.68805.log.gz 2024-09-12T14:19:32.460 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.66172.log 2024-09-12T14:19:32.461 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.80421.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.80421.log.gz 2024-09-12T14:19:32.461 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.81698.log 2024-09-12T14:19:32.461 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.66172.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.66172.log.gz 2024-09-12T14:19:32.461 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.38859.log 2024-09-12T14:19:32.462 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.81698.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.81698.log.gz 2024-09-12T14:19:32.462 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.67761.log 2024-09-12T14:19:32.463 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.38859.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.38859.log.gz 2024-09-12T14:19:32.463 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.153557.log 2024-09-12T14:19:32.464 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.67761.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.67761.log.gz 2024-09-12T14:19:32.464 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.73692.log 2024-09-12T14:19:32.465 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.153557.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.153557.log.gz 2024-09-12T14:19:32.465 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.148379.log 2024-09-12T14:19:32.465 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.73692.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.73692.log.gz 2024-09-12T14:19:32.466 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.89494.log 2024-09-12T14:19:32.466 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.148379.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.119117.log 2024-09-12T14:19:32.466 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.148379.log.gz 2024-09-12T14:19:32.467 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.89494.log: gzip 0.0% -5 --verbose -- replaced with /var/log/ceph/cluster1-client.mirror.89494.log.gz -- 2024-09-12T14:19:32.467 INFO:teuthology.orchestra.run.smithi130.stderr: /var/log/ceph/cluster1-client.mirror.62602.log 2024-09-12T14:19:32.468 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.119117.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.119117.log.gz 2024-09-12T14:19:32.468 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.89516.log 2024-09-12T14:19:32.469 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.62602.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.62602.log.gz 2024-09-12T14:19:32.469 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.76989.log 2024-09-12T14:19:32.469 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.89516.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.89516.log.gz 2024-09-12T14:19:32.469 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.71323.log 2024-09-12T14:19:32.470 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.76989.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.76989.log.gz 2024-09-12T14:19:32.470 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.97707.log 2024-09-12T14:19:32.472 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.71323.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.71323.log.gz 2024-09-12T14:19:32.472 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.91978.log 2024-09-12T14:19:32.472 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.97707.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.97707.log.gz 2024-09-12T14:19:32.472 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.122415.log 2024-09-12T14:19:32.474 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.91978.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.91978.log.gz 2024-09-12T14:19:32.474 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.0.34560.log 2024-09-12T14:19:32.474 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.122415.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.122415.log.gz 2024-09-12T14:19:32.474 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.128242.log 2024-09-12T14:19:32.475 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.0.34560.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.113942.log 2024-09-12T14:19:32.475 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.128242.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.128242.log.gz 2024-09-12T14:19:32.476 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.141509.log 2024-09-12T14:19:32.476 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.113942.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.113942.log.gz 2024-09-12T14:19:32.477 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.102746.log 2024-09-12T14:19:32.478 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.141509.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.141509.log.gz 2024-09-12T14:19:32.478 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-osd.1.log 2024-09-12T14:19:32.478 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.102746.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.102746.log.gz 2024-09-12T14:19:32.478 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.131299.log 2024-09-12T14:19:32.479 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-osd.1.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.148712.log 2024-09-12T14:19:32.481 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.131299.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.131299.log.gz 2024-09-12T14:19:32.481 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.64761.log 2024-09-12T14:19:32.481 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.148712.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.148712.log.gz 2024-09-12T14:19:32.481 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.150475.log 2024-09-12T14:19:32.482 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.64761.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.112236.log 2024-09-12T14:19:32.482 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.64761.log.gz 2024-09-12T14:19:32.483 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.150475.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.150475.log.gz 2024-09-12T14:19:32.483 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.142117.log 2024-09-12T14:19:32.484 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.112236.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.78833.log 2024-09-12T14:19:32.484 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.112236.log.gz 2024-09-12T14:19:32.484 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.142117.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.142117.log.gz 2024-09-12T14:19:32.484 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.123888.log 2024-09-12T14:19:32.485 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.78833.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.118261.log 2024-09-12T14:19:32.485 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.78833.log.gz 2024-09-12T14:19:32.486 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.123888.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.123888.log.gz 2024-09-12T14:19:32.486 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.113016.log 2024-09-12T14:19:32.487 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.118261.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.118261.log.gz 2024-09-12T14:19:32.487 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.38917.log 2024-09-12T14:19:32.487 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.113016.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.113016.log.gz 2024-09-12T14:19:32.487 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.84750.log 2024-09-12T14:19:32.488 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.38917.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.79970.log 2024-09-12T14:19:32.488 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.38917.log.gz 2024-09-12T14:19:32.489 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.84750.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.84750.log.gz 2024-09-12T14:19:32.489 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.153417.log 2024-09-12T14:19:32.489 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.79970.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.79970.log.gz 2024-09-12T14:19:32.489 INFO:teuthology.orchestra.run.smithi130.stderr: 89.9% -- replaced with /var/log/ceph/cluster2-client.mirror.0.34560.log.gz 2024-09-12T14:19:32.490 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.48696.log 2024-09-12T14:19:32.490 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.153417.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.153417.log.gz 2024-09-12T14:19:32.490 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.68992.log 2024-09-12T14:19:32.492 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.48696.log: 0.0%gzip -- replaced with /var/log/ceph/cluster1-client.mirror.48696.log.gz -5 2024-09-12T14:19:32.492 INFO:teuthology.orchestra.run.smithi130.stderr: --verbose -- /var/log/ceph/cluster1-client.mirror.113653.log 2024-09-12T14:19:32.492 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.68992.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.68992.log.gz 2024-09-12T14:19:32.492 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.113861.log 2024-09-12T14:19:32.493 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.113653.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.113653.log.gz 2024-09-12T14:19:32.493 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.86513.log 2024-09-12T14:19:32.493 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.113861.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.48385.log 0.0% 2024-09-12T14:19:32.493 INFO:teuthology.orchestra.run.smithi130.stderr: -- replaced with /var/log/ceph/cluster1-client.mirror.113861.log.gz 2024-09-12T14:19:32.495 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.86513.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.86513.log.gz 2024-09-12T14:19:32.495 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.72330.log 2024-09-12T14:19:32.495 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.48385.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.48385.log.gz 2024-09-12T14:19:32.495 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.74217.log 2024-09-12T14:19:32.496 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.72330.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.72330.log.gz 2024-09-12T14:19:32.496 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.116692.log 2024-09-12T14:19:32.496 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.74217.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.142583.log 2024-09-12T14:19:32.496 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.74217.log.gz 2024-09-12T14:19:32.497 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.116692.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.116692.log.gzgzip 2024-09-12T14:19:32.497 INFO:teuthology.orchestra.run.smithi130.stderr: -5 --verbose -- /var/log/ceph/cluster1-client.mirror.39338.log 2024-09-12T14:19:32.498 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.142583.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.142583.log.gz 2024-09-12T14:19:32.498 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.77995.log 2024-09-12T14:19:32.499 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.39338.log: 0.0%gzip -- replaced with /var/log/ceph/cluster1-client.mirror.39338.log.gz 2024-09-12T14:19:32.499 INFO:teuthology.orchestra.run.smithi130.stderr: -5 --verbose -- /var/log/ceph/cluster2-client.mirror.151660.log 2024-09-12T14:19:32.500 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.77995.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.77995.log.gz 2024-09-12T14:19:32.500 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.72216.log 2024-09-12T14:19:32.501 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.151660.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.151660.log.gz 2024-09-12T14:19:32.501 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.109107.log 2024-09-12T14:19:32.501 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.72216.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.72216.log.gz 2024-09-12T14:19:32.502 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.72956.log 2024-09-12T14:19:32.502 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.109107.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.109107.log.gz 2024-09-12T14:19:32.503 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.141945.log 2024-09-12T14:19:32.504 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.72956.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.138940.log 2024-09-12T14:19:32.504 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.72956.log.gz 2024-09-12T14:19:32.504 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.141945.log: 0.0%gzip -- replaced with /var/log/ceph/cluster2-client.mirror.141945.log.gz -5 2024-09-12T14:19:32.504 INFO:teuthology.orchestra.run.smithi130.stderr: --verbose -- /var/log/ceph/cluster1-client.mirror.47039.log 2024-09-12T14:19:32.505 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.138940.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.138940.log.gz 2024-09-12T14:19:32.505 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.39910.log 2024-09-12T14:19:32.505 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.47039.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.47039.log.gz 2024-09-12T14:19:32.505 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.60129.log 2024-09-12T14:19:32.506 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.39910.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.39910.log.gz 2024-09-12T14:19:32.506 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.38284.log 2024-09-12T14:19:32.507 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.60129.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.60129.log.gz 2024-09-12T14:19:32.507 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.153349.log 2024-09-12T14:19:32.508 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.38284.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.35194.log 2024-09-12T14:19:32.508 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.38284.log.gz 2024-09-12T14:19:32.509 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.153349.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.153349.log.gz 2024-09-12T14:19:32.509 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.74952.log 2024-09-12T14:19:32.510 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.35194.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.35194.log.gz 2024-09-12T14:19:32.510 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.146404.log 2024-09-12T14:19:32.510 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.74952.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.74952.log.gz 2024-09-12T14:19:32.510 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.68115.log 2024-09-12T14:19:32.511 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.146404.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.146404.log.gz 2024-09-12T14:19:32.511 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.58984.log 2024-09-12T14:19:32.511 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.68115.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.68115.log.gz 2024-09-12T14:19:32.512 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.142248.log 2024-09-12T14:19:32.513 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.58984.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.94140.log 2024-09-12T14:19:32.513 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.58984.log.gz 2024-09-12T14:19:32.513 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.142248.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.142248.log.gz 2024-09-12T14:19:32.513 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.52937.log 2024-09-12T14:19:32.514 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.94140.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.94140.log.gz 2024-09-12T14:19:32.514 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.121216.log 2024-09-12T14:19:32.515 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.52937.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.52937.log.gz 2024-09-12T14:19:32.515 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.103054.log 2024-09-12T14:19:32.516 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.121216.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.121216.log.gz 2024-09-12T14:19:32.516 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.104113.log 2024-09-12T14:19:32.517 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.103054.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.103054.log.gz 2024-09-12T14:19:32.517 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.121092.log 2024-09-12T14:19:32.518 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.104113.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.104113.log.gz 2024-09-12T14:19:32.518 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.116695.log 2024-09-12T14:19:32.518 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.121092.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.121092.log.gz 2024-09-12T14:19:32.519 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.74029.log 2024-09-12T14:19:32.519 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.116695.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.116695.log.gz 2024-09-12T14:19:32.519 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.117904.log 2024-09-12T14:19:32.519 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.74029.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.74029.log.gz 2024-09-12T14:19:32.520 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.55438.log 2024-09-12T14:19:32.521 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.117904.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.45009.log 2024-09-12T14:19:32.521 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.117904.log.gz 2024-09-12T14:19:32.521 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.55438.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.55438.log.gz 2024-09-12T14:19:32.521 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.71260.log 2024-09-12T14:19:32.522 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.45009.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.45009.log.gz 2024-09-12T14:19:32.522 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.34302.log 2024-09-12T14:19:32.523 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.71260.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.71260.log.gz 2024-09-12T14:19:32.523 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.89874.log 2024-09-12T14:19:32.524 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.34302.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.34302.log.gz 2024-09-12T14:19:32.524 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.60017.log 2024-09-12T14:19:32.525 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.89874.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.89874.log.gz 2024-09-12T14:19:32.525 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.69074.log 2024-09-12T14:19:32.526 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.60017.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.60017.log.gz 2024-09-12T14:19:32.526 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.108670.log 2024-09-12T14:19:32.526 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.69074.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.69074.log.gz 2024-09-12T14:19:32.527 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.123263.log 2024-09-12T14:19:32.527 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.108670.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.108670.log.gz 2024-09-12T14:19:32.528 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.65454.log 2024-09-12T14:19:32.528 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.123263.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.123263.log.gz 2024-09-12T14:19:32.528 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.59057.log 2024-09-12T14:19:32.529 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.65454.log: 0.0%gzip -- replaced with /var/log/ceph/cluster1-client.mirror.65454.log.gz 2024-09-12T14:19:32.529 INFO:teuthology.orchestra.run.smithi130.stderr: -5 --verbose -- /var/log/ceph/cluster2-client.mirror.56453.log 2024-09-12T14:19:32.530 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.59057.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.59057.log.gz 2024-09-12T14:19:32.530 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.38707.log 2024-09-12T14:19:32.531 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.56453.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.56453.log.gz 2024-09-12T14:19:32.531 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.68601.log 2024-09-12T14:19:32.531 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.38707.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.38707.log.gz 2024-09-12T14:19:32.531 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.120106.log 2024-09-12T14:19:32.532 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.68601.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.63484.log 2024-09-12T14:19:32.532 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.68601.log.gz 2024-09-12T14:19:32.533 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.120106.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.120106.log.gz 2024-09-12T14:19:32.533 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.121909.log 2024-09-12T14:19:32.534 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.63484.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.63484.log.gz 2024-09-12T14:19:32.534 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.67253.log 2024-09-12T14:19:32.535 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.121909.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.121909.log.gz 2024-09-12T14:19:32.535 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.124189.log 2024-09-12T14:19:32.536 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.67253.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.67253.log.gz 2024-09-12T14:19:32.536 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.83919.log 2024-09-12T14:19:32.537 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.124189.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.124189.log.gz 2024-09-12T14:19:32.537 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.130553.log 2024-09-12T14:19:32.538 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.83919.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.91395.log 2024-09-12T14:19:32.538 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.83919.log.gz 2024-09-12T14:19:32.538 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.130553.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.130553.log.gz 2024-09-12T14:19:32.538 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.107994.log 2024-09-12T14:19:32.539 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.91395.log: 0.0%gzip -- replaced with /var/log/ceph/cluster1-client.admin.91395.log.gz 2024-09-12T14:19:32.539 INFO:teuthology.orchestra.run.smithi130.stderr: -5 --verbose -- /var/log/ceph/cluster1-client.mirror.85635.log 2024-09-12T14:19:32.540 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.107994.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.107994.log.gz 2024-09-12T14:19:32.540 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.116939.log 2024-09-12T14:19:32.541 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.85635.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.46697.log 2024-09-12T14:19:32.541 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.85635.log.gz 2024-09-12T14:19:32.542 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.116939.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.116939.log.gz 2024-09-12T14:19:32.542 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.80212.log 2024-09-12T14:19:32.542 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.46697.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.46697.log.gz 2024-09-12T14:19:32.543 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.139810.log 2024-09-12T14:19:32.543 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.80212.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.80212.log.gz 2024-09-12T14:19:32.544 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.147684.log 2024-09-12T14:19:32.544 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.139810.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.139810.log.gz 2024-09-12T14:19:32.545 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.82402.log 2024-09-12T14:19:32.545 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.147684.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.147684.log.gz 2024-09-12T14:19:32.546 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.95717.log 2024-09-12T14:19:32.546 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.82402.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.82402.log.gz 2024-09-12T14:19:32.547 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.78526.log 2024-09-12T14:19:32.548 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.95717.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.95717.log.gz 2024-09-12T14:19:32.548 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.149074.log 2024-09-12T14:19:32.548 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.78526.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.78526.log.gz 2024-09-12T14:19:32.548 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.144829.log 2024-09-12T14:19:32.549 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.149074.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.149074.log.gz 2024-09-12T14:19:32.549 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.151351.log 2024-09-12T14:19:32.550 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.144829.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.144829.log.gz 2024-09-12T14:19:32.550 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.139508.log 2024-09-12T14:19:32.551 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.151351.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.151351.log.gz 2024-09-12T14:19:32.551 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.119729.log 2024-09-12T14:19:32.552 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.139508.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.139508.log.gz 2024-09-12T14:19:32.552 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.51337.log 2024-09-12T14:19:32.552 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.119729.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.119729.log.gz 2024-09-12T14:19:32.552 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.36745.log 2024-09-12T14:19:32.553 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.51337.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.51337.log.gz 2024-09-12T14:19:32.553 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.98018.log 2024-09-12T14:19:32.554 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.36745.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.36745.log.gz 2024-09-12T14:19:32.554 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.143034.log 2024-09-12T14:19:32.555 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.98018.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.98018.log.gz 2024-09-12T14:19:32.555 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.76336.log 2024-09-12T14:19:32.556 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.143034.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.143034.log.gz 2024-09-12T14:19:32.556 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.97326.log 2024-09-12T14:19:32.557 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.76336.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.76336.log.gz 2024-09-12T14:19:32.557 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.106140.log 2024-09-12T14:19:32.557 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.97326.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.97326.log.gz 2024-09-12T14:19:32.557 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.150914.log 2024-09-12T14:19:32.558 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.106140.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.106140.log.gz 2024-09-12T14:19:32.558 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.143160.log 2024-09-12T14:19:32.559 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.150914.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.150914.log.gz 2024-09-12T14:19:32.559 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.142465.log 2024-09-12T14:19:32.560 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.143160.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.143160.log.gz 2024-09-12T14:19:32.560 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.80514.log 2024-09-12T14:19:32.560 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.142465.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.142465.log.gz 2024-09-12T14:19:32.561 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.118188.log 2024-09-12T14:19:32.562 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.80514.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.80514.log.gz 2024-09-12T14:19:32.562 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.38267.log 2024-09-12T14:19:32.562 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.118188.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.118188.log.gz 2024-09-12T14:19:32.562 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.83503.log 2024-09-12T14:19:32.563 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.38267.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.38267.log.gz 2024-09-12T14:19:32.563 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.39157.log 2024-09-12T14:19:32.563 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.83503.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.83503.log.gz 2024-09-12T14:19:32.564 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.41377.log 2024-09-12T14:19:32.564 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.39157.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.39157.log.gz 2024-09-12T14:19:32.564 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.132350.log 2024-09-12T14:19:32.565 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.41377.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.41377.log.gz 2024-09-12T14:19:32.565 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.131296.log 2024-09-12T14:19:32.566 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.132350.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.132350.log.gz 2024-09-12T14:19:32.566 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.45741.log 2024-09-12T14:19:32.567 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.131296.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.131296.log.gz 2024-09-12T14:19:32.567 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.83810.log 2024-09-12T14:19:32.567 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.45741.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.45741.log.gz 2024-09-12T14:19:32.568 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.52306.log 2024-09-12T14:19:32.568 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.83810.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.83810.log.gz 2024-09-12T14:19:32.568 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.63272.log 2024-09-12T14:19:32.570 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.52306.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.52306.log.gzgzip 2024-09-12T14:19:32.570 INFO:teuthology.orchestra.run.smithi130.stderr: -5 --verbose -- /var/log/ceph/cluster1-client.mirror.62663.log 2024-09-12T14:19:32.570 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.63272.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.63272.log.gz 2024-09-12T14:19:32.570 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.88804.log 2024-09-12T14:19:32.570 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.62663.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.62663.log.gz 2024-09-12T14:19:32.571 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.107871.log 2024-09-12T14:19:32.571 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.88804.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.88804.log.gz 2024-09-12T14:19:32.571 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.100266.log 2024-09-12T14:19:32.572 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.107871.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.107871.log.gz 2024-09-12T14:19:32.572 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.63101.log 2024-09-12T14:19:32.573 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.100266.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.100266.log.gz 2024-09-12T14:19:32.573 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.130943.log 2024-09-12T14:19:32.574 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.63101.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.72328.log 2024-09-12T14:19:32.574 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.63101.log.gz 2024-09-12T14:19:32.574 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.130943.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.130943.log.gz 2024-09-12T14:19:32.575 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.86620.log 2024-09-12T14:19:32.575 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.72328.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.72328.log.gz 2024-09-12T14:19:32.576 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.85150.log 2024-09-12T14:19:32.576 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.86620.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.86620.log.gz 2024-09-12T14:19:32.577 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.123014.log 2024-09-12T14:19:32.577 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.85150.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.85150.log.gz 2024-09-12T14:19:32.577 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.142311.log 2024-09-12T14:19:32.578 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.123014.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.123014.log.gz 2024-09-12T14:19:32.578 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.88466.log 2024-09-12T14:19:32.579 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.142311.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.142311.log.gz 2024-09-12T14:19:32.579 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.98272.log 2024-09-12T14:19:32.579 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.88466.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.88466.log.gz 2024-09-12T14:19:32.580 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.109044.log 2024-09-12T14:19:32.580 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.98272.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.98272.log.gz 2024-09-12T14:19:32.580 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.99804.log 2024-09-12T14:19:32.581 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.109044.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.109044.log.gz 2024-09-12T14:19:32.581 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.117448.log 2024-09-12T14:19:32.582 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.99804.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.99804.log.gz 2024-09-12T14:19:32.582 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.108855.log 2024-09-12T14:19:32.582 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.117448.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.117448.log.gz 2024-09-12T14:19:32.582 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.132913.log 2024-09-12T14:19:32.583 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.108855.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.108855.log.gz 2024-09-12T14:19:32.583 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.112555.log 2024-09-12T14:19:32.584 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.132913.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.132913.log.gz 2024-09-12T14:19:32.584 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.144315.log 2024-09-12T14:19:32.585 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.112555.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.112555.log.gz 2024-09-12T14:19:32.585 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.39636.log 2024-09-12T14:19:32.585 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.144315.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.144315.log.gz 2024-09-12T14:19:32.586 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.41040.log 2024-09-12T14:19:32.586 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.39636.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.39636.log.gz 2024-09-12T14:19:32.586 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.100184.log 2024-09-12T14:19:32.587 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.41040.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.41040.log.gz 2024-09-12T14:19:32.587 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.144659.log 2024-09-12T14:19:32.588 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.100184.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.100184.log.gz 2024-09-12T14:19:32.588 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.53480.log 2024-09-12T14:19:32.588 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.144659.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.144659.log.gz 2024-09-12T14:19:32.589 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.80928.log 2024-09-12T14:19:32.589 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.53480.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.53480.log.gz 2024-09-12T14:19:32.589 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.61381.log 2024-09-12T14:19:32.590 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.80928.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.130284.log 2024-09-12T14:19:32.590 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.80928.log.gz 2024-09-12T14:19:32.591 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.61381.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.61381.log.gz 2024-09-12T14:19:32.591 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.87734.log 2024-09-12T14:19:32.592 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.130284.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.130284.log.gz 2024-09-12T14:19:32.592 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.37595.log 2024-09-12T14:19:32.592 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.87734.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.87734.log.gz 2024-09-12T14:19:32.592 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.99805.log 2024-09-12T14:19:32.593 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.37595.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.131753.log 2024-09-12T14:19:32.593 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.37595.log.gz 2024-09-12T14:19:32.594 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.99805.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.99805.log.gz 2024-09-12T14:19:32.594 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.132140.log 2024-09-12T14:19:32.595 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.131753.log: gzip 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.131753.log.gz -5 2024-09-12T14:19:32.595 INFO:teuthology.orchestra.run.smithi130.stderr: --verbose -- /var/log/ceph/cluster2-client.mirror.153329.log 2024-09-12T14:19:32.596 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.132140.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.132140.log.gz 2024-09-12T14:19:32.596 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.36481.log 2024-09-12T14:19:32.596 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.153329.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.153329.log.gz 2024-09-12T14:19:32.596 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.70599.log 2024-09-12T14:19:32.597 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.36481.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.36481.log.gz 2024-09-12T14:19:32.597 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.53609.log 2024-09-12T14:19:32.598 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.70599.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.70599.log.gz 2024-09-12T14:19:32.598 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.96231.log 2024-09-12T14:19:32.599 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.53609.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.53609.log.gz 2024-09-12T14:19:32.599 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.138939.log 2024-09-12T14:19:32.599 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.96231.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.96231.log.gz 2024-09-12T14:19:32.599 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.78017.log 2024-09-12T14:19:32.600 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.138939.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.138939.log.gz 2024-09-12T14:19:32.600 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.88972.log 2024-09-12T14:19:32.601 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.78017.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.78017.log.gz 2024-09-12T14:19:32.601 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.128256.log 2024-09-12T14:19:32.602 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.88972.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.88972.log.gz 2024-09-12T14:19:32.602 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.96861.log 2024-09-12T14:19:32.602 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.128256.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.128256.log.gz 2024-09-12T14:19:32.603 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.113374.log 2024-09-12T14:19:32.604 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.96861.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.96861.log.gz 2024-09-12T14:19:32.604 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.135919.log 2024-09-12T14:19:32.605 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.113374.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.113374.log.gz 2024-09-12T14:19:32.605 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.120270.log 2024-09-12T14:19:32.606 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.135919.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.135919.log.gz 2024-09-12T14:19:32.606 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.100017.log 2024-09-12T14:19:32.607 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.120270.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.120270.log.gz 2024-09-12T14:19:32.607 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.77139.log 2024-09-12T14:19:32.608 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.100017.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.100017.log.gz 2024-09-12T14:19:32.608 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.117381.log 2024-09-12T14:19:32.608 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.77139.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.77139.log.gz 2024-09-12T14:19:32.608 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.112783.log 2024-09-12T14:19:32.609 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.117381.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.117381.log.gz 2024-09-12T14:19:32.609 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.113814.log 2024-09-12T14:19:32.610 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.112783.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.79589.log 2024-09-12T14:19:32.611 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.112783.log.gz 2024-09-12T14:19:32.611 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.113814.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.113814.log.gz 2024-09-12T14:19:32.611 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.86303.log 2024-09-12T14:19:32.612 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.79589.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.79589.log.gz 2024-09-12T14:19:32.612 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.54584.log 2024-09-12T14:19:32.613 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.86303.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.86303.log.gz 2024-09-12T14:19:32.613 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.57430.log 2024-09-12T14:19:32.613 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.54584.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.54584.log.gz 2024-09-12T14:19:32.613 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.152648.log 2024-09-12T14:19:32.615 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.57430.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.57430.log.gz 2024-09-12T14:19:32.615 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.84476.log 2024-09-12T14:19:32.615 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.152648.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.41169.log 2024-09-12T14:19:32.615 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.152648.log.gz 2024-09-12T14:19:32.616 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.84476.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.84476.log.gz 2024-09-12T14:19:32.616 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.59667.log 2024-09-12T14:19:32.617 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.41169.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.41169.log.gz 2024-09-12T14:19:32.617 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.75894.log 2024-09-12T14:19:32.617 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.59667.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.59667.log.gz 2024-09-12T14:19:32.617 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.121614.log 2024-09-12T14:19:32.618 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.75894.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.75894.log.gz 2024-09-12T14:19:32.618 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.116367.log 2024-09-12T14:19:32.619 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.121614.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.121614.log.gz 2024-09-12T14:19:32.619 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.113819.log 2024-09-12T14:19:32.620 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.116367.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.116367.log.gz 2024-09-12T14:19:32.620 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.139739.log 2024-09-12T14:19:32.621 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.113819.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.113819.log.gz 2024-09-12T14:19:32.621 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.44713.log 2024-09-12T14:19:32.622 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.139739.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.139739.log.gz 2024-09-12T14:19:32.622 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.137957.log 2024-09-12T14:19:32.622 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.44713.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.51993.log 2024-09-12T14:19:32.623 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.44713.log.gz 2024-09-12T14:19:32.623 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.137957.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.137957.log.gz 2024-09-12T14:19:32.623 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.107931.log 2024-09-12T14:19:32.624 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.51993.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.51993.log.gz 2024-09-12T14:19:32.624 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.110975.log 2024-09-12T14:19:32.625 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.107931.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.107931.log.gz 2024-09-12T14:19:32.625 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.140763.log 2024-09-12T14:19:32.625 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.110975.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.110975.log.gz 2024-09-12T14:19:32.626 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.80130.log 2024-09-12T14:19:32.627 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.140763.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.140763.log.gz 2024-09-12T14:19:32.627 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.82429.log 2024-09-12T14:19:32.628 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.80130.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.80130.log.gz 2024-09-12T14:19:32.628 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.123134.log 2024-09-12T14:19:32.628 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.82429.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.82429.log.gz 2024-09-12T14:19:32.628 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.129003.log 2024-09-12T14:19:32.629 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.123134.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.123134.log.gz 2024-09-12T14:19:32.629 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.131213.log 2024-09-12T14:19:32.630 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.129003.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.129003.log.gz 2024-09-12T14:19:32.630 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.150941.log 2024-09-12T14:19:32.631 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.131213.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.131213.log.gz 2024-09-12T14:19:32.631 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.122968.log 2024-09-12T14:19:32.632 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.150941.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.150941.log.gz 2024-09-12T14:19:32.632 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.81121.log 2024-09-12T14:19:32.632 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.122968.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.122968.log.gzgzip 2024-09-12T14:19:32.632 INFO:teuthology.orchestra.run.smithi130.stderr: -5 --verbose -- /var/log/ceph/cluster2-client.admin.127526.log 2024-09-12T14:19:32.633 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.81121.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.81121.log.gz 2024-09-12T14:19:32.633 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.127868.log 2024-09-12T14:19:32.634 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.127526.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.127526.log.gz 2024-09-12T14:19:32.634 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.138387.log 2024-09-12T14:19:32.635 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.127868.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.127868.log.gz 2024-09-12T14:19:32.635 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.36590.log 2024-09-12T14:19:32.636 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.138387.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.138387.log.gz 2024-09-12T14:19:32.636 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.75287.log 2024-09-12T14:19:32.637 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.36590.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.36590.log.gz 2024-09-12T14:19:32.637 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.119938.log 2024-09-12T14:19:32.637 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.75287.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.75287.log.gz 2024-09-12T14:19:32.638 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.63903.log 2024-09-12T14:19:32.639 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.119938.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.119938.log.gz 2024-09-12T14:19:32.639 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.98673.log 2024-09-12T14:19:32.639 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.63903.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.63903.log.gz 2024-09-12T14:19:32.640 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.85949.log 2024-09-12T14:19:32.641 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.98673.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.98673.log.gz 2024-09-12T14:19:32.641 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.51592.log 2024-09-12T14:19:32.641 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.85949.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.74700.log 2024-09-12T14:19:32.642 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.85949.log.gz 2024-09-12T14:19:32.643 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.51592.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.51592.log.gz 2024-09-12T14:19:32.643 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.43137.log 2024-09-12T14:19:32.643 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.74700.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.74700.log.gz 2024-09-12T14:19:32.643 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.110453.log 2024-09-12T14:19:32.645 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.43137.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.43137.log.gz 2024-09-12T14:19:32.645 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.108983.log 2024-09-12T14:19:32.645 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.110453.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.110453.log.gz 2024-09-12T14:19:32.645 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.52535.log 2024-09-12T14:19:32.647 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.108983.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.108983.log.gz 2024-09-12T14:19:32.647 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.111294.log 2024-09-12T14:19:32.647 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.52535.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.52535.log.gz 2024-09-12T14:19:32.647 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.70164.log 2024-09-12T14:19:32.647 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.111294.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.111294.log.gz 2024-09-12T14:19:32.648 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.153484.log 2024-09-12T14:19:32.648 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.70164.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.70164.log.gz 2024-09-12T14:19:32.648 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.124378.log 2024-09-12T14:19:32.649 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.153484.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.153484.log.gz 2024-09-12T14:19:32.649 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.110933.log 2024-09-12T14:19:32.650 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.124378.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.124378.log.gz 2024-09-12T14:19:32.650 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.112048.log 2024-09-12T14:19:32.651 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.110933.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.110933.log.gz 2024-09-12T14:19:32.651 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.76236.log 2024-09-12T14:19:32.652 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.112048.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.112048.log.gz 2024-09-12T14:19:32.652 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.46744.log 2024-09-12T14:19:32.652 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.76236.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.76236.log.gz 2024-09-12T14:19:32.653 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.82610.log 2024-09-12T14:19:32.653 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.46744.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.46744.log.gz 2024-09-12T14:19:32.654 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.73041.log 2024-09-12T14:19:32.654 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.82610.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.82610.log.gz 2024-09-12T14:19:32.654 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.60733.log 2024-09-12T14:19:32.655 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.73041.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.73041.log.gz 2024-09-12T14:19:32.655 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.111711.log 2024-09-12T14:19:32.656 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.60733.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.60733.log.gz 2024-09-12T14:19:32.656 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.91800.log 2024-09-12T14:19:32.657 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.111711.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.111711.log.gz 2024-09-12T14:19:32.657 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.110136.log 2024-09-12T14:19:32.658 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.91800.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.91800.log.gz 2024-09-12T14:19:32.658 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.105684.log 2024-09-12T14:19:32.659 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.110136.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.110136.log.gz 2024-09-12T14:19:32.659 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.34887.log 2024-09-12T14:19:32.660 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.105684.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.105684.log.gz 2024-09-12T14:19:32.660 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.110683.log 2024-09-12T14:19:32.661 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.34887.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.34887.log.gz 2024-09-12T14:19:32.661 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.80671.log 2024-09-12T14:19:32.662 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.110683.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.110683.log.gz 2024-09-12T14:19:32.662 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.46126.log 2024-09-12T14:19:32.662 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.80671.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.80671.log.gz 2024-09-12T14:19:32.663 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.124545.log 2024-09-12T14:19:32.663 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.46126.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.46126.log.gz 2024-09-12T14:19:32.663 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.133547.log 2024-09-12T14:19:32.664 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.124545.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.124545.log.gz 2024-09-12T14:19:32.664 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.145082.log 2024-09-12T14:19:32.665 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.133547.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.133547.log.gz 2024-09-12T14:19:32.665 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.81926.log 2024-09-12T14:19:32.666 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.145082.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.145082.log.gz 2024-09-12T14:19:32.666 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.45013.log 2024-09-12T14:19:32.667 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.81926.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.81926.log.gz 2024-09-12T14:19:32.667 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.117997.log 2024-09-12T14:19:32.668 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.45013.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.45013.log.gz 2024-09-12T14:19:32.668 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.44355.log 2024-09-12T14:19:32.669 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.117997.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.69231.log 2024-09-12T14:19:32.669 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.117997.log.gz 2024-09-12T14:19:32.670 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.44355.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.44355.log.gz 2024-09-12T14:19:32.670 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.56677.log 2024-09-12T14:19:32.670 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.69231.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.69231.log.gz 2024-09-12T14:19:32.670 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.61848.log 2024-09-12T14:19:32.671 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.56677.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.56677.log.gz 2024-09-12T14:19:32.671 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.41754.log 2024-09-12T14:19:32.672 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.61848.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.61848.log.gz 2024-09-12T14:19:32.672 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.118151.log 2024-09-12T14:19:32.673 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.41754.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.41754.log.gz 2024-09-12T14:19:32.673 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.44628.log 2024-09-12T14:19:32.674 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.118151.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.118151.log.gz 2024-09-12T14:19:32.674 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.132535.log 2024-09-12T14:19:32.674 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.44628.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.44628.log.gz 2024-09-12T14:19:32.675 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.122436.log 2024-09-12T14:19:32.675 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.132535.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.132535.log.gz 2024-09-12T14:19:32.675 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.57732.log 2024-09-12T14:19:32.676 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.122436.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.122436.log.gz 2024-09-12T14:19:32.676 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.90277.log 2024-09-12T14:19:32.677 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.57732.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.57732.log.gz 2024-09-12T14:19:32.677 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.134510.log 2024-09-12T14:19:32.678 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.90277.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.71412.log 2024-09-12T14:19:32.678 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.90277.log.gz 2024-09-12T14:19:32.679 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.134510.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.134510.log.gz 2024-09-12T14:19:32.679 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.78320.log 2024-09-12T14:19:32.680 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.71412.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.71412.log.gz 2024-09-12T14:19:32.680 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.36925.log 2024-09-12T14:19:32.680 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.78320.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.78320.log.gz 2024-09-12T14:19:32.681 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.58679.log 2024-09-12T14:19:32.681 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.36925.log: gzip 0.0% -5 -- replaced with /var/log/ceph/cluster1-client.mirror.36925.log.gz --verbose 2024-09-12T14:19:32.681 INFO:teuthology.orchestra.run.smithi130.stderr: -- /var/log/ceph/cluster2-client.admin.44770.log 2024-09-12T14:19:32.683 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.58679.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.58679.log.gz 2024-09-12T14:19:32.683 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.52034.log 2024-09-12T14:19:32.683 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.44770.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.44770.log.gz 2024-09-12T14:19:32.683 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.63353.log 2024-09-12T14:19:32.684 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.52034.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.52034.log.gz 2024-09-12T14:19:32.684 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.95039.log 2024-09-12T14:19:32.684 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.63353.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.63353.log.gz 2024-09-12T14:19:32.685 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.50417.log 2024-09-12T14:19:32.685 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.95039.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.114189.log 2024-09-12T14:19:32.686 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.95039.log.gz 2024-09-12T14:19:32.686 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.50417.log: 0.0%gzip -- replaced with /var/log/ceph/cluster1-client.mirror.50417.log.gz 2024-09-12T14:19:32.686 INFO:teuthology.orchestra.run.smithi130.stderr: -5 --verbose -- /var/log/ceph/cluster1-client.admin.115065.log 2024-09-12T14:19:32.687 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.114189.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.114189.log.gz 2024-09-12T14:19:32.687 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.129772.log 2024-09-12T14:19:32.687 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.115065.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.35492.log 2024-09-12T14:19:32.687 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.115065.log.gz 2024-09-12T14:19:32.688 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.129772.log: 0.0%gzip -- replaced with /var/log/ceph/cluster1-client.mirror.129772.log.gz -5 2024-09-12T14:19:32.688 INFO:teuthology.orchestra.run.smithi130.stderr: --verbose -- /var/log/ceph/cluster1-client.admin.103275.log 2024-09-12T14:19:32.689 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.35492.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.35492.log.gz 2024-09-12T14:19:32.689 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.150968.log 2024-09-12T14:19:32.689 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.103275.log: 0.0%gzip -- replaced with /var/log/ceph/cluster1-client.admin.103275.log.gz -5 2024-09-12T14:19:32.689 INFO:teuthology.orchestra.run.smithi130.stderr: --verbose -- /var/log/ceph/cluster1-client.mirror.50837.log 2024-09-12T14:19:32.690 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.150968.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.150968.log.gz 2024-09-12T14:19:32.690 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.127399.log 2024-09-12T14:19:32.691 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.50837.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.50837.log.gz 2024-09-12T14:19:32.691 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.153101.log 2024-09-12T14:19:32.692 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.127399.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.127399.log.gz 2024-09-12T14:19:32.692 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.144557.log 2024-09-12T14:19:32.692 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.153101.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.153101.log.gz 2024-09-12T14:19:32.693 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.134889.log 2024-09-12T14:19:32.693 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.144557.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.144557.log.gz 2024-09-12T14:19:32.693 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.121969.log 2024-09-12T14:19:32.694 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.134889.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.134889.log.gz 2024-09-12T14:19:32.694 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.85863.log 2024-09-12T14:19:32.695 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.121969.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.121969.log.gz 2024-09-12T14:19:32.695 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.46012.log 2024-09-12T14:19:32.695 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.85863.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.85863.log.gz 2024-09-12T14:19:32.696 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.43122.log 2024-09-12T14:19:32.696 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.46012.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.49492.log 2024-09-12T14:19:32.696 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.46012.log.gz 2024-09-12T14:19:32.697 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.43122.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.43122.log.gz 2024-09-12T14:19:32.697 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.143360.log 2024-09-12T14:19:32.698 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.49492.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.49492.log.gz 2024-09-12T14:19:32.698 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.112848.log 2024-09-12T14:19:32.699 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.143360.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.143360.log.gz 2024-09-12T14:19:32.699 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.82870.log 2024-09-12T14:19:32.700 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.112848.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.112848.log.gz 2024-09-12T14:19:32.700 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.119268.log 2024-09-12T14:19:32.701 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.82870.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.82870.log.gz 2024-09-12T14:19:32.701 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.37721.log 2024-09-12T14:19:32.701 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.119268.log: 0.0%gzip -- replaced with /var/log/ceph/cluster1-client.mirror.119268.log.gz -5 2024-09-12T14:19:32.701 INFO:teuthology.orchestra.run.smithi130.stderr: --verbose -- /var/log/ceph/cluster1-client.admin.115679.log 2024-09-12T14:19:32.702 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.37721.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.37721.log.gz 2024-09-12T14:19:32.702 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.153504.log 2024-09-12T14:19:32.703 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.115679.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.115679.log.gz 2024-09-12T14:19:32.704 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.43446.log 2024-09-12T14:19:32.704 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.153504.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.153504.log.gz 2024-09-12T14:19:32.704 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.102055.log 2024-09-12T14:19:32.705 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.43446.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.61591.log 2024-09-12T14:19:32.705 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.43446.log.gz 2024-09-12T14:19:32.705 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.102055.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.102055.log.gz 2024-09-12T14:19:32.706 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.104828.log 2024-09-12T14:19:32.707 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.61591.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.61591.log.gz 2024-09-12T14:19:32.707 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.49830.log 2024-09-12T14:19:32.707 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.104828.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.104828.log.gz 2024-09-12T14:19:32.707 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.58472.log 2024-09-12T14:19:32.708 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.49830.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.49830.log.gz 2024-09-12T14:19:32.709 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.136197.log 2024-09-12T14:19:32.709 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.58472.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.58472.log.gz 2024-09-12T14:19:32.709 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.58633.log 2024-09-12T14:19:32.711 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.136197.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.136197.log.gz 2024-09-12T14:19:32.711 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.84162.log 2024-09-12T14:19:32.711 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.58633.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.58633.log.gz 2024-09-12T14:19:32.711 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.134761.log 2024-09-12T14:19:32.711 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.84162.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.84162.log.gz 2024-09-12T14:19:32.711 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.68472.log 2024-09-12T14:19:32.714 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.134761.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.134761.log.gz 2024-09-12T14:19:32.714 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.52604.log 2024-09-12T14:19:32.714 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.68472.log: 0.0%gzip -- replaced with /var/log/ceph/cluster2-client.admin.68472.log.gz -5 2024-09-12T14:19:32.714 INFO:teuthology.orchestra.run.smithi130.stderr: --verbose -- /var/log/ceph/cluster1-client.admin.126984.log 2024-09-12T14:19:32.715 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.52604.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.52604.log.gz 2024-09-12T14:19:32.715 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.97467.log 2024-09-12T14:19:32.715 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.126984.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.126984.log.gz 2024-09-12T14:19:32.715 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.141919.log 2024-09-12T14:19:32.715 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.97467.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.56529.log 2024-09-12T14:19:32.715 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.97467.log.gz 2024-09-12T14:19:32.717 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.141919.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.141919.log.gz 2024-09-12T14:19:32.717 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.71869.log 2024-09-12T14:19:32.717 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.56529.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.56529.log.gz 2024-09-12T14:19:32.717 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.114346.log 2024-09-12T14:19:32.717 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.71869.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.71869.log.gz 2024-09-12T14:19:32.717 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.118847.log 2024-09-12T14:19:32.718 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.114346.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.114346.log.gz 2024-09-12T14:19:32.718 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.125808.log 2024-09-12T14:19:32.718 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.118847.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.118847.log.gz 2024-09-12T14:19:32.718 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.64301.log 2024-09-12T14:19:32.719 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.125808.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.125808.log.gz 2024-09-12T14:19:32.719 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.43182.log 2024-09-12T14:19:32.720 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.64301.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.64301.log.gz 2024-09-12T14:19:32.720 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.130816.log 2024-09-12T14:19:32.720 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.43182.log: gzip -5 --verbose -- 0.0% /var/log/ceph/cluster1-client.mirror.61298.log 2024-09-12T14:19:32.720 INFO:teuthology.orchestra.run.smithi130.stderr: -- replaced with /var/log/ceph/cluster1-client.mirror.43182.log.gz 2024-09-12T14:19:32.721 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.130816.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.130816.log.gz 2024-09-12T14:19:32.721 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.64451.log 2024-09-12T14:19:32.722 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.61298.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.61298.log.gz 2024-09-12T14:19:32.722 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.139072.log 2024-09-12T14:19:32.723 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.64451.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.39465.log 2024-09-12T14:19:32.723 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.64451.log.gz 2024-09-12T14:19:32.724 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.139072.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.139072.log.gz 2024-09-12T14:19:32.724 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.48802.log 2024-09-12T14:19:32.724 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.39465.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.39465.log.gz 2024-09-12T14:19:32.725 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.137265.log 2024-09-12T14:19:32.725 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.48802.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.48802.log.gz 2024-09-12T14:19:32.725 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.128818.log 2024-09-12T14:19:32.726 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.137265.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.137265.log.gz 2024-09-12T14:19:32.726 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.136360.log 2024-09-12T14:19:32.727 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.128818.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.128818.log.gz 2024-09-12T14:19:32.727 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.111397.log 2024-09-12T14:19:32.727 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.136360.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.136360.log.gz 2024-09-12T14:19:32.728 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.107712.log 2024-09-12T14:19:32.728 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.111397.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.111397.log.gz 2024-09-12T14:19:32.728 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.40179.log 2024-09-12T14:19:32.729 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.107712.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.107712.log.gz 2024-09-12T14:19:32.729 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.149619.log 2024-09-12T14:19:32.730 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.40179.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.40179.log.gz 2024-09-12T14:19:32.730 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.124018.log 2024-09-12T14:19:32.731 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.149619.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.149619.log.gz 2024-09-12T14:19:32.731 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.91938.log 2024-09-12T14:19:32.731 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.124018.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.124018.log.gz 2024-09-12T14:19:32.732 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.66024.log 2024-09-12T14:19:32.732 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.91938.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.91938.log.gz 2024-09-12T14:19:32.732 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.79281.log 2024-09-12T14:19:32.733 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.66024.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.66024.log.gz 2024-09-12T14:19:32.733 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.82908.log 2024-09-12T14:19:32.734 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.79281.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.79281.log.gz 2024-09-12T14:19:32.734 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.87416.log 2024-09-12T14:19:32.735 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.82908.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.82908.log.gz 2024-09-12T14:19:32.735 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.107869.log 2024-09-12T14:19:32.735 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.87416.log: gzip 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.87416.log.gz -5 2024-09-12T14:19:32.735 INFO:teuthology.orchestra.run.smithi130.stderr: --verbose -- /var/log/ceph/cluster1-client.admin.102791.log 2024-09-12T14:19:32.736 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.107869.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.107869.log.gz 2024-09-12T14:19:32.736 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.86643.log 2024-09-12T14:19:32.737 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.102791.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.102791.log.gz 2024-09-12T14:19:32.737 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.73462.log 2024-09-12T14:19:32.738 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.86643.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.86643.log.gz 2024-09-12T14:19:32.738 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.153075.log 2024-09-12T14:19:32.738 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.73462.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.73462.log.gz 2024-09-12T14:19:32.739 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.53776.log 2024-09-12T14:19:32.739 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.153075.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.153075.log.gz 2024-09-12T14:19:32.739 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.74426.log 2024-09-12T14:19:32.740 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.53776.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.53776.log.gz 2024-09-12T14:19:32.740 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.44050.log 2024-09-12T14:19:32.741 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.74426.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.74426.log.gz 2024-09-12T14:19:32.741 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.135961.log 2024-09-12T14:19:32.741 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.44050.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.44050.log.gz 2024-09-12T14:19:32.742 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.106622.log 2024-09-12T14:19:32.742 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.135961.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.135961.log.gz 2024-09-12T14:19:32.743 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.84770.log 2024-09-12T14:19:32.743 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.106622.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.106622.log.gz 2024-09-12T14:19:32.744 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.127802.log 2024-09-12T14:19:32.744 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.84770.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.84770.log.gz 2024-09-12T14:19:32.745 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.48475.log 2024-09-12T14:19:32.745 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.127802.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.127802.log.gz 2024-09-12T14:19:32.745 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.152171.log 2024-09-12T14:19:32.746 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.48475.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.48475.log.gz 2024-09-12T14:19:32.746 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.50143.log 2024-09-12T14:19:32.747 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.152171.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.152171.log.gz 2024-09-12T14:19:32.747 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.136462.log 2024-09-12T14:19:32.748 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.50143.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.50143.log.gz 2024-09-12T14:19:32.748 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.83286.log 2024-09-12T14:19:32.748 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.136462.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.136462.log.gz 2024-09-12T14:19:32.749 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.139738.log 2024-09-12T14:19:32.750 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.83286.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.150785.log 2024-09-12T14:19:32.750 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.83286.log.gz 2024-09-12T14:19:32.751 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.139738.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.139738.log.gz 2024-09-12T14:19:32.751 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.140735.log 2024-09-12T14:19:32.751 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.150785.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.150785.log.gz 2024-09-12T14:19:32.752 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.152251.log 2024-09-12T14:19:32.752 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.140735.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.140735.log.gz 2024-09-12T14:19:32.752 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.112174.log 2024-09-12T14:19:32.753 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.152251.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.53398.log 2024-09-12T14:19:32.753 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.152251.log.gz 2024-09-12T14:19:32.754 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.112174.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.112174.log.gz 2024-09-12T14:19:32.754 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.37301.log 2024-09-12T14:19:32.755 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.53398.log: 0.0%gzip -- replaced with /var/log/ceph/cluster1-client.mirror.53398.log.gz -5 2024-09-12T14:19:32.755 INFO:teuthology.orchestra.run.smithi130.stderr: --verbose -- /var/log/ceph/cluster1-client.mirror.50772.log 2024-09-12T14:19:32.755 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.37301.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.37301.log.gz 2024-09-12T14:19:32.755 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.106952.log 2024-09-12T14:19:32.756 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.50772.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.50772.log.gz 2024-09-12T14:19:32.756 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.52121.log 2024-09-12T14:19:32.757 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.106952.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.106952.log.gz 2024-09-12T14:19:32.757 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.90921.log 2024-09-12T14:19:32.758 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.52121.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.52121.log.gz 2024-09-12T14:19:32.758 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.101152.log 2024-09-12T14:19:32.759 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.90921.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.90921.log.gz 2024-09-12T14:19:32.759 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.126020.log 2024-09-12T14:19:32.759 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.101152.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.101152.log.gz 2024-09-12T14:19:32.759 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.52520.log 2024-09-12T14:19:32.760 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.126020.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.126020.log.gz 2024-09-12T14:19:32.760 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.66691.log 2024-09-12T14:19:32.761 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.52520.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.52520.log.gz 2024-09-12T14:19:32.761 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.116299.log 2024-09-12T14:19:32.762 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.66691.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.57336.log 2024-09-12T14:19:32.762 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.66691.log.gz 2024-09-12T14:19:32.763 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.116299.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.116299.log.gz 2024-09-12T14:19:32.763 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.83592.log 2024-09-12T14:19:32.764 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.57336.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.57336.log.gz 2024-09-12T14:19:32.764 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.61108.log 2024-09-12T14:19:32.764 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.83592.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.83592.log.gz 2024-09-12T14:19:32.765 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.99367.log 2024-09-12T14:19:32.765 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.61108.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.61108.log.gz 2024-09-12T14:19:32.765 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.47231.log 2024-09-12T14:19:32.766 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.99367.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.99367.log.gz 2024-09-12T14:19:32.766 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.102022.log 2024-09-12T14:19:32.767 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.47231.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.47231.log.gz 2024-09-12T14:19:32.767 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.88910.log 2024-09-12T14:19:32.768 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.102022.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.102022.log.gz 2024-09-12T14:19:32.768 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.108457.log 2024-09-12T14:19:32.769 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.88910.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.88910.log.gz 2024-09-12T14:19:32.769 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.119939.log 2024-09-12T14:19:32.770 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.108457.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.108457.log.gz 2024-09-12T14:19:32.770 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.101674.log 2024-09-12T14:19:32.771 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.119939.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.119939.log.gz 2024-09-12T14:19:32.771 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.38201.log 2024-09-12T14:19:32.772 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.101674.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.63568.log 2024-09-12T14:19:32.772 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.101674.log.gz 2024-09-12T14:19:32.772 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.38201.log: gzip 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.38201.log.gz -5 2024-09-12T14:19:32.773 INFO:teuthology.orchestra.run.smithi130.stderr: --verbose -- /var/log/ceph/cluster1-client.mirror.73964.log 2024-09-12T14:19:32.773 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.63568.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.63568.log.gz 2024-09-12T14:19:32.773 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.142969.log 2024-09-12T14:19:32.775 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.73964.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.73964.log.gz 2024-09-12T14:19:32.775 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.93456.log 2024-09-12T14:19:32.775 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.142969.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.142969.log.gz 2024-09-12T14:19:32.775 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.134236.log 2024-09-12T14:19:32.776 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.93456.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.93456.log.gz 2024-09-12T14:19:32.777 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.44707.log 2024-09-12T14:19:32.777 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.134236.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.134236.log.gz 2024-09-12T14:19:32.778 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.143750.log 2024-09-12T14:19:32.778 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.44707.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.44707.log.gz 2024-09-12T14:19:32.778 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.60479.log 2024-09-12T14:19:32.779 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.143750.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.143750.log.gz 2024-09-12T14:19:32.779 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.106953.log 2024-09-12T14:19:32.779 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.60479.log: 0.0%gzip -- replaced with /var/log/ceph/cluster1-client.mirror.60479.log.gz -5 2024-09-12T14:19:32.779 INFO:teuthology.orchestra.run.smithi130.stderr: --verbose -- /var/log/ceph/cluster1-client.mirror.48085.log 2024-09-12T14:19:32.781 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.106953.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.150694.log 2024-09-12T14:19:32.782 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.106953.log.gz 2024-09-12T14:19:32.782 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.48085.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.48085.log.gz 2024-09-12T14:19:32.782 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.115914.log 2024-09-12T14:19:32.782 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.150694.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.150694.log.gz 2024-09-12T14:19:32.782 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.117085.log 2024-09-12T14:19:32.784 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.115914.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.115914.log.gz 2024-09-12T14:19:32.784 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.93684.log 2024-09-12T14:19:32.784 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.117085.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.117085.log.gz 2024-09-12T14:19:32.784 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.143123.log 2024-09-12T14:19:32.784 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.93684.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.93684.log.gz 2024-09-12T14:19:32.784 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.88048.log 2024-09-12T14:19:32.786 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.143123.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.143123.log.gz 2024-09-12T14:19:32.786 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.78083.log 2024-09-12T14:19:32.786 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.88048.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.88048.log.gz 2024-09-12T14:19:32.786 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.38097.log 2024-09-12T14:19:32.786 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.78083.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.78083.log.gz 2024-09-12T14:19:32.786 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.117676.log 2024-09-12T14:19:32.787 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.38097.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.38097.log.gz 2024-09-12T14:19:32.787 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.74172.log 2024-09-12T14:19:32.788 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.117676.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.117676.log.gz 2024-09-12T14:19:32.788 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.61676.log 2024-09-12T14:19:32.789 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.74172.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.89202.log 2024-09-12T14:19:32.789 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.74172.log.gz 2024-09-12T14:19:32.790 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.61676.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.61676.log.gz 2024-09-12T14:19:32.790 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.41375.log 2024-09-12T14:19:32.790 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.89202.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.89202.log.gz 2024-09-12T14:19:32.791 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.120210.log 2024-09-12T14:19:32.791 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.41375.log: gzip -5 --verbose 0.0% -- /var/log/ceph/cluster1-client.mirror.74781.log -- replaced with /var/log/ceph/cluster1-client.mirror.41375.log.gz 2024-09-12T14:19:32.791 INFO:teuthology.orchestra.run.smithi130.stderr: 2024-09-12T14:19:32.792 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.120210.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.120210.log.gz 2024-09-12T14:19:32.792 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.95476.log 2024-09-12T14:19:32.793 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.74781.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.65201.log 2024-09-12T14:19:32.793 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.74781.log.gz 2024-09-12T14:19:32.794 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.95476.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.95476.log.gz 2024-09-12T14:19:32.794 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.151021.log 2024-09-12T14:19:32.794 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.65201.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.135729.log 2024-09-12T14:19:32.795 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.65201.log.gz 2024-09-12T14:19:32.795 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.47167.log 2024-09-12T14:19:32.795 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.151021.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.151021.log.gz 2024-09-12T14:19:32.796 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.135729.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.135729.log.gz 2024-09-12T14:19:32.796 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.76756.log 2024-09-12T14:19:32.797 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.47167.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.47167.log.gz 2024-09-12T14:19:32.797 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.48413.log 2024-09-12T14:19:32.798 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.76756.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.76756.log.gz 2024-09-12T14:19:32.798 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.86830.log 2024-09-12T14:19:32.799 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.48413.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.48413.log.gz 2024-09-12T14:19:32.799 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.66276.log 2024-09-12T14:19:32.799 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.86830.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.62012.log 2024-09-12T14:19:32.799 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.86830.log.gz 2024-09-12T14:19:32.800 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.66276.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.66276.log.gz 2024-09-12T14:19:32.800 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.56037.log 2024-09-12T14:19:32.801 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.62012.log: 0.0%gzip -- replaced with /var/log/ceph/cluster1-client.mirror.62012.log.gz 2024-09-12T14:19:32.801 INFO:teuthology.orchestra.run.smithi130.stderr: -5 --verbose -- /var/log/ceph/cluster1-client.mirror.136440.log 2024-09-12T14:19:32.802 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.56037.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.56037.log.gz 2024-09-12T14:19:32.802 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.59864.log 2024-09-12T14:19:32.803 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.136440.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.136440.log.gz 2024-09-12T14:19:32.803 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.43080.log 2024-09-12T14:19:32.804 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.59864.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.59864.log.gz 2024-09-12T14:19:32.804 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.94181.log 2024-09-12T14:19:32.804 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.43080.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.43080.log.gz 2024-09-12T14:19:32.805 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.64783.log 2024-09-12T14:19:32.806 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.94181.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.94181.log.gz 2024-09-12T14:19:32.806 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.103713.log 2024-09-12T14:19:32.806 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.64783.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.64783.log.gz 2024-09-12T14:19:32.806 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.127719.log 2024-09-12T14:19:32.807 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.103713.log: gzip -5 --verbose 0.0% -- /var/log/ceph/cluster1-client.mirror.50059.log -- replaced with /var/log/ceph/cluster1-client.admin.103713.log.gz 2024-09-12T14:19:32.807 INFO:teuthology.orchestra.run.smithi130.stderr: 2024-09-12T14:19:32.808 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.127719.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.127719.log.gz 2024-09-12T14:19:32.808 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.69155.log 2024-09-12T14:19:32.808 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.50059.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.50059.log.gz 2024-09-12T14:19:32.808 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.49449.log 2024-09-12T14:19:32.809 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.69155.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.71823.log 2024-09-12T14:19:32.809 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.69155.log.gz 2024-09-12T14:19:32.810 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.49449.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.49449.log.gz 2024-09-12T14:19:32.810 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.109655.log 2024-09-12T14:19:32.812 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.71823.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.102702.log 2024-09-12T14:19:32.813 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.71823.log.gz 2024-09-12T14:19:32.813 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.109655.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.109655.log.gz 2024-09-12T14:19:32.813 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.97372.log 2024-09-12T14:19:32.813 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.102702.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.102702.log.gz 2024-09-12T14:19:32.813 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.148820.log 2024-09-12T14:19:32.814 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.97372.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.97372.log.gz 2024-09-12T14:19:32.814 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.65834.log 2024-09-12T14:19:32.814 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.148820.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.148820.log.gz 2024-09-12T14:19:32.815 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.66215.log 2024-09-12T14:19:32.816 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.65834.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.105866.log 2024-09-12T14:19:32.816 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.65834.log.gz 2024-09-12T14:19:32.816 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.66215.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.66215.log.gz 2024-09-12T14:19:32.816 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.61506.log 2024-09-12T14:19:32.817 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.105866.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.105866.log.gz 2024-09-12T14:19:32.817 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.125934.log 2024-09-12T14:19:32.818 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.61506.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.61506.log.gz 2024-09-12T14:19:32.818 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.112384.log 2024-09-12T14:19:32.819 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.125934.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.125934.log.gz 2024-09-12T14:19:32.819 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.77216.log 2024-09-12T14:19:32.819 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.112384.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.81832.log 2024-09-12T14:19:32.819 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.112384.log.gz 2024-09-12T14:19:32.821 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.77216.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.77216.log.gz 2024-09-12T14:19:32.821 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.118456.log 2024-09-12T14:19:32.821 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.81832.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.81832.log.gz 2024-09-12T14:19:32.821 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.94601.log 2024-09-12T14:19:32.822 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.118456.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.118456.log.gz 2024-09-12T14:19:32.822 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.131359.log 2024-09-12T14:19:32.823 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.94601.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.94601.log.gz 2024-09-12T14:19:32.823 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.80057.log 2024-09-12T14:19:32.824 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.131359.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.131359.log.gz 2024-09-12T14:19:32.824 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.103658.log 2024-09-12T14:19:32.824 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.80057.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.80057.log.gz 2024-09-12T14:19:32.825 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.126935.log 2024-09-12T14:19:32.825 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.103658.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.103658.log.gz 2024-09-12T14:19:32.825 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.116713.log 2024-09-12T14:19:32.826 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.126935.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.126935.log.gz 2024-09-12T14:19:32.826 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.142153.log 2024-09-12T14:19:32.826 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.116713.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.88511.log 2024-09-12T14:19:32.827 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.116713.log.gz 2024-09-12T14:19:32.827 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.142153.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.142153.log.gz 2024-09-12T14:19:32.828 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.102117.log 2024-09-12T14:19:32.828 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.88511.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.88511.log.gz 2024-09-12T14:19:32.829 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.69075.log 2024-09-12T14:19:32.829 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.102117.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.102117.log.gz 2024-09-12T14:19:32.829 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.48632.log 2024-09-12T14:19:32.830 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.69075.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.69075.log.gz 2024-09-12T14:19:32.830 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.41376.log 2024-09-12T14:19:32.831 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.48632.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.48632.log.gz 2024-09-12T14:19:32.831 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.116148.log 2024-09-12T14:19:32.832 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.147244.log 2024-09-12T14:19:32.832 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.41376.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.41376.log.gz 2024-09-12T14:19:32.833 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.116148.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.76775.log 2024-09-12T14:19:32.833 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.116148.log.gz 2024-09-12T14:19:32.833 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.147244.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.147244.log.gz 2024-09-12T14:19:32.834 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.105524.log 2024-09-12T14:19:32.834 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.76775.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.76775.log.gz 2024-09-12T14:19:32.834 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.55061.log 2024-09-12T14:19:32.835 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.105524.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.105524.log.gz 2024-09-12T14:19:32.835 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.104105.log 2024-09-12T14:19:32.836 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.55061.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.55061.log.gz 2024-09-12T14:19:32.836 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.43631.log 2024-09-12T14:19:32.837 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.104105.log: gzip -5 --verbose 0.0% -- -- replaced with /var/log/ceph/cluster1-client.admin.104105.log.gz /var/log/ceph/cluster1-client.mirror.100563.log 2024-09-12T14:19:32.837 INFO:teuthology.orchestra.run.smithi130.stderr: 2024-09-12T14:19:32.837 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.43631.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.43631.log.gz 2024-09-12T14:19:32.837 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.76679.log 2024-09-12T14:19:32.838 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.100563.log: gzip 0.0% -5 -- replaced with /var/log/ceph/cluster1-client.mirror.100563.log.gz --verbose 2024-09-12T14:19:32.838 INFO:teuthology.orchestra.run.smithi130.stderr: -- /var/log/ceph/cluster1-client.mirror.61849.log 2024-09-12T14:19:32.839 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.76679.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.76679.log.gz 2024-09-12T14:19:32.839 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.48124.log 2024-09-12T14:19:32.839 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.61849.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.99008.log 2024-09-12T14:19:32.840 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.61849.log.gz 2024-09-12T14:19:32.840 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.48124.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.48124.log.gz 2024-09-12T14:19:32.841 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.49116.log 2024-09-12T14:19:32.841 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.99008.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.99008.log.gz 2024-09-12T14:19:32.841 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.149346.log 2024-09-12T14:19:32.842 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.49116.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.58150.log 2024-09-12T14:19:32.842 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.49116.log.gz 2024-09-12T14:19:32.843 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.149346.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.149346.log.gz 2024-09-12T14:19:32.843 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.120312.log 2024-09-12T14:19:32.844 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.58150.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.58150.log.gz 2024-09-12T14:19:32.844 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.69667.log 2024-09-12T14:19:32.844 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.120312.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.120312.log.gzgzip 2024-09-12T14:19:32.845 INFO:teuthology.orchestra.run.smithi130.stderr: -5 --verbose -- /var/log/ceph/cluster1-client.admin.91078.log 2024-09-12T14:19:32.845 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.111163.log 2024-09-12T14:19:32.846 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.69667.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.69667.log.gz 2024-09-12T14:19:32.846 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.91078.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.91078.log.gz 2024-09-12T14:19:32.846 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.135958.log 2024-09-12T14:19:32.848 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.111163.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.150565.log 2024-09-12T14:19:32.848 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.111163.log.gz 2024-09-12T14:19:32.848 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.135958.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.135958.log.gz 2024-09-12T14:19:32.848 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.130489.log 2024-09-12T14:19:32.850 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.150565.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.95785.log 2024-09-12T14:19:32.850 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.150565.log.gz 2024-09-12T14:19:32.850 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.130489.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.130489.log.gz 2024-09-12T14:19:32.850 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.103187.log 2024-09-12T14:19:32.851 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.95785.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.95785.log.gz 2024-09-12T14:19:32.851 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.151137.log 2024-09-12T14:19:32.852 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.103187.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.103187.log.gz 2024-09-12T14:19:32.852 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.133737.log 2024-09-12T14:19:32.853 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.151137.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.151137.log.gz 2024-09-12T14:19:32.854 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.90416.log 2024-09-12T14:19:32.854 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.133737.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.133737.log.gz 2024-09-12T14:19:32.854 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.53438.log 2024-09-12T14:19:32.854 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.90416.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.90416.log.gz 2024-09-12T14:19:32.854 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.52977.log 2024-09-12T14:19:32.856 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.53438.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.53438.log.gz 2024-09-12T14:19:32.856 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.35143.log 2024-09-12T14:19:32.856 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.52977.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.52977.log.gz 2024-09-12T14:19:32.856 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.50810.log 2024-09-12T14:19:32.856 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.35143.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.35143.log.gz 2024-09-12T14:19:32.856 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.101758.log 2024-09-12T14:19:32.857 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.50810.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.50810.log.gz 2024-09-12T14:19:32.858 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.144419.log 2024-09-12T14:19:32.858 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.101758.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.101758.log.gz 2024-09-12T14:19:32.858 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.137623.log 2024-09-12T14:19:32.859 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.144419.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.144419.log.gz 2024-09-12T14:19:32.859 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.139568.log 2024-09-12T14:19:32.859 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.137623.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.137623.log.gz 2024-09-12T14:19:32.859 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.147770.log 2024-09-12T14:19:32.860 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.139568.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.139568.log.gz 2024-09-12T14:19:32.860 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.145144.log 2024-09-12T14:19:32.861 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.147770.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.147770.log.gz 2024-09-12T14:19:32.861 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.141851.log 2024-09-12T14:19:32.861 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.145144.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.145144.log.gz 2024-09-12T14:19:32.861 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.39779.log 2024-09-12T14:19:32.862 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.141851.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.141851.log.gz 2024-09-12T14:19:32.862 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.139005.log 2024-09-12T14:19:32.863 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.39779.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.39779.log.gz 2024-09-12T14:19:32.863 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.43021.log 2024-09-12T14:19:32.864 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.139005.log: gzip 0.0% -5 -- replaced with /var/log/ceph/cluster1-client.admin.139005.log.gz 2024-09-12T14:19:32.864 INFO:teuthology.orchestra.run.smithi130.stderr: --verbose -- /var/log/ceph/cluster1-client.mirror.123786.log 2024-09-12T14:19:32.865 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.43021.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.43021.log.gz 2024-09-12T14:19:32.865 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.46656.log 2024-09-12T14:19:32.865 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.123786.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.123786.log.gz 2024-09-12T14:19:32.865 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.116924.log 2024-09-12T14:19:32.866 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.46656.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.46656.log.gz 2024-09-12T14:19:32.866 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.37447.log 2024-09-12T14:19:32.867 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.116924.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.116924.log.gz 2024-09-12T14:19:32.867 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.74830.log 2024-09-12T14:19:32.868 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.37447.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.37447.log.gz 2024-09-12T14:19:32.868 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.86743.log 2024-09-12T14:19:32.869 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.74830.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.74830.log.gz 2024-09-12T14:19:32.869 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.39129.log 2024-09-12T14:19:32.869 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.86743.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.86743.log.gz 2024-09-12T14:19:32.870 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.72454.log 2024-09-12T14:19:32.870 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.39129.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.39129.log.gz 2024-09-12T14:19:32.870 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.126695.log 2024-09-12T14:19:32.871 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.72454.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.72454.log.gz 2024-09-12T14:19:32.871 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.143749.log 2024-09-12T14:19:32.872 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.126695.log: 0.0%gzip -- replaced with /var/log/ceph/cluster1-client.admin.126695.log.gz -5 2024-09-12T14:19:32.872 INFO:teuthology.orchestra.run.smithi130.stderr: --verbose -- /var/log/ceph/cluster1-client.mirror.47494.log 2024-09-12T14:19:32.873 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.143749.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.143749.log.gz 2024-09-12T14:19:32.873 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.94046.log 2024-09-12T14:19:32.873 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.47494.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.47494.log.gz 2024-09-12T14:19:32.874 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.49766.log 2024-09-12T14:19:32.875 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.94046.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.94046.log.gz 2024-09-12T14:19:32.875 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.90297.log 2024-09-12T14:19:32.875 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.49766.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.49766.log.gz 2024-09-12T14:19:32.875 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.65793.log 2024-09-12T14:19:32.877 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.90297.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.90297.log.gz 2024-09-12T14:19:32.877 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.74635.log 2024-09-12T14:19:32.877 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.65793.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.65793.log.gz 2024-09-12T14:19:32.877 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.123114.log 2024-09-12T14:19:32.879 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.74635.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.74635.log.gz 2024-09-12T14:19:32.879 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.134367.log 2024-09-12T14:19:32.879 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.123114.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.123114.log.gz 2024-09-12T14:19:32.879 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.93725.log 2024-09-12T14:19:32.879 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.134367.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.134367.log.gz 2024-09-12T14:19:32.879 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.75772.log 2024-09-12T14:19:32.881 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.93725.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.93725.log.gz 2024-09-12T14:19:32.881 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.75243.log 2024-09-12T14:19:32.881 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.75772.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.75772.log.gz 2024-09-12T14:19:32.882 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.59822.log 2024-09-12T14:19:32.882 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.75243.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.75243.log.gz 2024-09-12T14:19:32.882 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.90833.log 2024-09-12T14:19:32.883 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.59822.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.59822.log.gz 2024-09-12T14:19:32.883 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.150870.log 2024-09-12T14:19:32.883 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.90833.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.90833.log.gz 2024-09-12T14:19:32.883 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.98567.log 2024-09-12T14:19:32.885 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.150870.log: gzip 0.0% -5 -- replaced with /var/log/ceph/cluster1-client.admin.150870.log.gz --verbose 2024-09-12T14:19:32.885 INFO:teuthology.orchestra.run.smithi130.stderr: -- /var/log/ceph/cluster1-client.mirror.51884.log 2024-09-12T14:19:32.885 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.98567.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.98567.log.gz 2024-09-12T14:19:32.885 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.45444.log 2024-09-12T14:19:32.886 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.51884.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.51884.log.gz 2024-09-12T14:19:32.886 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.91958.log 2024-09-12T14:19:32.887 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.45444.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.45444.log.gz 2024-09-12T14:19:32.887 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.128334.log 2024-09-12T14:19:32.887 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.91958.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.91958.log.gz 2024-09-12T14:19:32.887 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.53570.log 2024-09-12T14:19:32.888 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.128334.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.128334.log.gz 2024-09-12T14:19:32.888 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.104771.log 2024-09-12T14:19:32.889 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.53570.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.87966.log 2024-09-12T14:19:32.889 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.53570.log.gz 2024-09-12T14:19:32.890 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.104771.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.104771.log.gz 2024-09-12T14:19:32.891 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.61950.log 2024-09-12T14:19:32.891 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.104826.log 2024-09-12T14:19:32.892 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.87966.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.87966.log.gz 2024-09-12T14:19:32.892 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.61950.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.61950.log.gz 2024-09-12T14:19:32.893 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.111626.log 2024-09-12T14:19:32.893 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.104826.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.104826.log.gz 2024-09-12T14:19:32.893 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.74803.log 2024-09-12T14:19:32.894 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.111626.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.111626.log.gz 2024-09-12T14:19:32.894 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.118611.log 2024-09-12T14:19:32.895 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.74803.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.74803.log.gz 2024-09-12T14:19:32.895 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.52894.log 2024-09-12T14:19:32.895 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.118611.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.118611.log.gz 2024-09-12T14:19:32.895 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.108728.log 2024-09-12T14:19:32.896 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.52894.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.52894.log.gz 2024-09-12T14:19:32.896 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.42450.log 2024-09-12T14:19:32.896 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.108728.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.108728.log.gz 2024-09-12T14:19:32.896 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.44705.log 2024-09-12T14:19:32.897 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.42450.log: 0.0%gzip -- replaced with /var/log/ceph/cluster1-client.mirror.42450.log.gz 2024-09-12T14:19:32.898 INFO:teuthology.orchestra.run.smithi130.stderr: -5 --verbose -- /var/log/ceph/cluster1-client.mirror.147729.log 2024-09-12T14:19:32.898 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.44705.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.44705.log.gz 2024-09-12T14:19:32.898 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.128498.log 2024-09-12T14:19:32.899 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.147729.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.147729.log.gz 2024-09-12T14:19:32.899 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.94802.log 2024-09-12T14:19:32.899 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.128498.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.128498.log.gzgzip 2024-09-12T14:19:32.899 INFO:teuthology.orchestra.run.smithi130.stderr: -5 --verbose -- /var/log/ceph/cluster1-client.mirror.112092.log 2024-09-12T14:19:32.900 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.94802.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.94802.log.gz 2024-09-12T14:19:32.900 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.147602.log 2024-09-12T14:19:32.901 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.112092.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.112092.log.gz 2024-09-12T14:19:32.901 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.91429.log 2024-09-12T14:19:32.902 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.147602.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.147602.log.gz 2024-09-12T14:19:32.902 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.34444.log 2024-09-12T14:19:32.902 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.91429.log: gzip 0.0% -5 -- replaced with /var/log/ceph/cluster1-client.admin.91429.log.gz --verbose 2024-09-12T14:19:32.902 INFO:teuthology.orchestra.run.smithi130.stderr: -- /var/log/ceph/cluster2-client.mirror.92476.log 2024-09-12T14:19:32.903 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.34444.log: gzip 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.34444.log.gz -5 2024-09-12T14:19:32.903 INFO:teuthology.orchestra.run.smithi130.stderr: --verbose -- /var/log/ceph/cluster1-client.admin.127291.log 2024-09-12T14:19:32.904 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.92476.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.92476.log.gz 2024-09-12T14:19:32.904 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.92363.log 2024-09-12T14:19:32.904 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.127291.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.127291.log.gz 2024-09-12T14:19:32.905 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.129135.log 2024-09-12T14:19:32.905 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.92363.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.92363.log.gz 2024-09-12T14:19:32.906 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.62897.log 2024-09-12T14:19:32.907 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.129135.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.129135.log.gz 2024-09-12T14:19:32.907 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.86434.log 2024-09-12T14:19:32.907 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.62897.log: gzip 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.62897.log.gz -5 2024-09-12T14:19:32.907 INFO:teuthology.orchestra.run.smithi130.stderr: --verbose -- /var/log/ceph/cluster1-client.mirror.73755.log 2024-09-12T14:19:32.908 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.86434.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.86434.log.gz 2024-09-12T14:19:32.908 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.63230.log 2024-09-12T14:19:32.909 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.73755.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.73755.log.gz 2024-09-12T14:19:32.909 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.152514.log 2024-09-12T14:19:32.910 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.63230.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.63230.log.gz 2024-09-12T14:19:32.910 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.88046.log 2024-09-12T14:19:32.911 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.152514.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.152514.log.gz 2024-09-12T14:19:32.911 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.97895.log 2024-09-12T14:19:32.911 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.88046.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.88046.log.gz 2024-09-12T14:19:32.911 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.106863.log 2024-09-12T14:19:32.912 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.97895.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.97895.log.gz 2024-09-12T14:19:32.912 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.54677.log 2024-09-12T14:19:32.913 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.106863.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.106863.log.gz 2024-09-12T14:19:32.913 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.106026.log 2024-09-12T14:19:32.914 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.54677.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.54677.log.gz 2024-09-12T14:19:32.914 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.66882.log 2024-09-12T14:19:32.914 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.106026.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.106026.log.gz 2024-09-12T14:19:32.915 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.110304.log 2024-09-12T14:19:32.915 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.66882.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.66882.log.gz 2024-09-12T14:19:32.916 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.102147.log 2024-09-12T14:19:32.916 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.110304.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.110304.log.gz 2024-09-12T14:19:32.917 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.37135.log 2024-09-12T14:19:32.917 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.102147.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.102147.log.gz 2024-09-12T14:19:32.917 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.125190.log 2024-09-12T14:19:32.918 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.37135.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.37135.log.gz 2024-09-12T14:19:32.918 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.109005.log 2024-09-12T14:19:32.919 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.125190.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.125190.log.gz 2024-09-12T14:19:32.919 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.113968.log 2024-09-12T14:19:32.920 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.109005.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.109005.log.gz 2024-09-12T14:19:32.920 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.68231.log 2024-09-12T14:19:32.920 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.113968.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.113968.log.gz 2024-09-12T14:19:32.920 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.109487.log 2024-09-12T14:19:32.921 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.68231.log: gzip 0.0% -5 --verbose -- replaced with /var/log/ceph/cluster2-client.admin.68231.log.gz -- 2024-09-12T14:19:32.921 INFO:teuthology.orchestra.run.smithi130.stderr: /var/log/ceph/cluster1-client.mirror.136746.log 2024-09-12T14:19:32.922 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.109487.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.109487.log.gz 2024-09-12T14:19:32.922 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.140009.log 2024-09-12T14:19:32.922 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.136746.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.136746.log.gz 2024-09-12T14:19:32.923 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.124377.log 2024-09-12T14:19:32.923 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.140009.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.140009.log.gz 2024-09-12T14:19:32.923 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.87543.log 2024-09-12T14:19:32.924 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.124377.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.124377.log.gz 2024-09-12T14:19:32.924 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.120751.log 2024-09-12T14:19:32.925 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.87543.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.87543.log.gz 2024-09-12T14:19:32.925 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.120483.log 2024-09-12T14:19:32.925 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.120751.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.120751.log.gz 2024-09-12T14:19:32.926 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.134268.log 2024-09-12T14:19:32.926 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.120483.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.133105.log 2024-09-12T14:19:32.926 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.120483.log.gz 2024-09-12T14:19:32.927 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.134268.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.134268.log.gz 2024-09-12T14:19:32.927 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.99344.log 2024-09-12T14:19:32.928 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.133105.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.133105.log.gz 2024-09-12T14:19:32.928 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.54409.log 2024-09-12T14:19:32.929 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.99344.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.99344.log.gz 2024-09-12T14:19:32.929 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.63603.log 2024-09-12T14:19:32.930 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.54409.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.54409.log.gz 2024-09-12T14:19:32.930 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.148652.log 2024-09-12T14:19:32.931 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.63603.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.63603.log.gz 2024-09-12T14:19:32.931 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.120837.log 2024-09-12T14:19:32.932 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.148652.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.148652.log.gz 2024-09-12T14:19:32.932 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.130134.log 2024-09-12T14:19:32.932 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.120837.log: 0.0%gzip -- replaced with /var/log/ceph/cluster1-client.mirror.120837.log.gz 2024-09-12T14:19:32.932 INFO:teuthology.orchestra.run.smithi130.stderr: -5 --verbose -- /var/log/ceph/cluster1-client.mirror.52221.log 2024-09-12T14:19:32.933 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.130134.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.130134.log.gzgzip 2024-09-12T14:19:32.934 INFO:teuthology.orchestra.run.smithi130.stderr: -5 --verbose -- /var/log/ceph/cluster1-client.mirror.42460.log 2024-09-12T14:19:32.934 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.52221.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.52221.log.gzgzip 2024-09-12T14:19:32.934 INFO:teuthology.orchestra.run.smithi130.stderr: -5 --verbose -- /var/log/ceph/cluster2-client.admin.56199.log 2024-09-12T14:19:32.934 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.42460.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.42460.log.gz 2024-09-12T14:19:32.935 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.147894.log 2024-09-12T14:19:32.935 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.56199.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.96131.log 2024-09-12T14:19:32.935 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.56199.log.gz 2024-09-12T14:19:32.936 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.147894.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.146661.log 2024-09-12T14:19:32.936 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.147894.log.gz 2024-09-12T14:19:32.937 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.96131.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.96131.log.gz 2024-09-12T14:19:32.937 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.76906.log 2024-09-12T14:19:32.937 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.146661.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.146661.log.gz 2024-09-12T14:19:32.937 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.61151.log 2024-09-12T14:19:32.938 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.76906.log: gzip 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.76906.log.gz -5 2024-09-12T14:19:32.938 INFO:teuthology.orchestra.run.smithi130.stderr: --verbose -- /var/log/ceph/cluster1-client.mirror.86996.log 2024-09-12T14:19:32.939 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.61151.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.61151.log.gz 2024-09-12T14:19:32.939 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.123578.log 2024-09-12T14:19:32.940 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.86996.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.130905.log 2024-09-12T14:19:32.940 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.86996.log.gz 2024-09-12T14:19:32.940 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.123578.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.123578.log.gz 2024-09-12T14:19:32.941 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.90179.log 2024-09-12T14:19:32.941 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.130905.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.130905.log.gz 2024-09-12T14:19:32.942 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.43146.log 2024-09-12T14:19:32.942 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.90179.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.90179.log.gz 2024-09-12T14:19:32.942 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.79922.log 2024-09-12T14:19:32.943 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.43146.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.43146.log.gz 2024-09-12T14:19:32.943 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.46489.log 2024-09-12T14:19:32.944 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.79922.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.79922.log.gz 2024-09-12T14:19:32.944 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.129906.log 2024-09-12T14:19:32.945 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.46489.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.112719.log 2024-09-12T14:19:32.945 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.46489.log.gz 2024-09-12T14:19:32.945 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.129906.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.129906.log.gz 2024-09-12T14:19:32.946 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.34188.log 2024-09-12T14:19:32.947 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.112719.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.112719.log.gz 2024-09-12T14:19:32.947 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.37196.log 2024-09-12T14:19:32.948 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.34188.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.88912.log 2024-09-12T14:19:32.948 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.34188.log.gz 2024-09-12T14:19:32.948 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.37196.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.37196.log.gz 2024-09-12T14:19:32.948 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.146342.log 2024-09-12T14:19:32.949 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.88912.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.88912.log.gz 2024-09-12T14:19:32.949 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.110869.log 2024-09-12T14:19:32.950 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.146342.log: 0.0%gzip -- replaced with /var/log/ceph/cluster1-client.mirror.146342.log.gz 2024-09-12T14:19:32.950 INFO:teuthology.orchestra.run.smithi130.stderr: -5 --verbose -- /var/log/ceph/cluster1-client.mirror.73941.log 2024-09-12T14:19:32.951 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.110869.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.110869.log.gz 2024-09-12T14:19:32.951 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.46261.log 2024-09-12T14:19:32.951 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.73941.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.73941.log.gz 2024-09-12T14:19:32.951 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.112363.log 2024-09-12T14:19:32.952 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.46261.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.46261.log.gz 2024-09-12T14:19:32.952 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.114844.log 2024-09-12T14:19:32.953 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.112363.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.112363.log.gz 2024-09-12T14:19:32.953 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.72812.log 2024-09-12T14:19:32.954 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.114844.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.54197.log 2024-09-12T14:19:32.954 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.114844.log.gz 2024-09-12T14:19:32.954 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.72812.log: 0.0%gzip -- replaced with /var/log/ceph/cluster1-client.mirror.72812.log.gz -5 2024-09-12T14:19:32.954 INFO:teuthology.orchestra.run.smithi130.stderr: --verbose -- /var/log/ceph/cluster1-client.admin.149897.log 2024-09-12T14:19:32.955 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.54197.log: 0.0%gzip -- replaced with /var/log/ceph/cluster1-client.mirror.54197.log.gz 2024-09-12T14:19:32.955 INFO:teuthology.orchestra.run.smithi130.stderr: -5 --verbose -- /var/log/ceph/cluster2-client.mirror.93097.log 2024-09-12T14:19:32.955 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.149897.log: 0.0%gzip -- replaced with /var/log/ceph/cluster1-client.admin.149897.log.gz -5 2024-09-12T14:19:32.956 INFO:teuthology.orchestra.run.smithi130.stderr: --verbose -- /var/log/ceph/cluster1-client.admin.56010.log 2024-09-12T14:19:32.956 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.93097.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.93097.log.gz 2024-09-12T14:19:32.957 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.36927.log 2024-09-12T14:19:32.958 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.56010.log: 0.0%gzip -- replaced with /var/log/ceph/cluster1-client.admin.56010.log.gz 2024-09-12T14:19:32.958 INFO:teuthology.orchestra.run.smithi130.stderr: -5 --verbose -- /var/log/ceph/cluster1-client.mirror.98566.log 2024-09-12T14:19:32.958 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.36927.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.36927.log.gz 2024-09-12T14:19:32.958 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.42991.log 2024-09-12T14:19:32.959 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.98566.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.98566.log.gz 2024-09-12T14:19:32.959 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.148547.log 2024-09-12T14:19:32.960 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.42991.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.42991.log.gz 2024-09-12T14:19:32.960 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.146509.log 2024-09-12T14:19:32.960 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.148547.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.148547.log.gz 2024-09-12T14:19:32.961 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.47517.log 2024-09-12T14:19:32.961 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.146509.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.146509.log.gz 2024-09-12T14:19:32.961 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.69376.log 2024-09-12T14:19:32.963 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.47517.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.47517.log.gz 2024-09-12T14:19:32.963 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.120001.log 2024-09-12T14:19:32.963 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.69376.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.74387.log 2024-09-12T14:19:32.963 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.69376.log.gz 2024-09-12T14:19:32.964 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.120001.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.120001.log.gz 2024-09-12T14:19:32.965 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.38998.log 2024-09-12T14:19:32.965 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.74387.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.74387.log.gz 2024-09-12T14:19:32.965 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.66522.log 2024-09-12T14:19:32.965 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.38998.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.72891.log 2024-09-12T14:19:32.966 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.38998.log.gz 2024-09-12T14:19:32.967 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.66522.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.66522.log.gz 2024-09-12T14:19:32.967 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.79524.log 2024-09-12T14:19:32.967 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.72891.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.72891.log.gzgzip 2024-09-12T14:19:32.967 INFO:teuthology.orchestra.run.smithi130.stderr: -5 --verbose -- /var/log/ceph/cluster1-client.admin.149528.log 2024-09-12T14:19:32.967 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.79524.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.79524.log.gz 2024-09-12T14:19:32.968 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.44862.log 2024-09-12T14:19:32.968 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.149528.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.149528.log.gz 2024-09-12T14:19:32.968 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.44728.log 2024-09-12T14:19:32.969 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.44862.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.102164.log 2024-09-12T14:19:32.969 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.44862.log.gz 2024-09-12T14:19:32.970 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.44728.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.77502.log 2024-09-12T14:19:32.970 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.44728.log.gz 2024-09-12T14:19:32.971 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.102164.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.102164.log.gz 2024-09-12T14:19:32.971 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.109906.log 2024-09-12T14:19:32.972 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.77502.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.77502.log.gz 2024-09-12T14:19:32.972 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.132685.log 2024-09-12T14:19:32.972 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.109906.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.109906.log.gz 2024-09-12T14:19:32.972 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.143648.log 2024-09-12T14:19:32.973 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.132685.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.122310.log 2024-09-12T14:19:32.973 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.132685.log.gz 2024-09-12T14:19:32.974 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.143648.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.143648.log.gz 2024-09-12T14:19:32.974 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.121908.log 2024-09-12T14:19:32.975 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.122310.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.122310.log.gz 2024-09-12T14:19:32.975 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.90967.log 2024-09-12T14:19:32.976 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.121908.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.121908.log.gz 2024-09-12T14:19:32.976 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.90367.log 2024-09-12T14:19:32.976 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.90967.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.90967.log.gz 2024-09-12T14:19:32.977 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.142663.log 2024-09-12T14:19:32.977 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.90367.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.90367.log.gz 2024-09-12T14:19:32.978 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.101024.log 2024-09-12T14:19:32.978 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.142663.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.142663.log.gz 2024-09-12T14:19:32.978 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.96800.log 2024-09-12T14:19:32.979 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.101024.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.101024.log.gzgzip 2024-09-12T14:19:32.979 INFO:teuthology.orchestra.run.smithi130.stderr: -5 --verbose -- /var/log/ceph/cluster1-client.mirror.65855.log 2024-09-12T14:19:32.980 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.96800.log: gzip 0.0% -5 -- replaced with /var/log/ceph/cluster1-client.mirror.96800.log.gz --verbose -- 2024-09-12T14:19:32.980 INFO:teuthology.orchestra.run.smithi130.stderr: /var/log/ceph/cluster1-client.mirror.41797.log 2024-09-12T14:19:32.981 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.117541.log 2024-09-12T14:19:32.981 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.65855.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.65855.log.gz 2024-09-12T14:19:32.981 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.41797.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.41797.log.gz 2024-09-12T14:19:32.982 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.141489.log 2024-09-12T14:19:32.982 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.117541.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.117541.log.gz 2024-09-12T14:19:32.983 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.67651.log 2024-09-12T14:19:32.984 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.141489.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.141489.log.gz 2024-09-12T14:19:32.984 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.99871.log 2024-09-12T14:19:32.984 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.67651.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.67651.log.gz 2024-09-12T14:19:32.984 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.69983.log 2024-09-12T14:19:32.985 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.99871.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.99871.log.gz 2024-09-12T14:19:32.985 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.42459.log 2024-09-12T14:19:32.986 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.69983.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.69983.log.gz 2024-09-12T14:19:32.986 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.145670.log 2024-09-12T14:19:32.987 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.42459.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.42459.log.gz 2024-09-12T14:19:32.987 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.97497.log 2024-09-12T14:19:32.988 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.145670.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.145670.log.gz 2024-09-12T14:19:32.988 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.56947.log 2024-09-12T14:19:32.989 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.97497.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.78831.log 2024-09-12T14:19:32.989 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.97497.log.gz 2024-09-12T14:19:32.989 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.56947.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.56947.log.gz 2024-09-12T14:19:32.990 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.76233.log 2024-09-12T14:19:32.990 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.78831.log: gzip 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.78831.log.gz -5 2024-09-12T14:19:32.990 INFO:teuthology.orchestra.run.smithi130.stderr: --verbose -- /var/log/ceph/cluster1-client.mirror.108037.log 2024-09-12T14:19:32.991 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.76233.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.84640.log 2024-09-12T14:19:32.991 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.76233.log.gz 2024-09-12T14:19:32.992 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.108037.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.108037.log.gz 2024-09-12T14:19:32.992 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.129031.log 2024-09-12T14:19:32.993 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.84640.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.84640.log.gz 2024-09-12T14:19:32.993 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.146659.log 2024-09-12T14:19:32.993 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.129031.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.129031.log.gz 2024-09-12T14:19:32.993 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.43607.log 2024-09-12T14:19:32.994 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.146659.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.146659.log.gz 2024-09-12T14:19:32.994 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.83198.log 2024-09-12T14:19:32.995 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.43607.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.43607.log.gz 2024-09-12T14:19:32.995 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.61042.log 2024-09-12T14:19:32.996 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.83198.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.83198.log.gz 2024-09-12T14:19:32.996 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.69687.log 2024-09-12T14:19:32.997 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.61042.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.56640.log 2024-09-12T14:19:32.997 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.61042.log.gz 2024-09-12T14:19:32.998 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.69687.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.69687.log.gz 2024-09-12T14:19:32.998 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.38308.log 2024-09-12T14:19:32.998 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.56640.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.56640.log.gz 2024-09-12T14:19:32.999 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.54299.log 2024-09-12T14:19:32.999 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.38308.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.38308.log.gz 2024-09-12T14:19:32.999 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.106405.log 2024-09-12T14:19:33.000 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.54299.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.54299.log.gz 2024-09-12T14:19:33.000 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.72662.log 2024-09-12T14:19:33.001 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.106405.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.106405.log.gz 2024-09-12T14:19:33.001 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.51109.log 2024-09-12T14:19:33.002 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.72662.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.72662.log.gz 2024-09-12T14:19:33.002 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.113201.log 2024-09-12T14:19:33.002 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.51109.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.51109.log.gz 2024-09-12T14:19:33.003 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.153028.log 2024-09-12T14:19:33.003 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.113201.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.113201.log.gz 2024-09-12T14:19:33.003 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.154383.log 2024-09-12T14:19:33.004 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.153028.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.153028.log.gz 2024-09-12T14:19:33.004 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.84415.log 2024-09-12T14:19:33.005 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.154383.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.154383.log.gz 2024-09-12T14:19:33.005 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.123072.log 2024-09-12T14:19:33.006 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.84415.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.50249.log 2024-09-12T14:19:33.006 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.84415.log.gz 2024-09-12T14:19:33.006 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.123072.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.46334.log 2024-09-12T14:19:33.006 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.123072.log.gz 2024-09-12T14:19:33.007 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.50249.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.50249.log.gz 2024-09-12T14:19:33.007 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.40790.log 2024-09-12T14:19:33.008 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.46334.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.46334.log.gz 2024-09-12T14:19:33.008 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.125394.log 2024-09-12T14:19:33.009 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.40790.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.40790.log.gz 2024-09-12T14:19:33.009 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.144637.log 2024-09-12T14:19:33.009 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.125394.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.125394.log.gz 2024-09-12T14:19:33.009 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.110805.log 2024-09-12T14:19:33.010 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.144637.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.144637.log.gz 2024-09-12T14:19:33.010 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.111795.log 2024-09-12T14:19:33.011 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.110805.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.110805.log.gz 2024-09-12T14:19:33.012 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.106311.log 2024-09-12T14:19:33.012 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.111795.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.111795.log.gz 2024-09-12T14:19:33.012 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.94971.log 2024-09-12T14:19:33.013 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.106311.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.106311.log.gz 2024-09-12T14:19:33.013 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.108144.log 2024-09-12T14:19:33.013 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.94971.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.94971.log.gz 2024-09-12T14:19:33.013 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.87228.log 2024-09-12T14:19:33.014 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.108144.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.108144.log.gz 2024-09-12T14:19:33.014 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.70237.log 2024-09-12T14:19:33.015 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.87228.log: gzip 0.0% -5 --verbose -- replaced with /var/log/ceph/cluster1-client.mirror.87228.log.gz -- 2024-09-12T14:19:33.015 INFO:teuthology.orchestra.run.smithi130.stderr: /var/log/ceph/cluster1-client.admin.106754.log 2024-09-12T14:19:33.016 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.70237.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.70237.log.gz 2024-09-12T14:19:33.016 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.152800.log 2024-09-12T14:19:33.016 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.106754.log: gzip -5 --verbose 0.0% -- /var/log/ceph/cluster1-client.mirror.53774.log -- replaced with /var/log/ceph/cluster1-client.admin.106754.log.gz 2024-09-12T14:19:33.016 INFO:teuthology.orchestra.run.smithi130.stderr: 2024-09-12T14:19:33.017 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.152800.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.92045.log 2024-09-12T14:19:33.017 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.152800.log.gz 2024-09-12T14:19:33.019 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.53774.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.53774.log.gz 2024-09-12T14:19:33.020 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.96546.log 2024-09-12T14:19:33.020 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.92045.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.92045.log.gz 2024-09-12T14:19:33.020 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.86199.log 2024-09-12T14:19:33.022 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.96546.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.96546.log.gz 2024-09-12T14:19:33.022 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.112953.log 2024-09-12T14:19:33.022 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.86199.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.86199.log.gz 2024-09-12T14:19:33.022 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.96484.log 2024-09-12T14:19:33.022 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.112953.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.112953.log.gz 2024-09-12T14:19:33.022 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.142179.log 2024-09-12T14:19:33.022 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.96484.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.96484.log.gz 2024-09-12T14:19:33.022 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.137838.log 2024-09-12T14:19:33.024 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.142179.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.48547.log 2024-09-12T14:19:33.024 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.142179.log.gz 2024-09-12T14:19:33.024 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.137838.log: 0.0%gzip -- replaced with /var/log/ceph/cluster1-client.mirror.137838.log.gz 2024-09-12T14:19:33.024 INFO:teuthology.orchestra.run.smithi130.stderr: -5 --verbose -- /var/log/ceph/cluster1-client.admin.103805.log 2024-09-12T14:19:33.024 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.48547.log: gzip 0.0% -5 -- replaced with /var/log/ceph/cluster1-client.mirror.48547.log.gz --verbose 2024-09-12T14:19:33.024 INFO:teuthology.orchestra.run.smithi130.stderr: -- /var/log/ceph/cluster1-client.mirror.144807.log 2024-09-12T14:19:33.026 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.103805.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.103805.log.gz 2024-09-12T14:19:33.026 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.143539.log 2024-09-12T14:19:33.026 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.144807.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.144807.log.gz 2024-09-12T14:19:33.026 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.104273.log 2024-09-12T14:19:33.027 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.143539.log: gzip 0.0% -5 -- replaced with /var/log/ceph/cluster1-client.mirror.143539.log.gz --verbose 2024-09-12T14:19:33.027 INFO:teuthology.orchestra.run.smithi130.stderr: -- /var/log/ceph/cluster1-client.mirror.52351.log 2024-09-12T14:19:33.027 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.104273.log: gzip 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.104273.log.gz -5 2024-09-12T14:19:33.027 INFO:teuthology.orchestra.run.smithi130.stderr: --verbose -- /var/log/ceph/cluster1-client.mirror.137602.log 2024-09-12T14:19:33.029 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.52351.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.52351.log.gz 2024-09-12T14:19:33.029 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.98944.log 2024-09-12T14:19:33.029 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.137602.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.137602.log.gz 2024-09-12T14:19:33.029 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.119641.log 2024-09-12T14:19:33.029 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.98944.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.98944.log.gz 2024-09-12T14:19:33.029 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.135663.log 2024-09-12T14:19:33.030 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.119641.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.66801.log 2024-09-12T14:19:33.030 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.119641.log.gz 2024-09-12T14:19:33.030 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.135663.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.135663.log.gz 2024-09-12T14:19:33.031 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.38666.log 2024-09-12T14:19:33.031 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.66801.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.66801.log.gz 2024-09-12T14:19:33.032 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.112743.log 2024-09-12T14:19:33.033 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.38666.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.38666.log.gz 2024-09-12T14:19:33.033 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.123220.log 2024-09-12T14:19:33.033 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.112743.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.112743.log.gz 2024-09-12T14:19:33.034 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.107782.log 2024-09-12T14:19:33.034 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.123220.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.123220.log.gz 2024-09-12T14:19:33.034 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.126099.log 2024-09-12T14:19:33.035 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.107782.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.112257.log 2024-09-12T14:19:33.035 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.107782.log.gz 2024-09-12T14:19:33.036 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.126099.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.126099.log.gz 2024-09-12T14:19:33.036 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.98462.log 2024-09-12T14:19:33.037 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.112257.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.112257.log.gz 2024-09-12T14:19:33.037 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.124519.log 2024-09-12T14:19:33.038 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.98462.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.98462.log.gz 2024-09-12T14:19:33.038 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.114621.log 2024-09-12T14:19:33.039 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.124519.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.124519.log.gz 2024-09-12T14:19:33.039 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.102309.log 2024-09-12T14:19:33.039 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.114621.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.114621.log.gz 2024-09-12T14:19:33.039 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.134719.log 2024-09-12T14:19:33.040 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.102309.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.102309.log.gz 2024-09-12T14:19:33.041 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.110408.log 2024-09-12T14:19:33.041 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.134719.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.134719.log.gz 2024-09-12T14:19:33.041 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.147135.log 2024-09-12T14:19:33.042 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.110408.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.110408.log.gz 2024-09-12T14:19:33.042 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.48175.log 2024-09-12T14:19:33.043 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.147135.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.147135.log.gz 2024-09-12T14:19:33.043 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.48410.log 2024-09-12T14:19:33.044 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.48175.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.48175.log.gz 2024-09-12T14:19:33.044 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.86370.log 2024-09-12T14:19:33.045 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.48410.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.48410.log.gz 2024-09-12T14:19:33.045 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.78655.log 2024-09-12T14:19:33.046 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.86370.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.86370.log.gz 2024-09-12T14:19:33.046 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.51068.log 2024-09-12T14:19:33.047 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.78655.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.78655.log.gz 2024-09-12T14:19:33.047 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.71296.log 2024-09-12T14:19:33.047 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.51068.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.51068.log.gzgzip 2024-09-12T14:19:33.047 INFO:teuthology.orchestra.run.smithi130.stderr: -5 --verbose -- /var/log/ceph/cluster1-client.mirror.35632.log 2024-09-12T14:19:33.048 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.71296.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.71296.log.gz 2024-09-12T14:19:33.048 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.124060.log 2024-09-12T14:19:33.049 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.35632.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.35632.log.gz 2024-09-12T14:19:33.049 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.71126.log 2024-09-12T14:19:33.050 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.124060.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.124060.log.gz 2024-09-12T14:19:33.050 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.105953.log 2024-09-12T14:19:33.051 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.71126.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.71126.log.gz 2024-09-12T14:19:33.051 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.66557.log 2024-09-12T14:19:33.051 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.105953.log: gzip 0.0% -5 --verbose -- replaced with /var/log/ceph/cluster1-client.admin.105953.log.gz -- 2024-09-12T14:19:33.051 INFO:teuthology.orchestra.run.smithi130.stderr: /var/log/ceph/cluster2-client.mirror.44596.log 2024-09-12T14:19:33.052 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.66557.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.141208.log 2024-09-12T14:19:33.052 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.66557.log.gz 2024-09-12T14:19:33.053 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.44596.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.44596.log.gz 2024-09-12T14:19:33.053 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.121387.log 2024-09-12T14:19:33.054 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.141208.log: gzip -5 --verbose 0.0% -- -- replaced with /var/log/ceph/cluster1-client.admin.141208.log.gz /var/log/ceph/cluster1-client.admin.128075.log 2024-09-12T14:19:33.054 INFO:teuthology.orchestra.run.smithi130.stderr: 2024-09-12T14:19:33.055 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.121387.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.121387.log.gz 2024-09-12T14:19:33.055 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.132015.log 2024-09-12T14:19:33.056 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.128075.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.128075.log.gz 2024-09-12T14:19:33.056 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.104321.log 2024-09-12T14:19:33.057 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.132015.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.132015.log.gz 2024-09-12T14:19:33.057 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.37864.log 2024-09-12T14:19:33.058 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.104321.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.104321.log.gz 2024-09-12T14:19:33.058 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.137824.log 2024-09-12T14:19:33.059 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.37864.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.37864.log.gz 2024-09-12T14:19:33.059 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.72242.log 2024-09-12T14:19:33.059 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.137824.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.57783.log 2024-09-12T14:19:33.060 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.137824.log.gz 2024-09-12T14:19:33.061 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.72242.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.72242.log.gz 2024-09-12T14:19:33.061 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.41504.log 2024-09-12T14:19:33.061 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.57783.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.57783.log.gz 2024-09-12T14:19:33.062 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.113321.log 2024-09-12T14:19:33.062 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.41504.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.41504.log.gz 2024-09-12T14:19:33.063 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.145712.log 2024-09-12T14:19:33.064 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.113321.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.113321.log.gz 2024-09-12T14:19:33.064 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.41799.log 2024-09-12T14:19:33.064 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.145712.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.145712.log.gz 2024-09-12T14:19:33.064 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.81374.log 2024-09-12T14:19:33.066 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.41799.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.145602.log 2024-09-12T14:19:33.066 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.41799.log.gz 2024-09-12T14:19:33.066 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.81374.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.81374.log.gz 2024-09-12T14:19:33.066 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.127154.log 2024-09-12T14:19:33.067 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.145602.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.145602.log.gz 2024-09-12T14:19:33.067 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.128975.log 2024-09-12T14:19:33.067 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.127154.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.127154.log.gz 2024-09-12T14:19:33.068 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.119025.log 2024-09-12T14:19:33.068 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.128975.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.128975.log.gz 2024-09-12T14:19:33.068 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.95609.log 2024-09-12T14:19:33.069 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.119025.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.119025.log.gz 2024-09-12T14:19:33.069 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.72705.log 2024-09-12T14:19:33.070 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.95609.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.95609.log.gz 2024-09-12T14:19:33.070 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.63692.log 2024-09-12T14:19:33.071 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.72705.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.72705.log.gz 2024-09-12T14:19:33.071 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.140697.log 2024-09-12T14:19:33.072 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.63692.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.63692.log.gz 2024-09-12T14:19:33.072 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.49956.log 2024-09-12T14:19:33.072 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.140697.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.140697.log.gz 2024-09-12T14:19:33.072 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.101525.log 2024-09-12T14:19:33.073 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.49956.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.96278.log 2024-09-12T14:19:33.074 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.49956.log.gz 2024-09-12T14:19:33.074 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.101525.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.101525.log.gz 2024-09-12T14:19:33.074 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.152893.log 2024-09-12T14:19:33.075 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.96278.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.96278.log.gz 2024-09-12T14:19:33.075 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.102923.log 2024-09-12T14:19:33.076 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.152893.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.152893.log.gz 2024-09-12T14:19:33.076 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.78601.log 2024-09-12T14:19:33.077 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.102923.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.102923.log.gzgzip 2024-09-12T14:19:33.077 INFO:teuthology.orchestra.run.smithi130.stderr: -5 --verbose -- /var/log/ceph/cluster1-client.mirror.136611.log 2024-09-12T14:19:33.078 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.78601.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.78601.log.gz 2024-09-12T14:19:33.078 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.125560.log 2024-09-12T14:19:33.078 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.136611.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.150127.log 2024-09-12T14:19:33.078 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.136611.log.gz 2024-09-12T14:19:33.079 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.125560.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.125560.log.gz 2024-09-12T14:19:33.079 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.91451.log 2024-09-12T14:19:33.080 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.150127.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.150127.log.gz 2024-09-12T14:19:33.080 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.47431.log 2024-09-12T14:19:33.081 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.91451.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.91451.log.gz 2024-09-12T14:19:33.081 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.92585.log 2024-09-12T14:19:33.082 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.47431.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.47431.log.gz 2024-09-12T14:19:33.082 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.99657.log 2024-09-12T14:19:33.083 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.92585.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.92585.log.gz 2024-09-12T14:19:33.083 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.153443.log 2024-09-12T14:19:33.083 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.99657.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.99657.log.gz 2024-09-12T14:19:33.084 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.132287.log 2024-09-12T14:19:33.085 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.153443.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.153443.log.gz 2024-09-12T14:19:33.085 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.145353.log 2024-09-12T14:19:33.086 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.132287.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.132287.log.gz 2024-09-12T14:19:33.086 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.122835.log 2024-09-12T14:19:33.086 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.145353.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.145353.log.gz 2024-09-12T14:19:33.087 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.62559.log 2024-09-12T14:19:33.087 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.122835.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.122835.log.gz 2024-09-12T14:19:33.087 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.108895.log 2024-09-12T14:19:33.088 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.62559.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.62559.log.gz 2024-09-12T14:19:33.088 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.56865.log 2024-09-12T14:19:33.089 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.108895.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.108895.log.gz 2024-09-12T14:19:33.089 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.55658.log 2024-09-12T14:19:33.090 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.56865.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.56865.log.gz 2024-09-12T14:19:33.090 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.70325.log 2024-09-12T14:19:33.091 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.55658.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.55658.log.gz 2024-09-12T14:19:33.091 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.97012.log 2024-09-12T14:19:33.092 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.70325.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.65625.log 2024-09-12T14:19:33.092 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.70325.log.gz 2024-09-12T14:19:33.093 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.97012.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.97012.log.gz 2024-09-12T14:19:33.093 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.132600.log 2024-09-12T14:19:33.094 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.65625.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.65625.log.gz 2024-09-12T14:19:33.094 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.67826.log 2024-09-12T14:19:33.094 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.132600.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.132600.log.gz 2024-09-12T14:19:33.095 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.123848.log 2024-09-12T14:19:33.095 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.67826.log: gzip 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.67826.log.gz -5 2024-09-12T14:19:33.095 INFO:teuthology.orchestra.run.smithi130.stderr: --verbose -- /var/log/ceph/cluster1-client.mirror.100185.log 2024-09-12T14:19:33.096 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.123848.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.123848.log.gz 2024-09-12T14:19:33.096 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.103231.log 2024-09-12T14:19:33.097 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.100185.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.117495.log 2024-09-12T14:19:33.097 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.100185.log.gz 2024-09-12T14:19:33.098 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.103231.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.103231.log.gz 2024-09-12T14:19:33.098 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.147557.log 2024-09-12T14:19:33.099 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.117495.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.117495.log.gz 2024-09-12T14:19:33.099 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.73398.log 2024-09-12T14:19:33.099 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.147557.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.147557.log.gz 2024-09-12T14:19:33.100 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.124248.log 2024-09-12T14:19:33.100 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.73398.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.73398.log.gz 2024-09-12T14:19:33.101 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.35802.log 2024-09-12T14:19:33.101 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.124248.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.124248.log.gz 2024-09-12T14:19:33.102 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.130421.log 2024-09-12T14:19:33.102 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.35802.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.35802.log.gz 2024-09-12T14:19:33.103 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.135095.log 2024-09-12T14:19:33.103 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.130421.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.130421.log.gz 2024-09-12T14:19:33.103 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.53693.log 2024-09-12T14:19:33.104 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.135095.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.135095.log.gz 2024-09-12T14:19:33.104 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.87710.log 2024-09-12T14:19:33.105 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.53693.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.53693.log.gz 2024-09-12T14:19:33.105 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.40307.log 2024-09-12T14:19:33.106 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.87710.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.87710.log.gz 2024-09-12T14:19:33.106 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.107319.log 2024-09-12T14:19:33.106 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.40307.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.40307.log.gz 2024-09-12T14:19:33.107 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.89771.log 2024-09-12T14:19:33.108 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.107319.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.107319.log.gz 2024-09-12T14:19:33.108 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.143707.log 2024-09-12T14:19:33.108 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.89771.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.89771.log.gz 2024-09-12T14:19:33.109 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.125869.log 2024-09-12T14:19:33.109 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.143707.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.143707.log.gz 2024-09-12T14:19:33.110 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.101508.log 2024-09-12T14:19:33.110 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.125869.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.125869.log.gz 2024-09-12T14:19:33.110 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.40096.log 2024-09-12T14:19:33.111 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.101508.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.101508.log.gz 2024-09-12T14:19:33.111 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.81439.log 2024-09-12T14:19:33.112 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.40096.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.40096.log.gz 2024-09-12T14:19:33.112 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.68728.log 2024-09-12T14:19:33.113 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.81439.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.92868.log 2024-09-12T14:19:33.113 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.81439.log.gz 2024-09-12T14:19:33.114 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.68728.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.68728.log.gz 2024-09-12T14:19:33.114 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.40956.log 2024-09-12T14:19:33.115 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.92868.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.111863.log 2024-09-12T14:19:33.115 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.92868.log.gz 2024-09-12T14:19:33.115 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.40956.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.40956.log.gz 2024-09-12T14:19:33.115 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.142617.log 2024-09-12T14:19:33.116 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.111863.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.111863.log.gz 2024-09-12T14:19:33.116 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.120881.log 2024-09-12T14:19:33.117 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.142617.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.142617.log.gz 2024-09-12T14:19:33.117 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.133736.log 2024-09-12T14:19:33.118 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.120881.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.94933.log 2024-09-12T14:19:33.118 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.120881.log.gz 2024-09-12T14:19:33.119 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.133736.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.133736.log.gz 2024-09-12T14:19:33.119 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.41038.log 2024-09-12T14:19:33.119 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.94933.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.94933.log.gz 2024-09-12T14:19:33.120 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.81657.log 2024-09-12T14:19:33.120 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.41038.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.41038.log.gz 2024-09-12T14:19:33.120 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.66106.log 2024-09-12T14:19:33.121 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.81657.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.81657.log.gz 2024-09-12T14:19:33.121 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.108291.log 2024-09-12T14:19:33.122 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.66106.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.66106.log.gz 2024-09-12T14:19:33.123 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.110934.log 2024-09-12T14:19:33.123 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.108291.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.108291.log.gz 2024-09-12T14:19:33.123 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.132956.log 2024-09-12T14:19:33.124 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.110934.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.110934.log.gz 2024-09-12T14:19:33.124 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.95610.log 2024-09-12T14:19:33.125 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.132956.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.132956.log.gz 2024-09-12T14:19:33.125 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.113199.log 2024-09-12T14:19:33.126 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.95610.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.95610.log.gz 2024-09-12T14:19:33.126 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.53479.log 2024-09-12T14:19:33.126 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.113199.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.113199.log.gz 2024-09-12T14:19:33.126 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.110618.log 2024-09-12T14:19:33.127 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.53479.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.53479.log.gz 2024-09-12T14:19:33.127 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.110725.log 2024-09-12T14:19:33.128 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.110618.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.110618.log.gz 2024-09-12T14:19:33.128 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.44873.log 2024-09-12T14:19:33.128 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.110725.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.110725.log.gz 2024-09-12T14:19:33.129 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.91343.log 2024-09-12T14:19:33.130 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.44873.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.44873.log.gz 2024-09-12T14:19:33.130 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.143584.log 2024-09-12T14:19:33.130 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.91343.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.91343.log.gz 2024-09-12T14:19:33.131 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.75433.log 2024-09-12T14:19:33.131 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.143584.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.143584.log.gz 2024-09-12T14:19:33.132 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.97580.log 2024-09-12T14:19:33.133 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.75433.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.75433.log.gz 2024-09-12T14:19:33.133 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.67298.log 2024-09-12T14:19:33.133 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.97580.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.97580.log.gz 2024-09-12T14:19:33.133 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.45617.log 2024-09-12T14:19:33.134 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.67298.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.67298.log.gz 2024-09-12T14:19:33.134 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.70640.log 2024-09-12T14:19:33.135 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.45617.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.45617.log.gz 2024-09-12T14:19:33.135 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.108515.log 2024-09-12T14:19:33.136 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.70640.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.70640.log.gz 2024-09-12T14:19:33.136 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.121937.log 2024-09-12T14:19:33.137 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.108515.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.108515.log.gz 2024-09-12T14:19:33.137 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.39399.log 2024-09-12T14:19:33.138 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.121937.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.121937.log.gz 2024-09-12T14:19:33.138 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.76610.log 2024-09-12T14:19:33.138 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.39399.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.39399.log.gz 2024-09-12T14:19:33.139 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.49011.log 2024-09-12T14:19:33.139 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.76610.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.76610.log.gz 2024-09-12T14:19:33.140 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.46240.log 2024-09-12T14:19:33.140 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.49011.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.49011.log.gz 2024-09-12T14:19:33.140 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.129363.log 2024-09-12T14:19:33.141 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.46240.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.66971.log 2024-09-12T14:19:33.141 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.46240.log.gz 2024-09-12T14:19:33.142 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.129363.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.129363.log.gz 2024-09-12T14:19:33.142 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.73165.log 2024-09-12T14:19:33.143 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.66971.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.66971.log.gz 2024-09-12T14:19:33.143 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.112851.log 2024-09-12T14:19:33.144 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.73165.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.73165.log.gz 2024-09-12T14:19:33.144 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.118811.log 2024-09-12T14:19:33.145 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.112851.log: 0.0%gzip -- replaced with /var/log/ceph/cluster1-client.mirror.112851.log.gz 2024-09-12T14:19:33.145 INFO:teuthology.orchestra.run.smithi130.stderr: -5 --verbose -- /var/log/ceph/cluster2-client.mirror.70693.log 2024-09-12T14:19:33.145 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.118811.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.118811.log.gz 2024-09-12T14:19:33.146 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.78799.log 2024-09-12T14:19:33.146 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.70693.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.70693.log.gz 2024-09-12T14:19:33.147 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.88425.log 2024-09-12T14:19:33.147 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.78799.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.78799.log.gz 2024-09-12T14:19:33.148 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.78687.log 2024-09-12T14:19:33.148 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.88425.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.88425.log.gz 2024-09-12T14:19:33.149 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.123661.log 2024-09-12T14:19:33.149 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.78687.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.78687.log.gz 2024-09-12T14:19:33.149 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.150540.log 2024-09-12T14:19:33.150 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.123661.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.123661.log.gz 2024-09-12T14:19:33.150 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.133460.log 2024-09-12T14:19:33.151 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.150540.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.150540.log.gz 2024-09-12T14:19:33.151 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.48758.log 2024-09-12T14:19:33.152 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.133460.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.133460.log.gz 2024-09-12T14:19:33.152 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.105117.log 2024-09-12T14:19:33.153 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.48758.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.48758.log.gz 2024-09-12T14:19:33.153 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.95239.log 2024-09-12T14:19:33.153 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.105117.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.105117.log.gz 2024-09-12T14:19:33.154 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.107870.log 2024-09-12T14:19:33.154 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.95239.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.75242.log 2024-09-12T14:19:33.154 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.95239.log.gz 2024-09-12T14:19:33.155 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.107870.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.107870.log.gz 2024-09-12T14:19:33.155 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.97577.log 2024-09-12T14:19:33.156 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.75242.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.75242.log.gz 2024-09-12T14:19:33.156 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.57770.log 2024-09-12T14:19:33.157 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.97577.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.97577.log.gz 2024-09-12T14:19:33.157 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.138609.log 2024-09-12T14:19:33.158 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.57770.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.57770.log.gz 2024-09-12T14:19:33.158 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.35422.log 2024-09-12T14:19:33.158 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.138609.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.138609.log.gz 2024-09-12T14:19:33.159 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.44245.log 2024-09-12T14:19:33.160 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.35422.log: 0.0%gzip -- replaced with /var/log/ceph/cluster1-client.admin.35422.log.gz 2024-09-12T14:19:33.160 INFO:teuthology.orchestra.run.smithi130.stderr: -5 --verbose -- /var/log/ceph/cluster1-client.mirror.143188.log 2024-09-12T14:19:33.160 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.44245.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.44245.log.gz 2024-09-12T14:19:33.161 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.50479.log 2024-09-12T14:19:33.161 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.143188.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.143188.log.gz 2024-09-12T14:19:33.162 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.62053.log 2024-09-12T14:19:33.162 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.50479.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.50479.log.gz 2024-09-12T14:19:33.163 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.92269.log 2024-09-12T14:19:33.163 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.62053.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.62053.log.gz 2024-09-12T14:19:33.163 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.45918.log 2024-09-12T14:19:33.164 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.92269.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.92269.log.gz 2024-09-12T14:19:33.164 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.133649.log 2024-09-12T14:19:33.165 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.45918.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.45918.log.gz 2024-09-12T14:19:33.165 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.146023.log 2024-09-12T14:19:33.166 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.133649.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.133649.log.gz 2024-09-12T14:19:33.166 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.41147.log 2024-09-12T14:19:33.167 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.146023.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.146023.log.gz 2024-09-12T14:19:33.167 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.137203.log 2024-09-12T14:19:33.167 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.41147.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.41147.log.gz 2024-09-12T14:19:33.168 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.144979.log 2024-09-12T14:19:33.169 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.137203.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.137203.log.gz 2024-09-12T14:19:33.169 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.38478.log 2024-09-12T14:19:33.169 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.144979.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.144979.log.gz 2024-09-12T14:19:33.169 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.88633.log 2024-09-12T14:19:33.170 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.38478.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.38478.log.gz 2024-09-12T14:19:33.170 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.141167.log 2024-09-12T14:19:33.171 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.88633.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.88633.log.gz 2024-09-12T14:19:33.171 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.88762.log 2024-09-12T14:19:33.172 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.141167.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.141167.log.gz 2024-09-12T14:19:33.172 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.87038.log 2024-09-12T14:19:33.172 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.88762.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.88762.log.gz 2024-09-12T14:19:33.172 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.151852.log 2024-09-12T14:19:33.173 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.87038.log: gzip 0.0% -5 --verbose -- replaced with /var/log/ceph/cluster1-client.mirror.87038.log.gz -- 2024-09-12T14:19:33.173 INFO:teuthology.orchestra.run.smithi130.stderr: /var/log/ceph/cluster1-client.mirror.59910.log 2024-09-12T14:19:33.174 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.151852.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.151852.log.gz 2024-09-12T14:19:33.174 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.110978.log 2024-09-12T14:19:33.174 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.59910.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.59910.log.gz 2024-09-12T14:19:33.175 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.149388.log 2024-09-12T14:19:33.175 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.110978.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.110978.log.gz 2024-09-12T14:19:33.175 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.68190.log 2024-09-12T14:19:33.176 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.149388.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.149388.log.gz 2024-09-12T14:19:33.176 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.120362.log 2024-09-12T14:19:33.177 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.68190.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.68190.log.gz 2024-09-12T14:19:33.177 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.90246.log 2024-09-12T14:19:33.177 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.120362.log: 0.0%gzip -- replaced with /var/log/ceph/cluster1-client.mirror.120362.log.gz -5 2024-09-12T14:19:33.178 INFO:teuthology.orchestra.run.smithi130.stderr: --verbose -- /var/log/ceph/cluster1-client.mirror.132076.log 2024-09-12T14:19:33.178 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.90246.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.90246.log.gz 2024-09-12T14:19:33.178 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.152488.log 2024-09-12T14:19:33.179 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.132076.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.132076.log.gz 2024-09-12T14:19:33.179 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.50439.log 2024-09-12T14:19:33.180 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.152488.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.152488.log.gz 2024-09-12T14:19:33.180 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.55636.log 2024-09-12T14:19:33.181 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.50439.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.50439.log.gz 2024-09-12T14:19:33.181 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.87968.log 2024-09-12T14:19:33.181 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.55636.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.55636.log.gz 2024-09-12T14:19:33.181 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.148151.log 2024-09-12T14:19:33.182 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.87968.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.108083.log 2024-09-12T14:19:33.182 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.87968.log.gz 2024-09-12T14:19:33.183 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.148151.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.148151.log.gz 2024-09-12T14:19:33.183 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.76148.log 2024-09-12T14:19:33.183 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.108083.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.108083.log.gz 2024-09-12T14:19:33.184 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.125937.log 2024-09-12T14:19:33.184 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.76148.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.76148.log.gz 2024-09-12T14:19:33.184 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.139006.log 2024-09-12T14:19:33.185 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.125937.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.125937.log.gz 2024-09-12T14:19:33.185 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.145671.log 2024-09-12T14:19:33.186 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.139006.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.139006.log.gz 2024-09-12T14:19:33.186 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.139900.log 2024-09-12T14:19:33.186 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.145671.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.145671.log.gz 2024-09-12T14:19:33.187 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.65562.log 2024-09-12T14:19:33.187 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.139900.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.139900.log.gz 2024-09-12T14:19:33.187 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.88385.log 2024-09-12T14:19:33.188 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.65562.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.65562.log.gz 2024-09-12T14:19:33.188 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.117407.log 2024-09-12T14:19:33.189 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.88385.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.88385.log.gz 2024-09-12T14:19:33.189 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.73399.log 2024-09-12T14:19:33.189 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.117407.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.117407.log.gz 2024-09-12T14:19:33.190 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.122584.log 2024-09-12T14:19:33.190 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.73399.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.73399.log.gz 2024-09-12T14:19:33.190 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.84414.log 2024-09-12T14:19:33.191 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.122584.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.154036.log 2024-09-12T14:19:33.191 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.122584.log.gz 2024-09-12T14:19:33.192 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.84414.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.84414.log.gz 2024-09-12T14:19:33.192 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.83199.log 2024-09-12T14:19:33.193 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.154036.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.154036.log.gz 2024-09-12T14:19:33.193 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.147996.log 2024-09-12T14:19:33.194 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.83199.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.83199.log.gz 2024-09-12T14:19:33.194 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.122153.log 2024-09-12T14:19:33.194 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.147996.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.147996.log.gz 2024-09-12T14:19:33.194 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.47075.log 2024-09-12T14:19:33.195 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.122153.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.122153.log.gz 2024-09-12T14:19:33.195 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.60481.log 2024-09-12T14:19:33.196 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.47075.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.47075.log.gz 2024-09-12T14:19:33.196 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.105912.log 2024-09-12T14:19:33.197 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.60481.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.60481.log.gz 2024-09-12T14:19:33.197 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.100141.log 2024-09-12T14:19:33.197 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.105912.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.105912.log.gz 2024-09-12T14:19:33.197 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.74761.log 2024-09-12T14:19:33.198 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.100141.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.100141.log.gz 2024-09-12T14:19:33.199 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.54939.log 2024-09-12T14:19:33.199 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.74761.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.74761.log.gz 2024-09-12T14:19:33.199 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.66881.log 2024-09-12T14:19:33.199 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.54939.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.54939.log.gz 2024-09-12T14:19:33.199 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.83745.log 2024-09-12T14:19:33.200 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.66881.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.66881.log.gz 2024-09-12T14:19:33.200 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.91801.log 2024-09-12T14:19:33.201 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.83745.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.83745.log.gz 2024-09-12T14:19:33.201 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.38140.log 2024-09-12T14:19:33.201 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.91801.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.91801.log.gz 2024-09-12T14:19:33.202 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.36202.log 2024-09-12T14:19:33.202 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.38140.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.38140.log.gz 2024-09-12T14:19:33.202 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.84961.log 2024-09-12T14:19:33.203 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.36202.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.36202.log.gz 2024-09-12T14:19:33.203 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.68230.log 2024-09-12T14:19:33.204 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.84961.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.84961.log.gz 2024-09-12T14:19:33.204 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.123135.log 2024-09-12T14:19:33.205 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.68230.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.68230.log.gz 2024-09-12T14:19:33.205 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.101193.log 2024-09-12T14:19:33.206 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.123135.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.123135.log.gz 2024-09-12T14:19:33.206 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.152873.log 2024-09-12T14:19:33.206 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.101193.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.101193.log.gz 2024-09-12T14:19:33.206 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.126917.log 2024-09-12T14:19:33.208 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.152873.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.128408.log 2024-09-12T14:19:33.208 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.152873.log.gz 2024-09-12T14:19:33.208 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.126917.log: gzip 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.126917.log.gz -5 2024-09-12T14:19:33.208 INFO:teuthology.orchestra.run.smithi130.stderr: --verbose -- /var/log/ceph/cluster1-client.mirror.89308.log 2024-09-12T14:19:33.209 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.128408.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.128408.log.gz 2024-09-12T14:19:33.209 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.37032.log 2024-09-12T14:19:33.209 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.89308.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.89308.log.gz 2024-09-12T14:19:33.209 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.67540.log 2024-09-12T14:19:33.210 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.37032.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.37032.log.gz 2024-09-12T14:19:33.210 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.144003.log 2024-09-12T14:19:33.211 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.67540.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.67540.log.gz 2024-09-12T14:19:33.211 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.123746.log 2024-09-12T14:19:33.211 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.144003.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.144003.log.gz 2024-09-12T14:19:33.212 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.83657.log 2024-09-12T14:19:33.212 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.123746.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.123746.log.gz 2024-09-12T14:19:33.212 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.144869.log 2024-09-12T14:19:33.213 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.83657.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.83657.log.gz 2024-09-12T14:19:33.213 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.41649.log 2024-09-12T14:19:33.214 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.144869.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.144869.log.gz 2024-09-12T14:19:33.214 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.100751.log 2024-09-12T14:19:33.215 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.41649.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.41649.log.gz 2024-09-12T14:19:33.215 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.76066.log 2024-09-12T14:19:33.215 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.100751.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.100751.log.gzgzip 2024-09-12T14:19:33.215 INFO:teuthology.orchestra.run.smithi130.stderr: -5 --verbose -- /var/log/ceph/cluster1-client.mirror.101988.log 2024-09-12T14:19:33.216 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.76066.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.123535.log 2024-09-12T14:19:33.216 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.76066.log.gz 2024-09-12T14:19:33.217 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.101988.log: 0.0%gzip -- replaced with /var/log/ceph/cluster1-client.mirror.101988.log.gz 2024-09-12T14:19:33.217 INFO:teuthology.orchestra.run.smithi130.stderr: -5 --verbose -- /var/log/ceph/cluster1-client.admin.54963.log 2024-09-12T14:19:33.218 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.123535.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.104220.log 2024-09-12T14:19:33.218 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.123535.log.gz 2024-09-12T14:19:33.218 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.54963.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.54963.log.gz 2024-09-12T14:19:33.219 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.73058.log 2024-09-12T14:19:33.219 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.104220.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.104220.log.gz 2024-09-12T14:19:33.220 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.122544.log 2024-09-12T14:19:33.220 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.73058.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.39089.log 2024-09-12T14:19:33.220 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.73058.log.gz 2024-09-12T14:19:33.221 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.122544.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.122544.log.gz 2024-09-12T14:19:33.221 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.147220.log 2024-09-12T14:19:33.222 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.39089.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.39089.log.gz 2024-09-12T14:19:33.222 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.150320.log 2024-09-12T14:19:33.223 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.147220.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.147220.log.gz 2024-09-12T14:19:33.223 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.130753.log 2024-09-12T14:19:33.224 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.150320.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.39696.log 2024-09-12T14:19:33.224 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.150320.log.gz 2024-09-12T14:19:33.225 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.130753.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.130753.log.gz 2024-09-12T14:19:33.225 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.146865.log 2024-09-12T14:19:33.225 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.39696.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.39696.log.gz 2024-09-12T14:19:33.225 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.100691.log 2024-09-12T14:19:33.226 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.146865.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.146865.log.gz 2024-09-12T14:19:33.226 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.139954.log 2024-09-12T14:19:33.227 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.100691.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.100691.log.gz 2024-09-12T14:19:33.227 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.40053.log 2024-09-12T14:19:33.227 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.139954.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.139954.log.gz 2024-09-12T14:19:33.228 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.113417.log 2024-09-12T14:19:33.228 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.40053.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.40053.log.gz 2024-09-12T14:19:33.228 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.75333.log 2024-09-12T14:19:33.229 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.113417.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.113417.log.gz 2024-09-12T14:19:33.229 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.127476.log 2024-09-12T14:19:33.230 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.75333.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.75333.log.gz 2024-09-12T14:19:33.230 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.105410.log 2024-09-12T14:19:33.230 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.127476.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.127476.log.gz 2024-09-12T14:19:33.231 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.115845.log 2024-09-12T14:19:33.231 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.105410.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.105410.log.gz 2024-09-12T14:19:33.231 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.86220.log 2024-09-12T14:19:33.232 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.115845.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.115845.log.gz 2024-09-12T14:19:33.232 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.74341.log 2024-09-12T14:19:33.233 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.86220.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.86220.log.gz 2024-09-12T14:19:33.233 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.90212.log 2024-09-12T14:19:33.234 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.74341.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.74341.log.gz 2024-09-12T14:19:33.234 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.113141.log 2024-09-12T14:19:33.234 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.90212.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.90212.log.gz 2024-09-12T14:19:33.235 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.125218.log 2024-09-12T14:19:33.235 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.113141.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.113141.log.gz 2024-09-12T14:19:33.235 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.65122.log 2024-09-12T14:19:33.236 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.125218.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.125218.log.gz 2024-09-12T14:19:33.236 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.130527.log 2024-09-12T14:19:33.237 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.65122.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.65122.log.gz 2024-09-12T14:19:33.237 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.92649.log 2024-09-12T14:19:33.237 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.130527.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.130527.log.gz 2024-09-12T14:19:33.238 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.66711.log 2024-09-12T14:19:33.238 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.92649.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.92649.log.gz 2024-09-12T14:19:33.238 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.98418.log 2024-09-12T14:19:33.239 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.66711.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.82341.log 2024-09-12T14:19:33.239 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.66711.log.gz 2024-09-12T14:19:33.240 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.98418.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.98418.log.gz 2024-09-12T14:19:33.240 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.52493.log 2024-09-12T14:19:33.240 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.82341.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.82341.log.gz 2024-09-12T14:19:33.241 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.103912.log 2024-09-12T14:19:33.241 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.52493.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.52493.log.gz 2024-09-12T14:19:33.241 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.108666.log 2024-09-12T14:19:33.242 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.103912.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.103912.log.gz 2024-09-12T14:19:33.242 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.57272.log 2024-09-12T14:19:33.243 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.108666.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.108666.log.gz 2024-09-12T14:19:33.243 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.127553.log 2024-09-12T14:19:33.243 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.57272.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.57272.log.gz 2024-09-12T14:19:33.244 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.135540.log 2024-09-12T14:19:33.244 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.127553.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.127553.log.gz 2024-09-12T14:19:33.244 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.38632.log 2024-09-12T14:19:33.245 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.135540.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.152779.log 2024-09-12T14:19:33.245 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.135540.log.gz 2024-09-12T14:19:33.246 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.38632.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.38632.log.gz 2024-09-12T14:19:33.246 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.49345.log 2024-09-12T14:19:33.247 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.152779.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.152779.log.gz 2024-09-12T14:19:33.247 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.60857.log 2024-09-12T14:19:33.248 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.49345.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.49345.log.gz 2024-09-12T14:19:33.248 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.90637.log 2024-09-12T14:19:33.248 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.60857.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.60857.log.gz 2024-09-12T14:19:33.249 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.101423.log 2024-09-12T14:19:33.249 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.90637.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.90637.log.gz 2024-09-12T14:19:33.249 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.48846.log 2024-09-12T14:19:33.250 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.101423.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.101423.log.gz 2024-09-12T14:19:33.250 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.143583.log 2024-09-12T14:19:33.251 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.48846.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.48846.log.gz 2024-09-12T14:19:33.251 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.49870.log 2024-09-12T14:19:33.251 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.143583.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.143583.log.gz 2024-09-12T14:19:33.252 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.136609.log 2024-09-12T14:19:33.253 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.49870.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.49870.log.gz 2024-09-12T14:19:33.253 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.92481.log 2024-09-12T14:19:33.253 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.136609.log: 0.0%gzip -- replaced with /var/log/ceph/cluster1-client.mirror.136609.log.gz 2024-09-12T14:19:33.253 INFO:teuthology.orchestra.run.smithi130.stderr: -5 --verbose -- /var/log/ceph/cluster2-client.admin.127556.log 2024-09-12T14:19:33.254 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.92481.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.92481.log.gz 2024-09-12T14:19:33.254 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.79122.log 2024-09-12T14:19:33.255 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.127556.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.127556.log.gz 2024-09-12T14:19:33.255 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.141530.log 2024-09-12T14:19:33.255 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.79122.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.79122.log.gz 2024-09-12T14:19:33.255 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.40348.log 2024-09-12T14:19:33.256 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.141530.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.141530.log.gz 2024-09-12T14:19:33.256 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.97769.log 2024-09-12T14:19:33.257 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.40348.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.40348.log.gz 2024-09-12T14:19:33.257 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.141349.log 2024-09-12T14:19:33.258 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.97769.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.97769.log.gz 2024-09-12T14:19:33.258 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.100308.log 2024-09-12T14:19:33.259 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.141349.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.141349.log.gz 2024-09-12T14:19:33.259 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.58358.log 2024-09-12T14:19:33.259 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.100308.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.100308.log.gz 2024-09-12T14:19:33.259 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.144678.log 2024-09-12T14:19:33.260 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.58358.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.58358.log.gz 2024-09-12T14:19:33.260 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.101977.log 2024-09-12T14:19:33.261 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.144678.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.144678.log.gz 2024-09-12T14:19:33.261 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.127884.log 2024-09-12T14:19:33.262 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.101977.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.101977.log.gz 2024-09-12T14:19:33.262 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.102584.log 2024-09-12T14:19:33.263 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.127884.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.127884.log.gz 2024-09-12T14:19:33.263 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.100056.log 2024-09-12T14:19:33.264 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.102584.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.102584.log.gz 2024-09-12T14:19:33.264 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.122858.log 2024-09-12T14:19:33.264 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.100056.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.100056.log.gz 2024-09-12T14:19:33.265 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.151687.log 2024-09-12T14:19:33.265 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.122858.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.122858.log.gz 2024-09-12T14:19:33.266 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.89772.log 2024-09-12T14:19:33.266 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.151687.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.151687.log.gz 2024-09-12T14:19:33.267 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.52517.log 2024-09-12T14:19:33.267 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.89772.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.74253.log 2024-09-12T14:19:33.267 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.89772.log.gz 2024-09-12T14:19:33.268 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.52517.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.52517.log.gz 2024-09-12T14:19:33.268 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.132472.log 2024-09-12T14:19:33.269 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.74253.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.74253.log.gz 2024-09-12T14:19:33.269 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.100434.log 2024-09-12T14:19:33.270 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.132472.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.132472.log.gz 2024-09-12T14:19:33.270 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.46375.log 2024-09-12T14:19:33.271 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.100434.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.100434.log.gz 2024-09-12T14:19:33.271 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.88172.log 2024-09-12T14:19:33.272 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.46375.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.46375.log.gz 2024-09-12T14:19:33.272 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.67539.log 2024-09-12T14:19:33.272 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.88172.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.88172.log.gz 2024-09-12T14:19:33.273 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.107212.log 2024-09-12T14:19:33.273 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.67539.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.67539.log.gz 2024-09-12T14:19:33.273 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.63271.log 2024-09-12T14:19:33.274 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.107212.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.107212.log.gz 2024-09-12T14:19:33.274 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.96632.log 2024-09-12T14:19:33.274 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.63271.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.63271.log.gz 2024-09-12T14:19:33.275 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2.log 2024-09-12T14:19:33.275 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.96632.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.96632.log.gz 2024-09-12T14:19:33.276 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.81584.log 2024-09-12T14:19:33.276 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.81812.log 2024-09-12T14:19:33.277 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.81584.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.81584.log.gz 2024-09-12T14:19:33.277 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.144355.log 2024-09-12T14:19:33.278 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.81812.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.81812.log.gz 2024-09-12T14:19:33.278 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.73438.log 2024-09-12T14:19:33.279 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.144355.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.144355.log.gz 2024-09-12T14:19:33.279 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.105342.log 2024-09-12T14:19:33.280 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.73438.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.73438.log.gz 2024-09-12T14:19:33.280 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.68729.log 2024-09-12T14:19:33.281 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.105342.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.105342.log.gz 2024-09-12T14:19:33.281 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.149155.log 2024-09-12T14:19:33.282 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.68729.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.68729.log.gz 2024-09-12T14:19:33.282 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.106275.log 2024-09-12T14:19:33.283 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.149155.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.149155.log.gz 2024-09-12T14:19:33.283 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.122689.log 2024-09-12T14:19:33.284 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.106275.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.106275.log.gz 2024-09-12T14:19:33.284 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.128327.log 2024-09-12T14:19:33.284 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.122689.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.122689.log.gz 2024-09-12T14:19:33.285 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.114756.log 2024-09-12T14:19:33.285 INFO:teuthology.orchestra.run.smithi130.stderr: 88.0% -- replaced with /var/log/ceph/cluster2.log.gz 2024-09-12T14:19:33.286 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.128327.log: 0.0%gzip -- replaced with /var/log/ceph/cluster2-client.mirror.128327.log.gz 2024-09-12T14:19:33.286 INFO:teuthology.orchestra.run.smithi130.stderr: -5 --verbose -- /var/log/ceph/cluster1-client.mirror.96724.log 2024-09-12T14:19:33.286 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.114756.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.114756.log.gz 2024-09-12T14:19:33.287 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.43181.log 2024-09-12T14:19:33.287 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.96724.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.96724.log.gz 2024-09-12T14:19:33.287 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.41919.log 2024-09-12T14:19:33.288 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.43181.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.43181.log.gz 2024-09-12T14:19:33.288 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.111293.log 2024-09-12T14:19:33.289 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.41919.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.41919.log.gz 2024-09-12T14:19:33.289 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.41505.log 2024-09-12T14:19:33.290 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.111293.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.111293.log.gz 2024-09-12T14:19:33.290 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.81401.log 2024-09-12T14:19:33.291 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.41505.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.41505.log.gz 2024-09-12T14:19:33.291 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.43896.log 2024-09-12T14:19:33.291 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.81401.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.81401.log.gz 2024-09-12T14:19:33.292 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.63314.log 2024-09-12T14:19:33.292 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.43896.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.43896.log.gz 2024-09-12T14:19:33.293 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.74049.log 2024-09-12T14:19:33.293 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.63314.log: 0.0%gzip -- replaced with /var/log/ceph/cluster1-client.mirror.63314.log.gz -5 --verbose 2024-09-12T14:19:33.293 INFO:teuthology.orchestra.run.smithi130.stderr: -- /var/log/ceph/cluster1-client.admin.68007.log 2024-09-12T14:19:33.294 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.74049.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.74049.log.gz 2024-09-12T14:19:33.294 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.147057.log 2024-09-12T14:19:33.295 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.68007.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.68007.log.gz 2024-09-12T14:19:33.295 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.78775.log 2024-09-12T14:19:33.296 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.147057.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.147057.log.gz 2024-09-12T14:19:33.296 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.93403.log 2024-09-12T14:19:33.297 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.78775.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.78775.log.gz 2024-09-12T14:19:33.297 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.112468.log 2024-09-12T14:19:33.298 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.93403.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.93403.log.gz 2024-09-12T14:19:33.298 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.120211.log 2024-09-12T14:19:33.299 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.112468.log: gzip -5 0.0% --verbose -- replaced with /var/log/ceph/cluster1-client.mirror.112468.log.gz -- 2024-09-12T14:19:33.299 INFO:teuthology.orchestra.run.smithi130.stderr: /var/log/ceph/cluster2-client.admin.105052.log 2024-09-12T14:19:33.300 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.120211.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.120211.log.gz 2024-09-12T14:19:33.300 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.70995.log 2024-09-12T14:19:33.301 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.105052.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.105052.log.gz 2024-09-12T14:19:33.301 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.66994.log 2024-09-12T14:19:33.302 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.70995.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.70995.log.gz 2024-09-12T14:19:33.302 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.124412.log 2024-09-12T14:19:33.302 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.66994.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.130987.log 2024-09-12T14:19:33.303 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.66994.log.gz 2024-09-12T14:19:33.303 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.124412.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.124412.log.gz 2024-09-12T14:19:33.303 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.48237.log 2024-09-12T14:19:33.304 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.130987.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.130987.log.gz 2024-09-12T14:19:33.304 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.75286.log 2024-09-12T14:19:33.305 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.48237.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.48237.log.gz 2024-09-12T14:19:33.305 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.120002.log 2024-09-12T14:19:33.306 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.75286.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.75286.log.gz 2024-09-12T14:19:33.306 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.60856.log 2024-09-12T14:19:33.307 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.120002.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.120002.log.gz 2024-09-12T14:19:33.307 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.63941.log 2024-09-12T14:19:33.308 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.60856.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.60856.log.gzgzip 2024-09-12T14:19:33.308 INFO:teuthology.orchestra.run.smithi130.stderr: -5 --verbose -- /var/log/ceph/cluster1-client.mirror.111334.log 2024-09-12T14:19:33.309 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.63941.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.63941.log.gz 2024-09-12T14:19:33.309 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.110535.log 2024-09-12T14:19:33.309 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.111334.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.111334.log.gz 2024-09-12T14:19:33.310 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.85737.log 2024-09-12T14:19:33.310 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.110535.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.110535.log.gz 2024-09-12T14:19:33.311 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.145350.log 2024-09-12T14:19:33.312 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.85737.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.85737.log.gz 2024-09-12T14:19:33.312 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.80561.log 2024-09-12T14:19:33.312 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.145350.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.145350.log.gz 2024-09-12T14:19:33.313 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.102044.log 2024-09-12T14:19:33.313 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.80561.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.80561.log.gz 2024-09-12T14:19:33.314 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.125921.log 2024-09-12T14:19:33.314 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.102044.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.102044.log.gz 2024-09-12T14:19:33.315 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.141094.log 2024-09-12T14:19:33.315 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.125921.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.125921.log.gz 2024-09-12T14:19:33.316 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.60793.log 2024-09-12T14:19:33.316 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.141094.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.141094.log.gz 2024-09-12T14:19:33.316 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.67429.log 2024-09-12T14:19:33.317 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.60793.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.118168.log 2024-09-12T14:19:33.317 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.60793.log.gz 2024-09-12T14:19:33.318 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.67429.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.67429.log.gz 2024-09-12T14:19:33.318 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.147180.log 2024-09-12T14:19:33.319 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.118168.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.118168.log.gz 2024-09-12T14:19:33.319 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.60437.log 2024-09-12T14:19:33.320 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.147180.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.147180.log.gz 2024-09-12T14:19:33.320 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.121551.log 2024-09-12T14:19:33.321 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.60437.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.60437.log.gz 2024-09-12T14:19:33.321 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.147769.log 2024-09-12T14:19:33.321 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.121551.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.45905.log 2024-09-12T14:19:33.322 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.121551.log.gz 2024-09-12T14:19:33.322 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.147769.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.147769.log.gz 2024-09-12T14:19:33.322 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.143962.log 2024-09-12T14:19:33.323 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.45905.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.45905.log.gz 2024-09-12T14:19:33.323 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.147579.log 2024-09-12T14:19:33.324 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.143962.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.143962.log.gz 2024-09-12T14:19:33.324 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.96503.log 2024-09-12T14:19:33.325 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.147579.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.147579.log.gz 2024-09-12T14:19:33.325 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.53895.log 2024-09-12T14:19:33.325 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.96503.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.96503.log.gz 2024-09-12T14:19:33.326 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.46469.log 2024-09-12T14:19:33.327 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.53895.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.53895.log.gz 2024-09-12T14:19:33.327 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.140226.log 2024-09-12T14:19:33.327 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.46469.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.46469.log.gz 2024-09-12T14:19:33.328 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.58036.log 2024-09-12T14:19:33.328 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.140226.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.140226.log.gz 2024-09-12T14:19:33.328 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.113268.log 2024-09-12T14:19:33.329 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.58036.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.125839.log 2024-09-12T14:19:33.329 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.58036.log.gz 2024-09-12T14:19:33.330 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.113268.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.113268.log.gz 2024-09-12T14:19:33.330 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.119615.log 2024-09-12T14:19:33.331 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.125839.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.125839.log.gz 2024-09-12T14:19:33.331 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.54388.log 2024-09-12T14:19:33.332 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.119615.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.119615.log.gz 2024-09-12T14:19:33.332 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.49699.log 2024-09-12T14:19:33.333 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.54388.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.54388.log.gz 2024-09-12T14:19:33.333 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.47337.log 2024-09-12T14:19:33.334 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.49699.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.49699.log.gz 2024-09-12T14:19:33.334 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.95891.log 2024-09-12T14:19:33.334 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.47337.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.47337.log.gz 2024-09-12T14:19:33.334 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.87058.log 2024-09-12T14:19:33.335 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.95891.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.95891.log.gz 2024-09-12T14:19:33.335 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.131142.log 2024-09-12T14:19:33.336 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.87058.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.87058.log.gz 2024-09-12T14:19:33.336 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.134112.log 2024-09-12T14:19:33.337 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.131142.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.131142.log.gz 2024-09-12T14:19:33.337 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.81771.log 2024-09-12T14:19:33.337 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.134112.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.134112.log.gz 2024-09-12T14:19:33.338 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.88424.log 2024-09-12T14:19:33.339 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.81771.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.139340.log 2024-09-12T14:19:33.339 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.81771.log.gz 2024-09-12T14:19:33.339 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.88424.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.88424.log.gz 2024-09-12T14:19:33.339 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.77597.log 2024-09-12T14:19:33.340 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.139340.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.139340.log.gz 2024-09-12T14:19:33.340 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.144169.log 2024-09-12T14:19:33.341 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.77597.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.77597.log.gz 2024-09-12T14:19:33.342 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.57691.log 2024-09-12T14:19:33.342 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.144169.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.144169.log.gz 2024-09-12T14:19:33.343 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.117267.log 2024-09-12T14:19:33.343 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.57691.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.57691.log.gz 2024-09-12T14:19:33.343 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.58431.log 2024-09-12T14:19:33.344 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.117267.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.117267.log.gz 2024-09-12T14:19:33.344 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.141235.log 2024-09-12T14:19:33.345 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.58431.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.48411.log 2024-09-12T14:19:33.345 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.58431.log.gz 2024-09-12T14:19:33.345 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.141235.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.141235.log.gz 2024-09-12T14:19:33.345 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.110246.log 2024-09-12T14:19:33.346 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.48411.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.48411.log.gz 2024-09-12T14:19:33.346 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.67187.log 2024-09-12T14:19:33.347 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.110246.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.110246.log.gz 2024-09-12T14:19:33.347 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.111522.log 2024-09-12T14:19:33.348 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.67187.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.129637.log 2024-09-12T14:19:33.348 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.67187.log.gz 2024-09-12T14:19:33.349 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.111522.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.111522.log.gz 2024-09-12T14:19:33.349 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.126986.log 2024-09-12T14:19:33.349 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.129637.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.129637.log.gz 2024-09-12T14:19:33.350 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.74574.log 2024-09-12T14:19:33.350 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.126986.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.126986.log.gz 2024-09-12T14:19:33.351 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.41187.log 2024-09-12T14:19:33.351 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.74574.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.74574.log.gz 2024-09-12T14:19:33.352 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.104323.log 2024-09-12T14:19:33.352 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.41187.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.41187.log.gz 2024-09-12T14:19:33.353 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.111546.log 2024-09-12T14:19:33.353 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.104323.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.104323.log.gz 2024-09-12T14:19:33.354 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.137389.log 2024-09-12T14:19:33.354 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.111546.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.111546.log.gz 2024-09-12T14:19:33.354 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.68441.log 2024-09-12T14:19:33.355 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.137389.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.137389.log.gz 2024-09-12T14:19:33.355 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.35586.log 2024-09-12T14:19:33.356 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.68441.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.68441.log.gz 2024-09-12T14:19:33.356 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.103543.log 2024-09-12T14:19:33.357 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.35586.log: gzip -5 0.0% --verbose -- -- replaced with /var/log/ceph/cluster1-client.admin.35586.log.gz /var/log/ceph/cluster1-client.mirror.49386.log 2024-09-12T14:19:33.357 INFO:teuthology.orchestra.run.smithi130.stderr: 2024-09-12T14:19:33.358 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.103543.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.103543.log.gz 2024-09-12T14:19:33.358 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.64873.log 2024-09-12T14:19:33.359 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.49386.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.49386.log.gz 2024-09-12T14:19:33.359 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.59188.log 2024-09-12T14:19:33.359 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.64873.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.64873.log.gz 2024-09-12T14:19:33.360 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.134578.log 2024-09-12T14:19:33.360 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.59188.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.59188.log.gz 2024-09-12T14:19:33.361 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.34938.log 2024-09-12T14:19:33.361 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.134578.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.134578.log.gz 2024-09-12T14:19:33.361 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.134930.log 2024-09-12T14:19:33.362 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.34938.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.114976.log 2024-09-12T14:19:33.362 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.34938.log.gz 2024-09-12T14:19:33.363 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.134930.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.134930.log.gz 2024-09-12T14:19:33.363 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.141375.log 2024-09-12T14:19:33.364 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.114976.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.114976.log.gz 2024-09-12T14:19:33.364 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.112593.log 2024-09-12T14:19:33.364 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.141375.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.141375.log.gz 2024-09-12T14:19:33.365 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.56225.log 2024-09-12T14:19:33.365 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.112593.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.112593.log.gz 2024-09-12T14:19:33.366 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.114800.log 2024-09-12T14:19:33.366 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.56225.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.56225.log.gz 2024-09-12T14:19:33.367 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.57203.log 2024-09-12T14:19:33.367 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.114800.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.114800.log.gz 2024-09-12T14:19:33.368 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.70392.log 2024-09-12T14:19:33.368 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.57203.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.57203.log.gz 2024-09-12T14:19:33.369 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.44392.log 2024-09-12T14:19:33.369 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.70392.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.70392.log.gz 2024-09-12T14:19:33.369 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.79237.log 2024-09-12T14:19:33.370 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.44392.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.44392.log.gz 2024-09-12T14:19:33.370 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.63819.log 2024-09-12T14:19:33.371 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.79237.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.109612.log 2024-09-12T14:19:33.371 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.79237.log.gz 2024-09-12T14:19:33.372 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.63819.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.63819.log.gz 2024-09-12T14:19:33.372 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.126763.log 2024-09-12T14:19:33.373 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.109612.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.109612.log.gz 2024-09-12T14:19:33.373 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.63146.log 2024-09-12T14:19:33.374 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.126763.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.126763.log.gz 2024-09-12T14:19:33.374 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.138851.log 2024-09-12T14:19:33.374 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.63146.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.63146.log.gz 2024-09-12T14:19:33.374 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.150717.log 2024-09-12T14:19:33.375 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.138851.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.138851.log.gz 2024-09-12T14:19:33.375 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.98017.log 2024-09-12T14:19:33.376 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.150717.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.150717.log.gz 2024-09-12T14:19:33.376 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.110009.log 2024-09-12T14:19:33.377 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.98017.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.98017.log.gz 2024-09-12T14:19:33.377 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.81413.log 2024-09-12T14:19:33.377 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.110009.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.110009.log.gz 2024-09-12T14:19:33.378 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.89141.log 2024-09-12T14:19:33.378 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.81413.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.81413.log.gz 2024-09-12T14:19:33.379 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.124594.log 2024-09-12T14:19:33.379 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.89141.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.89141.log.gz 2024-09-12T14:19:33.380 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.113140.log 2024-09-12T14:19:33.380 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.124594.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.124594.log.gz 2024-09-12T14:19:33.380 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.63775.log 2024-09-12T14:19:33.381 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.113140.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.113140.log.gz 2024-09-12T14:19:33.381 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.93859.log 2024-09-12T14:19:33.382 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.63775.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.63775.log.gz 2024-09-12T14:19:33.382 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.73249.log 2024-09-12T14:19:33.383 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.93859.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.93859.log.gz 2024-09-12T14:19:33.383 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.115736.log 2024-09-12T14:19:33.384 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.73249.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.73249.log.gz 2024-09-12T14:19:33.384 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.137965.log 2024-09-12T14:19:33.384 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.115736.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.115736.log.gz 2024-09-12T14:19:33.385 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.87317.log 2024-09-12T14:19:33.385 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.137965.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.137965.log.gz 2024-09-12T14:19:33.386 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.123847.log 2024-09-12T14:19:33.386 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.87317.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.87317.log.gz 2024-09-12T14:19:33.386 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.64871.log 2024-09-12T14:19:33.387 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.123847.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.123847.log.gz 2024-09-12T14:19:33.387 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.81745.log 2024-09-12T14:19:33.388 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.64871.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.64871.log.gz 2024-09-12T14:19:33.388 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.120774.log 2024-09-12T14:19:33.389 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.81745.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.81745.log.gz 2024-09-12T14:19:33.389 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.40008.log 2024-09-12T14:19:33.390 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.120774.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.120774.log.gz 2024-09-12T14:19:33.390 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.147959.log 2024-09-12T14:19:33.391 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.40008.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.40008.log.gz 2024-09-12T14:19:33.391 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.61847.log 2024-09-12T14:19:33.391 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.147959.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.147959.log.gz 2024-09-12T14:19:33.392 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.121322.log 2024-09-12T14:19:33.392 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.61847.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.61847.log.gz 2024-09-12T14:19:33.393 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.91276.log 2024-09-12T14:19:33.393 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.121322.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.121322.log.gz 2024-09-12T14:19:33.394 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.121830.log 2024-09-12T14:19:33.394 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.91276.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.91276.log.gz 2024-09-12T14:19:33.395 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.50626.log 2024-09-12T14:19:33.395 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.121830.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.121830.log.gz 2024-09-12T14:19:33.395 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.60364.log 2024-09-12T14:19:33.396 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.50626.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.50626.log.gz 2024-09-12T14:19:33.396 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.62433.log 2024-09-12T14:19:33.397 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.60364.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.60364.log.gz 2024-09-12T14:19:33.397 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.74251.log 2024-09-12T14:19:33.398 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.62433.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.62433.log.gz 2024-09-12T14:19:33.398 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.74006.log 2024-09-12T14:19:33.399 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.74251.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.74251.log.gz 2024-09-12T14:19:33.399 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.137535.log 2024-09-12T14:19:33.400 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.74006.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.87670.log 2024-09-12T14:19:33.400 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.74006.log.gz 2024-09-12T14:19:33.401 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.137535.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.137535.log.gz 2024-09-12T14:19:33.401 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.109956.log 2024-09-12T14:19:33.402 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.87670.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.87670.log.gz 2024-09-12T14:19:33.402 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.37886.log 2024-09-12T14:19:33.402 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.109956.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.109956.log.gz 2024-09-12T14:19:33.402 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.49220.log 2024-09-12T14:19:33.403 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.37886.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.99937.log 2024-09-12T14:19:33.403 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.37886.log.gz 2024-09-12T14:19:33.404 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.49220.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.108417.log 2024-09-12T14:19:33.404 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.49220.log.gz 2024-09-12T14:19:33.404 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.99937.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.99937.log.gz 2024-09-12T14:19:33.405 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.51698.log 2024-09-12T14:19:33.405 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.108417.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.108417.log.gz 2024-09-12T14:19:33.406 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.54820.log 2024-09-12T14:19:33.407 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.51698.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.51698.log.gz 2024-09-12T14:19:33.407 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.37134.log 2024-09-12T14:19:33.407 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.54820.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.125351.log 2024-09-12T14:19:33.407 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.54820.log.gz 2024-09-12T14:19:33.408 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.37134.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.37134.log.gz 2024-09-12T14:19:33.408 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.141463.log 2024-09-12T14:19:33.409 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.125351.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.125351.log.gz 2024-09-12T14:19:33.409 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.148500.log 2024-09-12T14:19:33.410 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.141463.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.141463.log.gz 2024-09-12T14:19:33.410 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.129336.log 2024-09-12T14:19:33.411 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.148500.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.148500.log.gz 2024-09-12T14:19:33.411 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.38372.log 2024-09-12T14:19:33.412 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.129336.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.125709.log 2024-09-12T14:19:33.412 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.129336.log.gz 2024-09-12T14:19:33.412 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.38372.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.38372.log.gz 2024-09-12T14:19:33.412 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.125008.log 2024-09-12T14:19:33.413 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.125709.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.125709.log.gz 2024-09-12T14:19:33.413 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.106161.log 2024-09-12T14:19:33.414 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.125008.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.125008.log.gz 2024-09-12T14:19:33.414 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.70070.log 2024-09-12T14:19:33.415 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.106161.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.106161.log.gz 2024-09-12T14:19:33.415 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.138122.log 2024-09-12T14:19:33.416 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.70070.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.70070.log.gz 2024-09-12T14:19:33.416 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.111946.log 2024-09-12T14:19:33.417 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.138122.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.138122.log.gz 2024-09-12T14:19:33.417 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.94368.log 2024-09-12T14:19:33.418 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.111946.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.112509.log 2024-09-12T14:19:33.418 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.111946.log.gz 2024-09-12T14:19:33.418 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.94368.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.94368.log.gz 2024-09-12T14:19:33.419 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.37929.log 2024-09-12T14:19:33.420 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.112509.log: gzip 0.0% -5 -- replaced with /var/log/ceph/cluster1-client.mirror.112509.log.gz --verbose 2024-09-12T14:19:33.420 INFO:teuthology.orchestra.run.smithi130.stderr: -- /var/log/ceph/cluster2-client.admin.93228.log 2024-09-12T14:19:33.420 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.37929.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.37929.log.gz 2024-09-12T14:19:33.421 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.137244.log 2024-09-12T14:19:33.422 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.93228.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.93228.log.gz 2024-09-12T14:19:33.422 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.63604.log 2024-09-12T14:19:33.423 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.137244.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.137244.log.gz 2024-09-12T14:19:33.423 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.106468.log 2024-09-12T14:19:33.423 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.63604.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.63604.log.gz 2024-09-12T14:19:33.424 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.91209.log 2024-09-12T14:19:33.424 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.106468.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.106468.log.gz 2024-09-12T14:19:33.424 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.110726.log 2024-09-12T14:19:33.425 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.91209.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.52770.log 2024-09-12T14:19:33.425 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.91209.log.gz 2024-09-12T14:19:33.425 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.110726.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.103856.log 2024-09-12T14:19:33.426 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.110726.log.gz 2024-09-12T14:19:33.426 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.52770.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.52770.log.gz 2024-09-12T14:19:33.426 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.91885.log 2024-09-12T14:19:33.427 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.103856.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.103856.log.gz 2024-09-12T14:19:33.427 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.133713.log 2024-09-12T14:19:33.428 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.91885.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.91885.log.gz 2024-09-12T14:19:33.428 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.127964.log 2024-09-12T14:19:33.429 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.133713.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.133713.log.gz 2024-09-12T14:19:33.429 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.97255.log 2024-09-12T14:19:33.430 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.127964.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.127964.log.gz 2024-09-12T14:19:33.430 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.34402.log 2024-09-12T14:19:33.431 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.97255.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.97255.log.gz 2024-09-12T14:19:33.431 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.70257.log 2024-09-12T14:19:33.431 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.34402.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.34402.log.gz 2024-09-12T14:19:33.432 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.100733.log 2024-09-12T14:19:33.432 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.70257.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.70257.log.gz 2024-09-12T14:19:33.433 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.95917.log 2024-09-12T14:19:33.433 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.100733.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.100733.log.gz 2024-09-12T14:19:33.434 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.42009.log 2024-09-12T14:19:33.434 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.95917.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.95917.log.gz 2024-09-12T14:19:33.434 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.37763.log 2024-09-12T14:19:33.435 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.42009.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.42009.log.gz 2024-09-12T14:19:33.435 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.50208.log 2024-09-12T14:19:33.436 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.37763.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.37763.log.gz 2024-09-12T14:19:33.436 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.94575.log 2024-09-12T14:19:33.437 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.50208.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.50208.log.gz 2024-09-12T14:19:33.437 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.98568.log 2024-09-12T14:19:33.438 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.94575.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.152284.log 2024-09-12T14:19:33.438 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.94575.log.gz 2024-09-12T14:19:33.438 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.98568.log: 0.0%gzip -- replaced with /var/log/ceph/cluster1-client.mirror.98568.log.gz -5 2024-09-12T14:19:33.438 INFO:teuthology.orchestra.run.smithi130.stderr: --verbose -- /var/log/ceph/cluster1-client.admin.66845.log 2024-09-12T14:19:33.439 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.152284.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.152284.log.gz 2024-09-12T14:19:33.439 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.60901.log 2024-09-12T14:19:33.440 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.66845.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.66845.log.gz 2024-09-12T14:19:33.441 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.120003.log 2024-09-12T14:19:33.441 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.60901.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.60901.log.gz 2024-09-12T14:19:33.441 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.137769.log 2024-09-12T14:19:33.442 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.120003.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.120003.log.gz 2024-09-12T14:19:33.442 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.44876.log 2024-09-12T14:19:33.443 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.137769.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.137769.log.gz 2024-09-12T14:19:33.443 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.87313.log 2024-09-12T14:19:33.444 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.44876.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.152733.log 2024-09-12T14:19:33.444 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.44876.log.gz 2024-09-12T14:19:33.444 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.87313.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.87313.log.gz 2024-09-12T14:19:33.445 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.136021.log 2024-09-12T14:19:33.446 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.152733.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.152733.log.gz 2024-09-12T14:19:33.446 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.44285.log 2024-09-12T14:19:33.446 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.136021.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.136021.log.gz 2024-09-12T14:19:33.446 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.51108.log 2024-09-12T14:19:33.447 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.44285.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.51720.log 2024-09-12T14:19:33.447 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.44285.log.gz 2024-09-12T14:19:33.448 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.51108.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.51108.log.gz 2024-09-12T14:19:33.448 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.74216.log 2024-09-12T14:19:33.449 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.51720.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.51720.log.gz 2024-09-12T14:19:33.449 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.102101.log 2024-09-12T14:19:33.450 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.74216.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.74216.log.gz 2024-09-12T14:19:33.450 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.37951.log 2024-09-12T14:19:33.450 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.102101.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.102101.log.gz 2024-09-12T14:19:33.451 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.87631.log 2024-09-12T14:19:33.451 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.37951.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.126674.log 2024-09-12T14:19:33.452 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.37951.log.gz 2024-09-12T14:19:33.452 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.87631.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.37511.log 2024-09-12T14:19:33.453 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.87631.log.gz 2024-09-12T14:19:33.453 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.126674.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.126674.log.gz 2024-09-12T14:19:33.453 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.140960.log 2024-09-12T14:19:33.454 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.37511.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.37511.log.gz 2024-09-12T14:19:33.454 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.37636.log 2024-09-12T14:19:33.455 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.140960.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.140960.log.gz 2024-09-12T14:19:33.455 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.125433.log 2024-09-12T14:19:33.456 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.37636.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.129068.log 2024-09-12T14:19:33.456 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.37636.log.gz 2024-09-12T14:19:33.457 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.125433.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.125433.log.gz 2024-09-12T14:19:33.457 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.94537.log 2024-09-12T14:19:33.457 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.129068.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.129068.log.gz 2024-09-12T14:19:33.457 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.150784.log 2024-09-12T14:19:33.458 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.94537.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.94537.log.gz 2024-09-12T14:19:33.458 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.129409.log 2024-09-12T14:19:33.459 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.150784.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.150784.log.gz 2024-09-12T14:19:33.459 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.92212.log 2024-09-12T14:19:33.460 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.129409.log: gzip 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.129409.log.gz -5 2024-09-12T14:19:33.460 INFO:teuthology.orchestra.run.smithi130.stderr: --verbose -- /var/log/ceph/cluster2-client.mirror.117065.log 2024-09-12T14:19:33.461 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.92212.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.92212.log.gz 2024-09-12T14:19:33.461 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.54674.log 2024-09-12T14:19:33.462 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.117065.log: gzip -5 0.0% --verbose -- replaced with /var/log/ceph/cluster2-client.mirror.117065.log.gz -- 2024-09-12T14:19:33.462 INFO:teuthology.orchestra.run.smithi130.stderr: /var/log/ceph/cluster1-client.admin.140980.log 2024-09-12T14:19:33.462 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.54674.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.54674.log.gz 2024-09-12T14:19:33.463 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.47205.log 2024-09-12T14:19:33.463 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.140980.log: 0.0%gzip -- replaced with /var/log/ceph/cluster1-client.admin.140980.log.gz -5 --verbose 2024-09-12T14:19:33.463 INFO:teuthology.orchestra.run.smithi130.stderr: -- /var/log/ceph/cluster1-client.admin.43287.log 2024-09-12T14:19:33.464 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.47205.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.47205.log.gz 2024-09-12T14:19:33.464 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.99868.log 2024-09-12T14:19:33.465 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.43287.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.43287.log.gz 2024-09-12T14:19:33.465 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.48059.log 2024-09-12T14:19:33.465 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.99868.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.99868.log.gz 2024-09-12T14:19:33.466 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.125722.log 2024-09-12T14:19:33.466 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.48059.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.48059.log.gz 2024-09-12T14:19:33.466 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.45252.log 2024-09-12T14:19:33.467 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.125722.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.125722.log.gz 2024-09-12T14:19:33.467 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.137982.log 2024-09-12T14:19:33.468 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.45252.log: gzip 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.45252.log.gz -5 2024-09-12T14:19:33.468 INFO:teuthology.orchestra.run.smithi130.stderr: --verbose -- /var/log/ceph/cluster1-client.mirror.124921.log 2024-09-12T14:19:33.469 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.137982.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.137982.log.gz 2024-09-12T14:19:33.469 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.57676.log 2024-09-12T14:19:33.470 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.124921.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.124921.log.gz 2024-09-12T14:19:33.470 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.125778.log 2024-09-12T14:19:33.470 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.57676.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.57676.log.gzgzip 2024-09-12T14:19:33.471 INFO:teuthology.orchestra.run.smithi130.stderr: -5 --verbose -- /var/log/ceph/cluster1-client.mirror.111476.log 2024-09-12T14:19:33.471 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.125778.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.125778.log.gz 2024-09-12T14:19:33.471 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.47564.log 2024-09-12T14:19:33.472 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.111476.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.131492.log 2024-09-12T14:19:33.472 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.111476.log.gz 2024-09-12T14:19:33.473 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.47564.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.47564.log.gz 2024-09-12T14:19:33.473 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.63001.log 2024-09-12T14:19:33.474 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.131492.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.131492.log.gz 2024-09-12T14:19:33.474 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.108456.log 2024-09-12T14:19:33.474 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.63001.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.63001.log.gz 2024-09-12T14:19:33.475 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.88088.log 2024-09-12T14:19:33.475 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.108456.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.108456.log.gz 2024-09-12T14:19:33.475 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.144482.log 2024-09-12T14:19:33.476 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.88088.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.145751.log 2024-09-12T14:19:33.476 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.88088.log.gz 2024-09-12T14:19:33.477 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.144482.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.144482.log.gz 2024-09-12T14:19:33.477 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.105980.log 2024-09-12T14:19:33.478 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.145751.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.145751.log.gz 2024-09-12T14:19:33.478 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.123977.log 2024-09-12T14:19:33.479 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.105980.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.105980.log.gz 2024-09-12T14:19:33.479 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.37405.log 2024-09-12T14:19:33.479 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.123977.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.77516.log 2024-09-12T14:19:33.479 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.123977.log.gz 2024-09-12T14:19:33.480 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.37405.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.37405.log.gz 2024-09-12T14:19:33.480 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.44555.log 2024-09-12T14:19:33.481 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.77516.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.77516.log.gz 2024-09-12T14:19:33.481 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.79369.log 2024-09-12T14:19:33.482 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.44555.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.44555.log.gz 2024-09-12T14:19:33.482 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.87794.log 2024-09-12T14:19:33.483 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.79369.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.79369.log.gz 2024-09-12T14:19:33.483 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.107828.log 2024-09-12T14:19:33.483 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.87794.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.87794.log.gz 2024-09-12T14:19:33.483 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.75474.log 2024-09-12T14:19:33.484 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.107828.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.107828.log.gz 2024-09-12T14:19:33.484 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.40598.log 2024-09-12T14:19:33.485 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.75474.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.75474.log.gz 2024-09-12T14:19:33.485 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.42451.log 2024-09-12T14:19:33.486 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.40598.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.98336.log 2024-09-12T14:19:33.486 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.40598.log.gz 2024-09-12T14:19:33.486 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.42451.log: gzip 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.42451.log.gz 2024-09-12T14:19:33.487 INFO:teuthology.orchestra.run.smithi130.stderr: -5 --verbose -- /var/log/ceph/cluster1-client.mirror.41170.log 2024-09-12T14:19:33.487 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.98336.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.98336.log.gz 2024-09-12T14:19:33.487 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.144596.log 2024-09-12T14:19:33.488 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.41170.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.41170.log.gz 2024-09-12T14:19:33.488 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.104448.log 2024-09-12T14:19:33.489 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.144596.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.144596.log.gz 2024-09-12T14:19:33.489 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.87902.log 2024-09-12T14:19:33.490 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.104448.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.104448.log.gz 2024-09-12T14:19:33.490 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.101716.log 2024-09-12T14:19:33.490 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.87902.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.87902.log.gz 2024-09-12T14:19:33.490 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.52852.log 2024-09-12T14:19:33.491 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.101716.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.101716.log.gz 2024-09-12T14:19:33.491 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.53190.log 2024-09-12T14:19:33.492 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.52852.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.52852.log.gz 2024-09-12T14:19:33.492 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.93973.log 2024-09-12T14:19:33.493 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.53190.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.53190.log.gz 2024-09-12T14:19:33.493 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.149736.log 2024-09-12T14:19:33.493 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.93973.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.93973.log.gz 2024-09-12T14:19:33.494 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.133249.log 2024-09-12T14:19:33.494 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.149736.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.149736.log.gz 2024-09-12T14:19:33.494 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.133206.log 2024-09-12T14:19:33.495 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.133249.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.133249.log.gz 2024-09-12T14:19:33.495 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.45853.log 2024-09-12T14:19:33.496 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.133206.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.133206.log.gz 2024-09-12T14:19:33.496 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.120901.log 2024-09-12T14:19:33.497 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.45853.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.45853.log.gz 2024-09-12T14:19:33.497 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.126873.log 2024-09-12T14:19:33.497 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.120901.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.120901.log.gz 2024-09-12T14:19:33.498 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.137581.log 2024-09-12T14:19:33.498 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.126873.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.126873.log.gz 2024-09-12T14:19:33.498 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.134302.log 2024-09-12T14:19:33.499 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.137581.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.137581.log.gz 2024-09-12T14:19:33.499 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.148060.log 2024-09-12T14:19:33.500 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.134302.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.134302.log.gz 2024-09-12T14:19:33.500 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.144064.log 2024-09-12T14:19:33.501 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.148060.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.148060.log.gz 2024-09-12T14:19:33.501 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.59515.log 2024-09-12T14:19:33.501 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.144064.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.144064.log.gz 2024-09-12T14:19:33.501 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.54312.log 2024-09-12T14:19:33.502 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.59515.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.59515.log.gz 2024-09-12T14:19:33.502 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.49661.log 2024-09-12T14:19:33.503 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.54312.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.54312.log.gz 2024-09-12T14:19:33.503 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.136319.log 2024-09-12T14:19:33.504 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.49661.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.49661.log.gz 2024-09-12T14:19:33.504 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.81285.log 2024-09-12T14:19:33.505 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.136319.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.136319.log.gz 2024-09-12T14:19:33.505 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.51067.log 2024-09-12T14:19:33.505 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.81285.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.81285.log.gz 2024-09-12T14:19:33.505 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.52033.log 2024-09-12T14:19:33.506 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.51067.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.51067.log.gz 2024-09-12T14:19:33.506 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.73010.log 2024-09-12T14:19:33.507 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.52033.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.52033.log.gz 2024-09-12T14:19:33.507 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.145162.log 2024-09-12T14:19:33.508 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.73010.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.73010.log.gz 2024-09-12T14:19:33.508 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.39489.log 2024-09-12T14:19:33.508 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.145162.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.145162.log.gz 2024-09-12T14:19:33.509 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.84271.log 2024-09-12T14:19:33.509 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.39489.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.39489.log.gz 2024-09-12T14:19:33.510 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.131580.log 2024-09-12T14:19:33.510 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.84271.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.84271.log.gz 2024-09-12T14:19:33.510 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.65333.log 2024-09-12T14:19:33.511 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.131580.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.131580.log.gz 2024-09-12T14:19:33.511 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.101232.log 2024-09-12T14:19:33.512 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.65333.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.65333.log.gz 2024-09-12T14:19:33.512 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.129087.log 2024-09-12T14:19:33.513 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.101232.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.101232.log.gz 2024-09-12T14:19:33.513 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.52434.log 2024-09-12T14:19:33.513 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.129087.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.129087.log.gz 2024-09-12T14:19:33.513 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.125329.log 2024-09-12T14:19:33.514 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.52434.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.52434.log.gz 2024-09-12T14:19:33.514 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.89665.log 2024-09-12T14:19:33.515 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.125329.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.125329.log.gz 2024-09-12T14:19:33.515 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.131802.log 2024-09-12T14:19:33.516 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.89665.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.89665.log.gz 2024-09-12T14:19:33.516 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.78545.log 2024-09-12T14:19:33.517 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.131802.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.131802.log.gz 2024-09-12T14:19:33.517 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.127206.log 2024-09-12T14:19:33.517 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.150275.log 2024-09-12T14:19:33.518 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.78545.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.78545.log.gz 2024-09-12T14:19:33.518 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.127206.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.127206.log.gz 2024-09-12T14:19:33.518 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.88259.log 2024-09-12T14:19:33.519 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.150275.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.46603.log 2024-09-12T14:19:33.519 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.150275.log.gz 2024-09-12T14:19:33.520 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.88259.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.88259.log.gz 2024-09-12T14:19:33.520 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.116638.log 2024-09-12T14:19:33.520 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.46603.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.101825.log 2024-09-12T14:19:33.520 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.46603.log.gz 2024-09-12T14:19:33.521 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.116638.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.116638.log.gz 2024-09-12T14:19:33.521 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.148149.log 2024-09-12T14:19:33.522 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.101825.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.140546.log 0.0% 2024-09-12T14:19:33.522 INFO:teuthology.orchestra.run.smithi130.stderr: -- replaced with /var/log/ceph/cluster1-client.mirror.101825.log.gz 2024-09-12T14:19:33.523 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.148149.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.148149.log.gz 2024-09-12T14:19:33.523 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.103389.log 2024-09-12T14:19:33.524 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.140546.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.133461.log 2024-09-12T14:19:33.524 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.140546.log.gz 2024-09-12T14:19:33.524 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.103389.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.103389.log.gz 2024-09-12T14:19:33.524 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.49144.log 2024-09-12T14:19:33.525 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.133461.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.90558.log 2024-09-12T14:19:33.526 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.133461.log.gz 2024-09-12T14:19:33.526 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.49144.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.49144.log.gz 2024-09-12T14:19:33.526 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.102384.log 2024-09-12T14:19:33.527 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.90558.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.90558.log.gz 2024-09-12T14:19:33.527 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.66212.log 2024-09-12T14:19:33.528 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.102384.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.75997.log 0.0% 2024-09-12T14:19:33.528 INFO:teuthology.orchestra.run.smithi130.stderr: -- replaced with /var/log/ceph/cluster1-client.admin.102384.log.gz 2024-09-12T14:19:33.528 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.66212.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.66212.log.gz 2024-09-12T14:19:33.529 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.52367.log 2024-09-12T14:19:33.529 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.75997.log: gzip -5 --verbose -- 0.0% /var/log/ceph/cluster1-client.mirror.60796.log 2024-09-12T14:19:33.529 INFO:teuthology.orchestra.run.smithi130.stderr: -- replaced with /var/log/ceph/cluster1-client.mirror.75997.log.gz 2024-09-12T14:19:33.530 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.52367.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.52367.log.gz 2024-09-12T14:19:33.530 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.84047.log 2024-09-12T14:19:33.531 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.60796.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.125047.log 2024-09-12T14:19:33.531 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.60796.log.gz 2024-09-12T14:19:33.531 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.84047.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.84047.log.gz 2024-09-12T14:19:33.531 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.149321.log 2024-09-12T14:19:33.532 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.125047.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.125047.log.gz 2024-09-12T14:19:33.532 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.150760.log 2024-09-12T14:19:33.533 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.149321.log: gzip -5 --verbose -- 0.0% /var/log/ceph/cluster1-client.mirror.123659.log 2024-09-12T14:19:33.533 INFO:teuthology.orchestra.run.smithi130.stderr: -- replaced with /var/log/ceph/cluster1-client.mirror.149321.log.gz 2024-09-12T14:19:33.533 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.150760.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.150760.log.gz 2024-09-12T14:19:33.534 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.123174.log 2024-09-12T14:19:33.534 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.123659.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.123659.log.gz 2024-09-12T14:19:33.534 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.95392.log 2024-09-12T14:19:33.535 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.123174.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.61464.log 2024-09-12T14:19:33.535 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.123174.log.gz 2024-09-12T14:19:33.536 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.95392.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.95392.log.gz 2024-09-12T14:19:33.536 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.125743.log 2024-09-12T14:19:33.537 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.61464.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.61464.log.gz 2024-09-12T14:19:33.537 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.92645.log 2024-09-12T14:19:33.537 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.125743.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.125743.log.gz 2024-09-12T14:19:33.538 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.75591.log 2024-09-12T14:19:33.538 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.92645.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.92645.log.gz 2024-09-12T14:19:33.539 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.111396.log 2024-09-12T14:19:33.539 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.75591.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.38139.log 2024-09-12T14:19:33.539 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.75591.log.gz 2024-09-12T14:19:33.540 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.111396.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.41777.log 2024-09-12T14:19:33.540 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.111396.log.gz 2024-09-12T14:19:33.541 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.38139.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.82496.log 2024-09-12T14:19:33.541 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.38139.log.gz 2024-09-12T14:19:33.541 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.41777.log: gzip -5 --verbose -- 0.0% /var/log/ceph/cluster1-client.mirror.52935.log -- replaced with /var/log/ceph/cluster1-client.mirror.41777.log.gz 2024-09-12T14:19:33.542 INFO:teuthology.orchestra.run.smithi130.stderr: 2024-09-12T14:19:33.542 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.82496.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.82496.log.gz 2024-09-12T14:19:33.542 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.76676.log 2024-09-12T14:19:33.543 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.52935.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.52935.log.gz 2024-09-12T14:19:33.543 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.96891.log 2024-09-12T14:19:33.544 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.76676.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.76676.log.gz 2024-09-12T14:19:33.544 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.76295.log 2024-09-12T14:19:33.545 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.96891.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.96891.log.gz 2024-09-12T14:19:33.545 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.151607.log 2024-09-12T14:19:33.545 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.76295.log: gzip -5 0.0% --verbose -- -- replaced with /var/log/ceph/cluster1-client.mirror.76295.log.gz /var/log/ceph/cluster1-client.mirror.148988.log 2024-09-12T14:19:33.545 INFO:teuthology.orchestra.run.smithi130.stderr: 2024-09-12T14:19:33.546 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.151607.log: 0.0%gzip -- replaced with /var/log/ceph/cluster2-client.admin.151607.log.gz -5 2024-09-12T14:19:33.546 INFO:teuthology.orchestra.run.smithi130.stderr: --verbose -- /var/log/ceph/cluster1-client.mirror.89535.log 2024-09-12T14:19:33.547 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.148988.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.148988.log.gz 2024-09-12T14:19:33.547 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.70812.log 2024-09-12T14:19:33.548 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.89535.log: gzip 0.0% -5 -- replaced with /var/log/ceph/cluster1-client.mirror.89535.log.gz --verbose 2024-09-12T14:19:33.548 INFO:teuthology.orchestra.run.smithi130.stderr: -- /var/log/ceph/cluster1-client.admin.126101.log 2024-09-12T14:19:33.549 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.70812.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.70812.log.gz 2024-09-12T14:19:33.549 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.66456.log 2024-09-12T14:19:33.550 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.126101.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.126101.log.gz 2024-09-12T14:19:33.550 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.132830.log 2024-09-12T14:19:33.551 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.66456.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.66456.log.gz 2024-09-12T14:19:33.551 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.78822.log 2024-09-12T14:19:33.552 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.132830.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.132830.log.gz 2024-09-12T14:19:33.552 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.130357.log 2024-09-12T14:19:33.552 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.78822.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.78822.log.gz 2024-09-12T14:19:33.552 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.69755.log 2024-09-12T14:19:33.552 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.130357.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.130357.log.gz 2024-09-12T14:19:33.553 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.126052.log 2024-09-12T14:19:33.553 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.69755.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.69755.log.gz 2024-09-12T14:19:33.553 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.96992.log 2024-09-12T14:19:33.554 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.126052.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.126052.log.gz 2024-09-12T14:19:33.554 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.74173.log 2024-09-12T14:19:33.555 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.96992.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.96992.log.gz 2024-09-12T14:19:33.555 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.144252.log 2024-09-12T14:19:33.555 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.74173.log: gzip 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.74173.log.gz -5 2024-09-12T14:19:33.555 INFO:teuthology.orchestra.run.smithi130.stderr: --verbose -- /var/log/ceph/cluster1-client.mirror.108939.log 2024-09-12T14:19:33.556 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.144252.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.144252.log.gz 2024-09-12T14:19:33.556 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.127642.log 2024-09-12T14:19:33.557 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.108939.log: gzip 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.108939.log.gz -5 2024-09-12T14:19:33.557 INFO:teuthology.orchestra.run.smithi130.stderr: --verbose -- /var/log/ceph/cluster1-client.admin.46146.log 2024-09-12T14:19:33.557 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.127642.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.127642.log.gz 2024-09-12T14:19:33.558 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.131927.log 2024-09-12T14:19:33.558 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.46146.log: gzip -5 --verbose -- 0.0% /var/log/ceph/cluster1-client.admin.138411.log 2024-09-12T14:19:33.558 INFO:teuthology.orchestra.run.smithi130.stderr: -- replaced with /var/log/ceph/cluster1-client.admin.46146.log.gz 2024-09-12T14:19:33.559 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.131927.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.131927.log.gz 2024-09-12T14:19:33.559 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.73103.log 2024-09-12T14:19:33.560 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.138411.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.138411.log.gz 2024-09-12T14:19:33.560 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.123324.log 2024-09-12T14:19:33.561 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.73103.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.73103.log.gz 2024-09-12T14:19:33.561 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.87210.log 2024-09-12T14:19:33.561 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.123324.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.123324.log.gz 2024-09-12T14:19:33.562 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.41798.log 2024-09-12T14:19:33.562 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.87210.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.39401.log 2024-09-12T14:19:33.562 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.87210.log.gz 2024-09-12T14:19:33.563 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.41798.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.41798.log.gz 2024-09-12T14:19:33.563 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.83352.log 2024-09-12T14:19:33.564 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.39401.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.74153.log 2024-09-12T14:19:33.564 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.39401.log.gz 2024-09-12T14:19:33.565 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.83352.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.83352.log.gz 2024-09-12T14:19:33.565 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.134827.log 2024-09-12T14:19:33.565 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.74153.log: 0.0%gzip -- replaced with /var/log/ceph/cluster1-client.mirror.74153.log.gz 2024-09-12T14:19:33.565 INFO:teuthology.orchestra.run.smithi130.stderr: -5 --verbose -- /var/log/ceph/cluster1-client.mirror.50289.log 2024-09-12T14:19:33.566 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.134827.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.134827.log.gz 2024-09-12T14:19:33.566 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.97537.log 2024-09-12T14:19:33.567 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.50289.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.50289.log.gz 2024-09-12T14:19:33.567 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.72331.log 2024-09-12T14:19:33.567 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.97537.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.97537.log.gz 2024-09-12T14:19:33.568 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.108730.log 2024-09-12T14:19:33.568 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.72331.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.72331.log.gz 2024-09-12T14:19:33.569 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.108081.log 2024-09-12T14:19:33.569 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.108730.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.108730.log.gz 2024-09-12T14:19:33.570 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.144002.log 2024-09-12T14:19:33.571 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.108081.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.108081.log.gz 2024-09-12T14:19:33.571 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.84269.log 2024-09-12T14:19:33.572 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.144002.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.96945.log 2024-09-12T14:19:33.572 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.144002.log.gz 2024-09-12T14:19:33.572 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.84269.log: gzip -5 --verbose 0.0% -- -- replaced with /var/log/ceph/cluster1-client.mirror.84269.log.gz 2024-09-12T14:19:33.572 INFO:teuthology.orchestra.run.smithi130.stderr: /var/log/ceph/cluster2-client.mirror.81677.log 2024-09-12T14:19:33.573 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.96945.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.96945.log.gz 2024-09-12T14:19:33.573 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.84374.log 2024-09-12T14:19:33.574 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.81677.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.81677.log.gz 2024-09-12T14:19:33.574 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.116091.log 2024-09-12T14:19:33.575 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.84374.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.84374.log.gz 2024-09-12T14:19:33.575 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.58378.log 2024-09-12T14:19:33.575 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.116091.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.116091.log.gz 2024-09-12T14:19:33.575 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.37133.log 2024-09-12T14:19:33.576 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.58378.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.58378.log.gz 2024-09-12T14:19:33.576 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.52773.log 2024-09-12T14:19:33.577 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.37133.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.37133.log.gz 2024-09-12T14:19:33.577 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.81452.log 2024-09-12T14:19:33.578 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.52773.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.52773.log.gz 2024-09-12T14:19:33.578 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.43056.log 2024-09-12T14:19:33.579 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.81452.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.81452.log.gz 2024-09-12T14:19:33.579 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.145882.log 2024-09-12T14:19:33.579 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.43056.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.43056.log.gz 2024-09-12T14:19:33.580 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.91912.log 2024-09-12T14:19:33.580 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.145882.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.145882.log.gz 2024-09-12T14:19:33.580 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.147431.log 2024-09-12T14:19:33.581 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.91912.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.91912.log.gz 2024-09-12T14:19:33.581 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.136928.log 2024-09-12T14:19:33.582 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.147431.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.96340.log 2024-09-12T14:19:33.582 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.147431.log.gz 2024-09-12T14:19:33.583 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.136928.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.136928.log.gz 2024-09-12T14:19:33.583 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.148380.log 2024-09-12T14:19:33.583 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.96340.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.96340.log.gz 2024-09-12T14:19:33.583 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.115381.log 2024-09-12T14:19:33.584 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.148380.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.148380.log.gz 2024-09-12T14:19:33.584 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.107296.log 2024-09-12T14:19:33.585 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.115381.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.115381.log.gz 2024-09-12T14:19:33.585 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.103010.log 2024-09-12T14:19:33.586 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.107296.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.107296.log.gz 2024-09-12T14:19:33.586 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.132223.log 2024-09-12T14:19:33.586 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.103010.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.103010.log.gz 2024-09-12T14:19:33.586 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.51424.log 2024-09-12T14:19:33.587 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.132223.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.132223.log.gz 2024-09-12T14:19:33.587 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.118393.log 2024-09-12T14:19:33.588 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.51424.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.51424.log.gz 2024-09-12T14:19:33.588 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.67136.log 2024-09-12T14:19:33.589 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.118393.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.118393.log.gz 2024-09-12T14:19:33.589 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.125191.log 2024-09-12T14:19:33.589 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.67136.log: gzip 0.0% -5 -- replaced with /var/log/ceph/cluster1-client.admin.67136.log.gz 2024-09-12T14:19:33.590 INFO:teuthology.orchestra.run.smithi130.stderr: --verbose -- /var/log/ceph/cluster2-client.mirror.130207.log 2024-09-12T14:19:33.590 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.125191.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.125191.log.gz 2024-09-12T14:19:33.590 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.106732.log 2024-09-12T14:19:33.591 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.130207.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.130207.log.gz 2024-09-12T14:19:33.591 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.113985.log 2024-09-12T14:19:33.592 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.106732.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.106732.log.gz 2024-09-12T14:19:33.592 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.115662.log 2024-09-12T14:19:33.593 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.113985.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.113985.log.gz 2024-09-12T14:19:33.593 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.143189.log 2024-09-12T14:19:33.593 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.115662.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.115662.log.gz 2024-09-12T14:19:33.594 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.149597.log 2024-09-12T14:19:33.594 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.143189.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.143189.log.gz 2024-09-12T14:19:33.595 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.43445.log 2024-09-12T14:19:33.595 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.149597.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.149597.log.gz 2024-09-12T14:19:33.595 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.65204.log 2024-09-12T14:19:33.596 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.43445.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.43445.log.gz 2024-09-12T14:19:33.596 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.57881.log 2024-09-12T14:19:33.597 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.65204.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.65204.log.gz 2024-09-12T14:19:33.597 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.131863.log 2024-09-12T14:19:33.598 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.57881.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.57881.log.gz 2024-09-12T14:19:33.598 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.58264.log 2024-09-12T14:19:33.599 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.131863.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.131863.log.gz 2024-09-12T14:19:33.599 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.100749.log 2024-09-12T14:19:33.599 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.58264.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.58264.log.gz 2024-09-12T14:19:33.599 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.140168.log 2024-09-12T14:19:33.600 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.100749.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.100749.log.gz 2024-09-12T14:19:33.600 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.145187.log 2024-09-12T14:19:33.601 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.140168.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.133417.log 2024-09-12T14:19:33.601 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.140168.log.gz 2024-09-12T14:19:33.602 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.145187.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.101595.log 2024-09-12T14:19:33.602 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.145187.log.gz 2024-09-12T14:19:33.602 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.133417.log: gzip 0.0% -5 -- replaced with /var/log/ceph/cluster1-client.mirror.133417.log.gz --verbose 2024-09-12T14:19:33.603 INFO:teuthology.orchestra.run.smithi130.stderr: -- /var/log/ceph/cluster1-client.mirror.70278.log 2024-09-12T14:19:33.603 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.101595.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.101595.log.gz 2024-09-12T14:19:33.603 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.96841.log 2024-09-12T14:19:33.604 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.89394.log 2024-09-12T14:19:33.604 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.70278.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.70278.log.gz 2024-09-12T14:19:33.605 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.96841.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.151825.log 2024-09-12T14:19:33.605 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.96841.log.gz 2024-09-12T14:19:33.606 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.89394.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.89394.log.gz 2024-09-12T14:19:33.606 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.34989.log 2024-09-12T14:19:33.606 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.151825.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.151825.log.gz 2024-09-12T14:19:33.607 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.61065.log 2024-09-12T14:19:33.608 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.117313.log 2024-09-12T14:19:33.608 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.34989.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.34989.log.gz 2024-09-12T14:19:33.608 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.61065.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.61065.log.gz 2024-09-12T14:19:33.608 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.111796.log 2024-09-12T14:19:33.609 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.117313.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.92986.log 2024-09-12T14:19:33.609 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.117313.log.gz 2024-09-12T14:19:33.610 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.111796.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.111796.log.gz 2024-09-12T14:19:33.610 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.52348.log 2024-09-12T14:19:33.610 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.92986.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.92986.log.gz 2024-09-12T14:19:33.611 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.89332.log 2024-09-12T14:19:33.612 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.52348.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.52348.log.gz 2024-09-12T14:19:33.612 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.140624.log 2024-09-12T14:19:33.612 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.89332.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.89332.log.gz 2024-09-12T14:19:33.613 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.143706.log 2024-09-12T14:19:33.613 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.admin.140624.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.140624.log.gz 2024-09-12T14:19:33.613 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.54855.log 2024-09-12T14:19:33.614 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.143706.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.124607.log 2024-09-12T14:19:33.615 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.143706.log.gz 2024-09-12T14:19:33.615 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.admin.54855.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.54855.log.gz 2024-09-12T14:19:33.615 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.46314.log 2024-09-12T14:19:33.616 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.124607.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.124607.log.gz 2024-09-12T14:19:33.616 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.135209.log 2024-09-12T14:19:33.617 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster2-client.mirror.46314.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.85817.log 2024-09-12T14:19:33.617 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.46314.log.gz 2024-09-12T14:19:33.618 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.135209.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.135209.log.gz 2024-09-12T14:19:33.618 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.98945.log 2024-09-12T14:19:33.619 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.85817.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.85817.log.gz 2024-09-12T14:19:33.619 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.90190.log 2024-09-12T14:19:33.619 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.98945.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.143492.log 2024-09-12T14:19:33.619 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.98945.log.gz 2024-09-12T14:19:33.620 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.90190.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.90190.log.gz 2024-09-12T14:19:33.620 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.88214.log 2024-09-12T14:19:33.621 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.143492.log: 0.0%gzip -- replaced with /var/log/ceph/cluster1-client.mirror.143492.log.gz 2024-09-12T14:19:33.621 INFO:teuthology.orchestra.run.smithi130.stderr: -5 --verbose -- /var/log/ceph/cluster1-client.mirror.124671.log 2024-09-12T14:19:33.622 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.88214.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.88214.log.gz 2024-09-12T14:19:33.622 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.51762.log 2024-09-12T14:19:33.623 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.124671.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.88593.log 2024-09-12T14:19:33.623 INFO:teuthology.orchestra.run.smithi130.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.124671.log.gz 2024-09-12T14:19:33.623 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.51762.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.51762.log.gz 2024-09-12T14:19:33.624 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/cluster1-client.mirror.88593.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.88593.log.gz 2024-09-12T14:19:39.009 INFO:teuthology.orchestra.run.smithi130.stderr: 91.4% -- replaced with /var/log/ceph/cluster2-mgr.x.log.gz 2024-09-12T14:19:47.120 INFO:teuthology.orchestra.run.smithi060.stderr: 90.4% -- replaced with /var/log/ceph/cluster1-mon.a.log.gz 2024-09-12T14:19:58.567 INFO:teuthology.orchestra.run.smithi130.stderr: 90.2% -- replaced with /var/log/ceph/cluster2-mon.a.log.gz 2024-09-12T14:24:38.531 INFO:teuthology.orchestra.run.smithi130.stderr: 94.3% -- replaced with /var/log/ceph/cluster2-osd.1.log.gz 2024-09-12T14:25:00.926 INFO:teuthology.orchestra.run.smithi130.stderr: 94.3% -- replaced with /var/log/ceph/cluster2-osd.2.log.gz 2024-09-12T14:25:15.549 INFO:teuthology.orchestra.run.smithi130.stderr: 94.3% -- replaced with /var/log/ceph/cluster2-osd.0.log.gz 2024-09-12T14:25:15.552 INFO:teuthology.orchestra.run.smithi130.stderr: 2024-09-12T14:25:15.552 INFO:teuthology.orchestra.run.smithi130.stderr:real 5m46.246s 2024-09-12T14:25:15.552 INFO:teuthology.orchestra.run.smithi130.stderr:user 2m5.236s 2024-09-12T14:25:15.552 INFO:teuthology.orchestra.run.smithi130.stderr:sys 0m29.486s 2024-09-12T14:31:52.980 INFO:teuthology.orchestra.run.smithi060.stderr: 94.2% -- replaced with /var/log/ceph/cluster1-osd.1.log.gz 2024-09-12T14:32:05.440 INFO:teuthology.orchestra.run.smithi060.stderr: 94.3% -- replaced with /var/log/ceph/cluster1-osd.2.log.gz 2024-09-12T14:32:44.277 INFO:teuthology.orchestra.run.smithi060.stderr: 94.2% -- replaced with /var/log/ceph/cluster1-osd.0.log.gz 2024-09-12T14:32:44.376 INFO:teuthology.orchestra.run.smithi060.stderr: 2024-09-12T14:32:44.376 INFO:teuthology.orchestra.run.smithi060.stderr:real 13m15.061s 2024-09-12T14:32:44.376 INFO:teuthology.orchestra.run.smithi060.stderr:user 5m8.209s 2024-09-12T14:32:44.376 INFO:teuthology.orchestra.run.smithi060.stderr:sys 0m50.435s 2024-09-12T14:32:44.377 INFO:tasks.ceph:Archiving logs... 2024-09-12T14:32:44.378 DEBUG:teuthology.misc:Transferring archived files from smithi060:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-11_21:48:03-krbd-squid-testing-default-smithi/7901765/remote/smithi060/log 2024-09-12T14:32:44.379 DEBUG:teuthology.orchestra.run.smithi060:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-12T14:33:23.813 DEBUG:teuthology.misc:Transferring archived files from smithi130:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-11_21:48:03-krbd-squid-testing-default-smithi/7901765/remote/smithi130/log 2024-09-12T14:33:23.815 DEBUG:teuthology.orchestra.run.smithi130:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-12T14:33:45.257 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-12T14:33:45.301 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-12T14:33:45.301 DEBUG:teuthology.orchestra.run.smithi060:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-12T14:33:45.306 DEBUG:teuthology.orchestra.run.smithi130:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-12T14:33:45.427 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-12T14:33:45.438 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-12T14:33:45.438 DEBUG:teuthology.orchestra.run.smithi060:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-12T14:33:45.443 DEBUG:teuthology.orchestra.run.smithi130:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-12T14:33:45.474 INFO:teuthology.orchestra.run.smithi060.stderr:bash: line 1: ntpq: command not found 2024-09-12T14:33:45.477 INFO:teuthology.orchestra.run.smithi130.stderr:bash: line 1: ntpq: command not found 2024-09-12T14:33:45.583 INFO:teuthology.orchestra.run.smithi060.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-12T14:33:45.583 INFO:teuthology.orchestra.run.smithi060.stdout:=============================================================================== 2024-09-12T14:33:45.583 INFO:teuthology.orchestra.run.smithi060.stdout:^* hv01.front.sepia.ceph.com 3 8 377 8 -1314us[-1307us] +/- 47ms 2024-09-12T14:33:45.583 INFO:teuthology.orchestra.run.smithi060.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 6 +669us[ +669us] +/- 59ms 2024-09-12T14:33:45.583 INFO:teuthology.orchestra.run.smithi060.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 12 +1135us[+1142us] +/- 74ms 2024-09-12T14:33:45.584 INFO:teuthology.orchestra.run.smithi060.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T14:33:45.589 INFO:teuthology.orchestra.run.smithi130.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-12T14:33:45.589 INFO:teuthology.orchestra.run.smithi130.stdout:=============================================================================== 2024-09-12T14:33:45.589 INFO:teuthology.orchestra.run.smithi130.stdout:^* hv01.front.sepia.ceph.com 3 6 377 6 -1282us[-1256us] +/- 47ms 2024-09-12T14:33:45.589 INFO:teuthology.orchestra.run.smithi130.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 3 +733us[ +733us] +/- 59ms 2024-09-12T14:33:45.590 INFO:teuthology.orchestra.run.smithi130.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 72 +1163us[+1212us] +/- 73ms 2024-09-12T14:33:45.590 INFO:teuthology.orchestra.run.smithi130.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T14:33:45.591 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-12T14:33:45.601 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-12T14:33:45.602 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-12T14:33:45.615 DEBUG:teuthology.orchestra.run.smithi060:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-12T14:33:45.784 DEBUG:teuthology.orchestra.run.smithi130:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-12T14:33:46.038 DEBUG:teuthology.orchestra.run.smithi060:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-12T14:33:46.098 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T14:33:46.099 DEBUG:teuthology.orchestra.run.smithi130:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-12T14:33:46.161 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T14:33:46.161 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-12T14:33:46.171 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-12T14:33:46.184 INFO:teuthology.task.internal:Duration was 6320.822366 seconds 2024-09-12T14:33:46.185 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-12T14:33:46.198 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-12T14:33:46.199 DEBUG:teuthology.orchestra.run.smithi060:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-12T14:33:46.203 DEBUG:teuthology.orchestra.run.smithi130:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-12T14:33:46.358 INFO:teuthology.orchestra.run.smithi060.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-12T14:33:46.383 INFO:teuthology.orchestra.run.smithi130.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-12T14:33:46.594 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-12T14:33:46.595 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi060.front.sepia.ceph.com 2024-09-12T14:33:46.595 DEBUG:teuthology.orchestra.run.smithi060:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-12T14:33:46.650 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi130.front.sepia.ceph.com 2024-09-12T14:33:46.650 DEBUG:teuthology.orchestra.run.smithi130:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-12T14:33:46.699 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-12T14:33:46.699 DEBUG:teuthology.orchestra.run.smithi060:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-12T14:33:46.703 DEBUG:teuthology.orchestra.run.smithi130:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-12T14:33:46.762 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-12T14:33:46.762 DEBUG:teuthology.orchestra.run.smithi060:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-12T14:33:46.766 DEBUG:teuthology.orchestra.run.smithi130:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-12T14:33:46.984 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-12T14:33:46.996 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-12T14:33:46.996 DEBUG:teuthology.orchestra.run.smithi060:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-12T14:33:47.079 DEBUG:teuthology.orchestra.run.smithi130:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-12T14:33:47.136 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-12T14:33:47.346 DEBUG:teuthology.orchestra.run.smithi060:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-12T14:33:47.350 DEBUG:teuthology.orchestra.run.smithi130:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-12T14:33:47.430 INFO:teuthology.orchestra.run.smithi130.stdout:kernel.core_pattern = core 2024-09-12T14:33:47.468 INFO:teuthology.orchestra.run.smithi060.stdout:kernel.core_pattern = core 2024-09-12T14:33:47.499 DEBUG:teuthology.orchestra.run.smithi060:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-12T14:33:47.531 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T14:33:47.532 DEBUG:teuthology.orchestra.run.smithi130:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-12T14:33:47.565 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T14:33:47.565 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-12T14:33:47.656 INFO:teuthology.task.internal:Transferring archived files... 2024-09-12T14:33:47.657 DEBUG:teuthology.misc:Transferring archived files from smithi060:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-11_21:48:03-krbd-squid-testing-default-smithi/7901765/remote/smithi060 2024-09-12T14:33:47.657 DEBUG:teuthology.orchestra.run.smithi060:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-12T14:33:47.722 DEBUG:teuthology.misc:Transferring archived files from smithi130:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-11_21:48:03-krbd-squid-testing-default-smithi/7901765/remote/smithi130 2024-09-12T14:33:47.722 DEBUG:teuthology.orchestra.run.smithi130:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-12T14:33:47.787 INFO:teuthology.task.internal:Removing archive directory... 2024-09-12T14:33:47.787 DEBUG:teuthology.orchestra.run.smithi060:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-12T14:33:47.791 DEBUG:teuthology.orchestra.run.smithi130:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-12T14:33:47.826 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-12T14:33:47.838 INFO:teuthology.task.internal:Not uploading archives. 2024-09-12T14:33:47.838 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-12T14:33:47.851 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-12T14:33:47.852 DEBUG:teuthology.orchestra.run.smithi060:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-12T14:33:47.867 DEBUG:teuthology.orchestra.run.smithi130:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-12T14:33:47.897 INFO:teuthology.orchestra.run.smithi060.stdout: 393233 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 12 14:33 /home/ubuntu/cephtest 2024-09-12T14:33:47.902 INFO:teuthology.orchestra.run.smithi130.stdout: 393233 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 12 14:33 /home/ubuntu/cephtest 2024-09-12T14:33:47.904 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-12T14:33:47.914 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-12T14:33:47.946 INFO:teuthology.run:Summary data: cluster1.mon.a-kernel-sha1: 2ca44a2f35ca8f36983aaaa46d49312239ef519a cluster2.mon.a-kernel-sha1: 2ca44a2f35ca8f36983aaaa46d49312239ef519a description: krbd/mirror/{bluestore-bitmap clusters/2-node conf install/ceph ms_mode$/{legacy-rxbounce} tasks/compare-mirror-images} duration: 6320.822365522385 flavor: default owner: scheduled_teuthology@teuthology success: true 2024-09-12T14:33:47.946 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-12T14:33:48.027 INFO:teuthology.run:pass